Chapter 1. Using Tekton Results for OpenShift Pipelines observability

1. On your OpenShift Container Platform cluster, install LokiStack by using the Loki Operator and also install the OpenShift Logging Operator.

2. Create a ClusterLogForwarder.yaml manifest file for the ClusterLogForwarder custom resource (CR) with one of the following YAML manifests, depending on whether you installed OpenShift Logging version 6 or version 5:

   YAML manifest for the ClusterLogForwarder CR if you installed OpenShift Logging version 6

   apiVersion: observability.openshift.io/v1
   kind: ClusterLogForwarder
   metadata:
     name: collector
     namespace: openshift-logging
   spec:
     inputs:
     - application:
         selector:
           matchExpressions:
             - key: app.kubernetes.io/managed-by
               operator: In
               values: ["tekton-pipelines", "pipelinesascode.tekton.dev"]
       name: only-tekton
       type: application
     managementState: Managed
     outputs:
     - lokiStack:
         labelKeys:
           application:
             ignoreGlobal: true
             labelKeys:
             - log_type
             - kubernetes.namespace_name
             - openshift_cluster_id
         authentication:
           token:
             from: serviceAccount
         target:
           name: logging-loki
           namespace: openshift-logging
       name: default-lokistack
       tls:
         ca:
           configMapName: openshift-service-ca.crt
           key: service-ca.crt
       type: lokiStack
     pipelines:
     - inputRefs:
       - only-tekton
       name: default-logstore
       outputRefs:
       - default-lokistack
     serviceAccount:
       name: collector

   YAML manifest for the ClusterLogForwarder CR if you installed OpenShift Logging version 5

   apiVersion: "logging.openshift.io/v1"
   kind: ClusterLogForwarder
   metadata:
     name: instance
     namespace: openshift-logging
   spec:
     inputs:
     - name: only-tekton
       application:
         selector:
           matchLabels:
             app.kubernetes.io/managed-by: tekton-pipelines
     pipelines:
       - name: enable-default-log-store
         inputRefs: [ only-tekton ]
         outputRefs: [ default ]

3. Create the ClusterLogForwarder CR in the openshift-logging namespace by entering the following command:

   $ oc apply -n openshift-logging ClusterLogForwarder.yaml

4. Edit the TektonConfig custom resource (CR) by using the following command:

   $ oc edit TektonConfig config

   Make the following changes in the result spec:

   apiVersion: operator.tekton.dev/v1alpha1
   kind: TektonConfig
   metadata:
     name: config
   spec:
     result:
       loki_stack_name: logging-loki 1
       loki_stack_namespace: openshift-logging 2

   1

   The name of the LokiStack CR, typically logging-loki.

   2

   The name of the namespace where LokiStack is deployed, typically openshift-logging.

Procedure

1. Create a secret with the credentials for connecting to your PostgreSQL server by entering the following command:

   $ oc create secret generic tekton-results-postgres \
     --namespace=openshift-pipelines \
     --from-literal=POSTGRES_USER=<user> \
     --from-literal=POSTGRES_PASSWORD=<password>

2. Edit the TektonConfig custom resource (CR) by using the following command:

   $ oc edit TektonConfig config

   Make the following changes in the result spec:

   apiVersion: operator.tekton.dev/v1alpha1
   kind: TektonConfig
   metadata:
     name: config
   spec:
     result:
       is_external_db: true
       db_host: database.example.com 1
       db_port: 5342 2

   1

   Provide the host name of your PostgreSQL server.

   2

   Provide the port number of your PostgreSQL server.

Procedure

1. Set the RESULTS_API environment variable to the route to the Tekton Results API by entering the following command:

   $ export RESULTS_API=$(oc get route tekton-results-api-service -n openshift-pipelines --no-headers -o custom-columns=":spec.host"):443

2. Create an authentication token for the Tekton Results API by entering the following command:

   $ oc create token <service_account>

   Save the string that this command outputs.

3. Optional: Create the ~/.config/tkn/results.yaml file for automatic authentication with the Tekton Results API. The file must have the following contents:

   address: <tekton_results_route> 1
   token: <authentication_token> 2
   ssl:
      roots_file_path: /home/example/cert.pem 3
      server_name_override: tekton-results-api-service.openshift-pipelines.svc.cluster.local 4
    service_account:
      namespace: service_acc_1 5
      name: service_acc_1 6

   1

   The route to the Tekton Results API. Use the same value as you set for RESULTS_API.

   2

   The authentication token that was created by the oc create token command. If you provide this token, it overrides the service_account setting and opc uses this token to authenticate.

   3

   The location of the file with the SSL certificate that you configured for the API endpoint.

   4

   If you configured a custom target namespace for OpenShift Pipelines, replace openshift-pipelines with the name of this namespace.

   5 6

   The name of a service account for authenticating with the Tekton Results API. If you provided the authentication token, you do not need to provide the service_account parameters.

   Alternatively, if you do not create the ~/.config/tkn/results.yaml file, you can pass the token to each opc command by using the --authtoken option.

Procedure

1. List the names of all results that correspond to pipeline runs and task runs created in a namespace. Enter the following command:

   $ opc results list --addr ${RESULTS_API} <namespace_name>

   Example command

   $ opc results list --addr ${RESULTS_API} results-testing

   Example output

   Name                                                          Start                                   Update
   results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed  2023-06-29 02:49:53 +0530 IST           2023-06-29 02:50:05 +0530 IST
   results-testing/results/ad7eb937-90cc-4510-8380-defe51ad793f  2023-06-29 02:49:38 +0530 IST           2023-06-29 02:50:06 +0530 IST
   results-testing/results/d064ce6e-d851-4b4e-8db4-7605a23671e4  2023-06-29 02:49:45 +0530 IST           2023-06-29 02:49:56 +0530 IST

2. List the names of all records in a result by entering the following command:

   $ opc results records list --addr ${RESULTS_API} <result_name>

   Example command

   $ opc results records list --addr ${RESULTS_API} results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed

   Example output

   Name                                                                                                   Type                                    Start                                   Update
   results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e9c736db-5665-441f-922f-7c1d65c9d621  tekton.dev/v1.TaskRun              2023-06-29 02:49:53 +0530 IST           2023-06-29 02:49:57 +0530 IST
   results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/5de23a76-a12b-3a72-8a6a-4f15a3110a3e  results.tekton.dev/v1alpha2.Log         2023-06-29 02:49:57 +0530 IST           2023-06-29 02:49:57 +0530 IST
   results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/57ce92f9-9bf8-3a0a-aefb-dc20c3e2862d  results.tekton.dev/v1alpha2.Log         2023-06-29 02:50:05 +0530 IST           2023-06-29 02:50:05 +0530 IST
   results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e9a0c21a-f826-42ab-a9d7-a03bcefed4fd  tekton.dev/v1.TaskRun              2023-06-29 02:49:57 +0530 IST           2023-06-29 02:50:05 +0530 IST
   results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/04e2fbf2-8653-405f-bc42-a262bcf02bed  tekton.dev/v1.PipelineRun          2023-06-29 02:49:53 +0530 IST           2023-06-29 02:50:05 +0530 IST
   results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e6eea2f9-ec80-388c-9982-74a018a548e4  results.tekton.dev/v1alpha2.Log         2023-06-29 02:50:05 +0530 IST           2023-06-29 02:50:05 +0530 IST

3. Retrieve the YAML manifest for a pipeline run or task run from a record by entering the following command:

   $ opc results records get --addr ${RESULTS_API} <record_name> \
     | jq -r .data.value | base64 -d | \
     xargs -0 python3 -c 'import sys, yaml, json; j=json.loads(sys.argv[1]); print(yaml.safe_dump(j))'

   Example command

   $ opc results records get --addr ${RESULTS_API} \
     results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e9c736db-5665-441f-922f-7c1d65c9d621 | \
     jq -r .data.value | base64 -d | \
     xargs -0 python3 -c 'import sys, yaml, json; j=json.loads(sys.argv[1]); print(yaml.safe_dump(j))'

4. Optional: Retrieve the logging information for a task run from a record using the log record name. To get the log record name, replace records with logs in the record name. Enter the following command:

   $ opc results logs get --addr ${RESULTS_API} <log_record_name> | jq -r .data | base64 -d

   Example command

   $ opc results logs get --addr ${RESULTS_API} \
     results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/logs/e9c736db-5665-441f-922f-7c1d65c9d621 | \
     jq -r .data | base64 -d