Chapter 1. Using Tekton Results for OpenShift Pipelines observability
Tekton Results is a service that archives the complete information for every pipeline run and task run. You can prune the PipelineRun
and TaskRun
resources as necessary and use the Tekton Results API or the opc
command line utility to access their YAML manifests as well as logging information.
Tekton Results is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
1.1. Tekton Results concepts Copy linkLink copied to clipboard!
Tekton Results archives pipeline runs and task runs in the form of results and records.
For every PipelineRun
and TaskRun
custom resource (CR) that completes running, Tekton Results creates a record.
A result can contain one or several records. A record is always a part of exactly one result.
A result corresponds to a pipeline run, and includes the records for the PipelineRun
CR itself and for all the TaskRun
CRs that were started as a part of the pipeline run.
If a task run was started directly, without the use of a pipeline run, a result is created for this task run. This result contains the record for the same task run.
Each result has a name that includes the namespace in which the PipelineRun
or TaskRun
CR was created and the UUID of the CR. The format for the result name is <namespace_name>/results/<parent_run_uuid>
. In this format, <parent_run_uuid>
is the UUUD of a pipeline run or else of a task run that was started directly.
Example result name
results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed
results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed
Each record has a name that includes name of the result that contains the record, as well as the UUID of the PipelineRun
or TaskRun
CR to which the record corresponds. The format for the result name is <namespace_name>/results/<parent_run_uuid>/results/<run_uuid>
.
Example record name
results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e9c736db-5665-441f-922f-7c1d65c9d621
results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e9c736db-5665-441f-922f-7c1d65c9d621
The record includes the full YAML manifest of the TaskRun
or PipelineRun
CR as it existed after the completion of the run. This manifest contains the specification of the run, any annotation specified for the run, as well as certain information about the results of the run, such as the time when it was completed and whether the run was successful.
While the TaskRun
or PipelineRun
CR exists, you can view the YAML manifest by using the following command:
oc get pipelinerun <cr_name> -o yaml
$ oc get pipelinerun <cr_name> -o yaml
Tekton Results preserves this manifest after the TaskRun
or PipelineRun
CR is deleted and makes it available for viewing and searching.
Example YAML manifest of a pipeline run after its completion
Tekton Results also creates a log record that contains the logging information of all the tools that ran as a part of a pipeline run or task run.
You can access every result and record by its name. You can also use Common Expression Language (CEL) queries to search for results and records by the information they contain, including the YAML manifest.
1.2. Preparing to install Tekton Results Copy linkLink copied to clipboard!
You must complete several preparatory steps before installing Tekton Results.
1.2.1. Preparing a secret with an SSL certificate Copy linkLink copied to clipboard!
Tekton Results provides a REST API using the HTTPS protocol, which requires an SSL certificate. Provide a secret with this certificate. If you have an existing certificate provided by a certificate authority (CA), use this certificate, otherwise create a self-signed certificate.
Prerequisites
-
The
openssl
command-line utility is installed.
Procedure
If you do not have a certificate provided by a CA, create a self-signed certificate by entering the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
tekton-results-api-service.openshift-pipelines.svc.cluster.local
with the route endpoint that you plan to use for the Tekton Results API.Create a transport security layer (TLS) secret from the certificate by entering the following command:
oc create secret tls -n openshift-pipelines tekton-results-tls --cert=cert.pem --key=key.pem
$ oc create secret tls -n openshift-pipelines tekton-results-tls --cert=cert.pem --key=key.pem
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you want to use an existing certificate provided by a CA, replace
cert.pem
with the name of the file containing this certificate.
1.2.2. Preparing a secret with the database credentials Copy linkLink copied to clipboard!
Tekton Results uses a PostgreSQL database to store data. You can configure the installation to use either a PostgreSQL server that is automatically installed with Tekton Results or an external PostgreSQL server that already exists in your deployment. In both cases, provide a secret with the database credentials.
Procedure
Complete one of the following steps:
If you do not need to use an external PostgreSQL server, create a secret with the database user named
result
and a random password in theopenshift-pipelines
namespace by entering the following command:oc create secret generic tekton-results-postgres \ --namespace=openshift-pipelines \ --from-literal=POSTGRES_USER=result \ --from-literal=POSTGRES_PASSWORD=$(openssl rand -base64 20)
$ oc create secret generic tekton-results-postgres \ --namespace=openshift-pipelines \ --from-literal=POSTGRES_USER=result \ --from-literal=POSTGRES_PASSWORD=$(openssl rand -base64 20)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteIn this command and in subsequent commads, if you configured a custom target namespace for OpenShift Pipelines, use the name of this namespace instead of
openshift-pipelines
.If you want to use an external PostgreSQL database server to store Tekton Results data, create a secret with the credentials for this server by entering the following command:
oc create secret generic tekton-results-postgres \ --namespace=openshift-pipelines \ --from-literal=POSTGRES_USER=<user> \ --from-literal=POSTGRES_PASSWORD=<password>
$ oc create secret generic tekton-results-postgres \ --namespace=openshift-pipelines \ --from-literal=POSTGRES_USER=<user> \
1 --from-literal=POSTGRES_PASSWORD=<password>
2 Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<user>
with the username for the PostgreSQL user that Tekton Results must use. Replace<password>
with the password for the same account.
1.2.3. Preparing storage for logging information Copy linkLink copied to clipboard!
Tekton Results uses separate storage for logging information related to pipeline runs and task runs. You can configure any one of the following types of storage:
- Persistent volume claim (PVC) on your Red Hat OpenShift Pipelines cluster
- Google Cloud Storage
- S3 bucket storage
Procedure
Complete one of the following procedures:
To use a PVC, complete the following steps:
Create a file named
pvc.yaml
with the following definition for the PVC:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Apply the definition by entering the following command:
oc apply -n openshift-pipelines -f pvc.yaml
$ oc apply -n openshift-pipelines -f pvc.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
To use Google Cloud Storage, complete the following steps:
-
Create an application credentials file by using the
gcloud
command. For instructions about providing application credentials in a file, see User credentials provided by using the gcloud CLI in the Google Cloud documentation. Create a secret from the application credentials file by entering the following command:
oc create secret generic gcs-credentials \ --from-file=$HOME/.config/gcloud/application_default_credentials.json \ -n openshift-pipelines
$ oc create secret generic gcs-credentials \ --from-file=$HOME/.config/gcloud/application_default_credentials.json \ -n openshift-pipelines
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Adjust the path and filename of the application credentials file as necessary.
-
Create an application credentials file by using the
To use S3 bucket storage, complete the following steps:
Create a file named
s3_secret.yaml
with the following content:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a secret from the file by entering the following command:
oc create secret generic s3-credentials \ --from-file=s3_secret.yaml -n openshift-pipelines
$ oc create secret generic s3-credentials \ --from-file=s3_secret.yaml -n openshift-pipelines
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
1.3. Installing Tekton Results Copy linkLink copied to clipboard!
To install Tekton Results, you must provide the required resources and then create and apply a TektonResult
custom resource (CR). The OpenShift Pipelines Operator installs the Results services when you apply the TektonResult
custom resource.
Prerequisites
- You installed OpenShift Pipelines using the Operator.
- You prepared a secret with the SSL certificate.
- You prepared storage for the logging information.
- You prepared a secret with the database credentials.
Procedure
Create the resource definition file named
result.yaml
based on the following example. You can adjust the settings as necessary.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Add configuration for the storage for logging information to this file:
If you configured a persistent volume claim (PVC), add the following line to provide the name of the PVC:
logging_pvc_name: tekton-logs
logging_pvc_name: tekton-logs
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you configured Google Cloud Storage, add the following lines to provide the secret name, the credentials file name, and the name of the Google Cloud Storage bucket:
gcs_creds_secret_name: gcs-credentials gcs_creds_secret_key: application_default_credentials.json gcs_bucket_name: bucket-name
gcs_creds_secret_name: gcs-credentials gcs_creds_secret_key: application_default_credentials.json
1 gcs_bucket_name: bucket-name
2 Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you configured S3 bucket storage, add the following line to provide the name of the S3 secret:
secret_name: s3-credentials
secret_name: s3-credentials
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Optional: If you want to use an external PostgreSQL database server to store Tekton Results information, add the following lines to the file:
db_host: postgres.internal.example.com db_port: 5432 is_external_db: true
db_host: postgres.internal.example.com
1 db_port: 5432
2 is_external_db: true
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Apply the resource definition by entering the following command:
oc apply -n openshift-pipelines -f result.yaml
$ oc apply -n openshift-pipelines -f result.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Expose the route for the Tekton Results service API by entering the following command:
oc create route -n openshift-pipelines \ passthrough tekton-results-api-service \ --service=tekton-results-api-service --port=8080
$ oc create route -n openshift-pipelines \ passthrough tekton-results-api-service \ --service=tekton-results-api-service --port=8080
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
1.4. Querying Tekton Results using the opc command line utility Copy linkLink copied to clipboard!
You can use the opc
command line utility to query Tekton Results for results and records. To install the opc
command line utility, install the package for the tkn
command line utility. For instructions about installing this package, see Installing tkn.
You can use the names of records and results to retrieve the data in them.
You can search for results and records using Common Expression Language (CEL) queries. These searches display the UUIDs of the results or records. You can use the provided examples to create queries for common search types. You can also use reference information to create other queries.
1.4.1. Preparing the opc utility environment for querying Tekton Results Copy linkLink copied to clipboard!
Before you can query Tekton Results, you must prepare the environment for the opc
utility.
Prerequisites
- You installed Tekton Results.
-
You installed the
opc
utility.
Procedure
Set the
RESULTS_API
environment variable to the route to the Tekton Results API by entering the following command:export RESULTS_API=$(oc get route tekton-results-api-service -n openshift-pipelines --no-headers -o custom-columns=":spec.host"):443
$ export RESULTS_API=$(oc get route tekton-results-api-service -n openshift-pipelines --no-headers -o custom-columns=":spec.host"):443
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an authentication token for the Tekton Results API by entering the following command:
oc create token <service_account>
$ oc create token <service_account>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Save the string that this command outputs.
Optional: Create the
~/.config/tkn/results.yaml
file for automatic authentication with the Tekton Results API. The file must have the following contents:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- The route to the Tekton Results API. Use the same value as you set for
RESULTS_API
. - 2
- The authentication token that was created by the
oc create token
command. If you provide this token, it overrides theservice_account
setting andopc
uses this token to authenticate. - 3
- The location of the file with the SSL certificate that you configured for the API endpoint.
- 4
- If you configured a custom target namespace for OpenShift Pipelines, replace
openshift-pipelines
with the name of this namespace. - 5 6
- The name of a service account for authenticating with the Tekton Results API. If you provided the authentication token, you do not need to provide the
service_account
parameters.
Alternatively, if you do not create the
~/.config/tkn/results.yaml
file, you can pass the token to eachopc
command by using the--authtoken
option.
1.4.2. Querying for results and records by name Copy linkLink copied to clipboard!
You can list and query results and records using their names.
Prerequisites
- You installed Tekton Results.
-
You installed the
opc
utility and prepared its environment to query Tekton Results. -
You installed the
jq
package.
Procedure
List the names of all results that correspond to pipeline runs and task runs created in a namespace. Enter the following command:
opc results list --addr ${RESULTS_API} <namespace_name>
$ opc results list --addr ${RESULTS_API} <namespace_name>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example command
opc results list --addr ${RESULTS_API} results-testing
$ opc results list --addr ${RESULTS_API} results-testing
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Name Start Update results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed 2023-06-29 02:49:53 +0530 IST 2023-06-29 02:50:05 +0530 IST results-testing/results/ad7eb937-90cc-4510-8380-defe51ad793f 2023-06-29 02:49:38 +0530 IST 2023-06-29 02:50:06 +0530 IST results-testing/results/d064ce6e-d851-4b4e-8db4-7605a23671e4 2023-06-29 02:49:45 +0530 IST 2023-06-29 02:49:56 +0530 IST
Name Start Update results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed 2023-06-29 02:49:53 +0530 IST 2023-06-29 02:50:05 +0530 IST results-testing/results/ad7eb937-90cc-4510-8380-defe51ad793f 2023-06-29 02:49:38 +0530 IST 2023-06-29 02:50:06 +0530 IST results-testing/results/d064ce6e-d851-4b4e-8db4-7605a23671e4 2023-06-29 02:49:45 +0530 IST 2023-06-29 02:49:56 +0530 IST
Copy to Clipboard Copied! Toggle word wrap Toggle overflow List the names of all records in a result by entering the following command:
opc results records list --addr ${RESULTS_API} <result_name>
$ opc results records list --addr ${RESULTS_API} <result_name>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example command
opc results records list --addr ${RESULTS_API} results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed
$ opc results records list --addr ${RESULTS_API} results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieve the YAML manifest for a pipeline run or task run from a record by entering the following command:
opc results records get --addr ${RESULTS_API} <record_name> \ | jq -r .data.value | base64 -d | \ xargs -0 python3 -c 'import sys, yaml, json; j=json.loads(sys.argv[1]); print(yaml.safe_dump(j))'
$ opc results records get --addr ${RESULTS_API} <record_name> \ | jq -r .data.value | base64 -d | \ xargs -0 python3 -c 'import sys, yaml, json; j=json.loads(sys.argv[1]); print(yaml.safe_dump(j))'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example command
opc results records get --addr ${RESULTS_API} \ results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e9c736db-5665-441f-922f-7c1d65c9d621 | \ jq -r .data.value | base64 -d | \ xargs -0 python3 -c 'import sys, yaml, json; j=json.loads(sys.argv[1]); print(yaml.safe_dump(j))'
$ opc results records get --addr ${RESULTS_API} \ results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/records/e9c736db-5665-441f-922f-7c1d65c9d621 | \ jq -r .data.value | base64 -d | \ xargs -0 python3 -c 'import sys, yaml, json; j=json.loads(sys.argv[1]); print(yaml.safe_dump(j))'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Optional: Retrieve the logging information for a task run from a record using the log record name. To get the log record name, replace
records
withlogs
in the record name. Enter the following command:opc results logs get --addr ${RESULTS_API} <log_record_name> | jq -r .data | base64 -d
$ opc results logs get --addr ${RESULTS_API} <log_record_name> | jq -r .data | base64 -d
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example command
opc results logs get --addr ${RESULTS_API} \ results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/logs/e9c736db-5665-441f-922f-7c1d65c9d621 | \ jq -r .data | base64 -d
$ opc results logs get --addr ${RESULTS_API} \ results-testing/results/04e2fbf2-8653-405f-bc42-a262bcf02bed/logs/e9c736db-5665-441f-922f-7c1d65c9d621 | \ jq -r .data | base64 -d
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
1.4.3. Searching for results Copy linkLink copied to clipboard!
You can search for results using Common Expression Language (CEL) queries. For example, you can find results for pipeline runs that did not succeed. However, most of the relevant information is not contained in result objects; to search by the names, completion times, and other data, search for records.
Prerequisites
- You installed Tekton Results.
-
You installed the
opc
utility and prepared its environment to query Tekton Results.
Procedure
Search for results using a CEL query by entering the following command:
opc results list --addr ${RESULTS_API} --filter="<cel_query>" <namespace-name>
$ opc results list --addr ${RESULTS_API} --filter="<cel_query>" <namespace-name>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Replace <namespace_name>
with the namespace in which the pipeline runs or task runs were created.
Purpose | CEL query |
---|---|
The results of all runs that failed |
|
The results all pipeline runs that contained the annotations |
|
1.4.4. Searching for records Copy linkLink copied to clipboard!
You can search for records using Common Expression Language (CEL) queries. As each record contains full YAML information for a pipeline run or task run, you can find records by many different criteria.
Prerequisites
- You installed Tekton Results.
-
You installed the
opc
utility and prepared its environment to query Tekton Results.
Procedure
Search for records using a CEL query by entering the following command:
opc results records list --addr ${RESULTS_API} --filter="<cel_query>" <namespace_name>/result/-
$ opc results records list --addr ${RESULTS_API} --filter="<cel_query>" <namespace_name>/result/-
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<namespace_name>
with the namespace in which the pipeline runs or task runs were created. Alternatively, search for records within a single result by entering the following command:opc results records list --addr ${RESULTS_API} --filter="<cel_query>" <result_name>
$ opc results records list --addr ${RESULTS_API} --filter="<cel_query>" <result_name>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<result_name>
with the full name of the result.
Purpose | CEL query |
---|---|
Records of all task runs or pipeline runs that failed |
|
Records where the name of the |
|
Records for all task runs that were started by the |
|
Records of all pipeline runs and task runs that were created from a |
|
Records of all pipeline runs that were created from a |
|
Records of all task runs where the |
|
Records of all pipeline runs that took more than five minutes to complete |
|
Records of all pipeline runs and task runs that completed on October 7, 2023 |
|
Records of all pipeline runs that included three or more tasks |
|
Records of all pipeline runs that had annotations containing |
|
Records of all pipeline runs that had annotations containing |
|
1.4.5. Reference information for searching results Copy linkLink copied to clipboard!
You can use the following fields in Common Expression Language (CEL) queries for results:
CEL field | Description |
---|---|
|
The namespace in which the |
| Unique identifier for the result. |
|
Annotations added to the |
| The summary of the result. |
| The creation time of the result. |
| The last update time of the result. |
You can use the summary.status
field to determine whether the pipeline run was successful. This field can have the following values:
-
UNKNOWN
-
SUCCESS
-
FAILURE
-
TIMEOUT
-
CANCELLED
Do not use quote characters such as "
or '
to provide the value for this field.
1.4.6. Reference information for searching records Copy linkLink copied to clipboard!
You can use the following fields in Common Expression Language (CEL) queries for records:
CEL field | Description | Values |
---|---|---|
| Record name | |
| Record type identifier |
|
| The YAML data for the task run or pipeline run. In log records, this field contains the logging output. |
Because the data
field contains the entire YAML data for the task run or pipeline run, you can use all elements of this data in your CEL query. For example, data.status.completionTime
contains the completion time of the task run or pipeline run.