Securing OpenShift Pipelines

Procedure

1. Create a secret named mongo-url with the MONGO_SERVER_URL file that has the Mongo server URL value by entering the following command:

   $ oc create secret generic mongo-url -n tekton-chains \
     --from-file=MONGO_SERVER_URL=<path>/MONGO_SERVER_URL

   <path>

   The full path and name of the MONGO_SERVER_URL file that has the Mongo server URL value.

2. In the TektonConfig custom resource (CR), in the chain section, configure mounting the secret on the Tekton Chains controller and set the storage.docdb.mongo-server-url-dir parameter to the directory where you mount the secret, as shown in the following example:

   Example configuration for mounting the mongo-url secret

   apiVersion: operator.tekton.dev/v1
   kind: TektonConfig
   metadata:
     name: config
   spec:
   # ...
     chain:
       disabled: false
       storage.docdb.mongo-server-url-dir: /tmp/mongo-url
       options:
         deployments:
           tekton-chains-controller:
             spec:
               template:
                 spec:
                   containers:
                   - name: tekton-chains-controller
                     volumeMounts:
                     - mountPath: /tmp/mongo-url
                       name: mongo-url
                   volumes:
                   -  name: mongo-url
                      secret:
                       secretName: mongo-url
   # ...

Procedure

1. Create a secret named kms-secrets with the KMS_AUTH_TOKEN file that has the authentication token for the KMS server by entering the following command:

   $ oc create secret generic kms-secrets -n tekton-chains \
     --from-file=KMS_AUTH_TOKEN=<path_and_name>

   <path_and_name>

   The full path and name of the file that has the authentication token for the KMS server, for example, /home/user/KMS_AUTH_TOKEN. You can use another file name instead of KMS_AUTH_TOKEN.

2. In the TektonConfig custom resource (CR), in the chain section, configure mounting the secret on the Tekton Chains controller and set the signers.kms.auth.token-path parameter to the full pathname of the authentication token file, as shown in the following example:

   Example configuration for mounting the kms-secrets secret

   apiVersion: operator.tekton.dev/v1
   kind: TektonConfig
   metadata:
     name: config
   spec:
   # ...
     chain:
       disabled: false
       signers.kms.auth.token-path: /etc/kms-secrets/KMS_AUTH_TOKEN
       options:
         deployments:
           tekton-chains-controller:
             spec:
               template:
                 spec:
                   containers:
                   - name: tekton-chains-controller
                     volumeMounts:
                     - mountPath: /etc/kms-secrets
                       name: kms-secrets
                   volumes:
                   -  name: kms-secrets
                      secret:
                       secretName: kms-secrets
   # ...

Procedure

1. Edit the TektonConfig CR by running the following command:

   $ oc edit TektonConfig config

2. In the TektonConfig CR, set the generateSigningSecret value to true:

   Example of creating an ECDSA cosign key pair by using the TektonConfig CR

   apiVersion: operator.tekton.dev/v1
   kind: TektonConfig
   metadata:
     name: config
   spec:
   # ...
     chain:
       disabled: false
       generateSigningSecret: true
   # ...

   generateSigningSecret

   The default value is false. Setting the value to true generates the ecdsa key pair.

3. After a few minutes, extract the public key from the secret and store it, so that you can use it to verify artifact attestations. Run the following command to extract the key:

   $ oc extract -n openshift-pipelines secret/signing-secrets --keys=cosign.pub

Procedure

1. Generate the cosign.key and cosign.pub key pairs by running the following command:

   $ cosign generate-key-pair k8s://openshift-pipelines/signing-secrets

   Cosign prompts you for a password and then creates a Kubernetes secret.

2. Store the encrypted cosign.key private key and the cosign.password decryption password in the signing-secrets Kubernetes secret. Ensure that you store the private key as an encrypted Privacy Enhanced Mail (PEM) file of the ENCRYPTED COSIGN PRIVATE KEY type.

Procedure

1. Generate a public/private key pair by running the following command:

   $ skopeo generate-sigstore-key --output-prefix <mykey>

   <mykey>

   Replace <mykey> with a key name of your choice.

   Skopeo prompts you for a passphrase for the private key and then creates the key files named <mykey>.private and <mykey>.pub.

2. Encode the <mykey>.pub file using the base64 tool by running the following command:

   $ base64 -w 0 <mykey>.pub > b64.pub

3. Encode the <mykey>.private file using the base64 tool by running the following command:

   $ base64 -w 0 <mykey>.private > b64.private

4. Encode the passphrase using the base64 tool by running the following command:

   $ echo -n '<passphrase>' | base64 -w 0 > b64.passphrase

   <passphrase>

   Replace <passphrase> with the passphrase that you used for the key pair.

5. Create the signing-secrets secret in the openshift-pipelines namespace by running the following command:

   $ oc create secret generic signing-secrets -n openshift-pipelines

6. Edit the signing-secrets secret by running the following command:

   $ oc edit secret -n openshift-pipelines signing-secrets

   Add the encoded keys in the data of the secret in the following way:

   apiVersion: v1
   data:
     cosign.key: <Encoded <mykey>.private>
     cosign.password: <Encoded passphrase>
     cosign.pub: <Encoded <mykey>.pub>
   immutable: true
   kind: Secret
   metadata:
     name: signing-secrets
   # ...
   type: Opaque

   <mykey>

   Replace <Encoded <mykey>.private> with the content of the b64.private file.

   cosign.password

   Replace <Encoded passphrase> with the content of the b64.passphrase file.

   <mykey>

   Replace <Encoded <mykey>.pub> with the content of the b64.pub file.

Procedure

1. Delete the signing-secret secret by running the following command:

   $ oc delete secret signing-secrets -n openshift-pipelines

2. Re-create the key pairs and store them in the secret using your preferred signing scheme.

Procedure

1. Navigate to the Activity → PipelineRuns tab.
2. For the project whose SBOM you want to view, select its most recent pipeline run.

3. On the PipelineRun details page, select View SBOM.

   1. You can use your web browser to immediately search the SBOM for terms that indicate vulnerabilities in your software supply chain. For example, try searching for log4j.
   2. You can select Download to download the SBOM, or Expand to view it full-screen.

Procedure

1. Open terminal, log in to Developer or Administrator perspective, and then switch to the relevant project.

2. From the OpenShift web console, copy the download sbom command and run it on your terminal.

   Example cosign command

   $ cosign download sbom quay.io/<workspace>/user-workload@sha256

   1. (Optional) To view the full SBOM in a searchable format, run the following command to redirect the output:

      Example cosign command

      $ cosign download sbom quay.io/<workspace>/user-workload@sha256 > sbom.txt

Procedure

1. Create the YAML manifest for the secret in the secret.yaml file. In this manifest, specify the username and password or GitHub personal access token to access the target Git repository.

   apiVersion: v1
   kind: Secret
   metadata:
     name: basic-user-pass
     annotations:
       tekton.dev/git-0: https://github.com
   type: kubernetes.io/basic-auth
   stringData:
     username: <username>
     password: <password>

   name

   Name of the secret. In this example, basic-user-pass.

   <username>

   Username for the Git repository.

   <password>

   Password or personal access token for the Git repository.

2. Create the YAML manifest for the service account in the serviceaccount.yaml file. In this manifest, associate the secret with the service account.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: build-bot
   secrets:
     - name: basic-user-pass

   name

   Name of the service account. In this example, build-bot.

   - name: basic-user-pass

   Name of the secret. In this example, basic-user-pass.

3. Create the YAML manifest for the task run or pipeline run in the run.yaml file and associate the service account with the task run or pipeline run. Use one of the following examples:

   • Associate the service account with a TaskRun resource:

     apiVersion: tekton.dev/v1
     kind: TaskRun
     metadata:
       name: build-push-task-run-2
     spec:
       taskRunTemplate:
         serviceAccountName: build-bot
       taskRef:
         name: build-push

     name

     Name of the task run. In this example, build-push-task-run-2.

     serviceAccountName

     Name of the service account. In this example, build-bot.

     name

     Name of the task. In this example, build-push.

   • Associate the service account with a PipelineRun resource:

     apiVersion: tekton.dev/v1
     kind: PipelineRun
     metadata:
       name: demo-pipeline
       namespace: default
     spec:
       taskRunTemplate:
         serviceAccountName: build-bot
       pipelineRef:
         name: demo-pipeline

     name

     Name of the pipeline run. In this example, demo-pipeline.

     serviceAccountName

     Name of the service account. In this example, build-bot.

     name

     Name of the pipeline. In this example, demo-pipeline.

4. Apply the YAML manifests that you created by entering the following command:

   $ oc apply --filename secret.yaml,serviceaccount.yaml,run.yaml

Procedure

1. Generate an SSH private key, or copy an existing private key, which is usually available in the ~/.ssh/id_rsa file.

2. Create the YAML manifest for the secret in the secret.yaml file. In this manifest, set the value of ssh-privatekey to the content of the SSH private key file, and set the value of known_hosts to the content of the known hosts file.

   apiVersion: v1
   kind: Secret
   metadata:
     name: ssh-key
     annotations:
       tekton.dev/git-0: github.com
   type: kubernetes.io/ssh-auth
   stringData:
     ssh-privatekey:
     known_hosts:

   metadata.name

   Name of the secret containing the SSH private key. In this example, ssh-key.

   ssh-privatekey

   The content of the SSH private key file.

   known_hosts

   The content of the known hosts file.

   Important

   If you omit the known hosts file, OpenShift Pipelines accepts the public key of any server.

3. Optional: Specify a custom SSH port by adding :<port_number> to the end of the annotation value. For example, tekton.dev/git-0: github.com:2222.

4. Create the YAML manifest for the service account in the serviceaccount.yaml file. In this manifest, associate the secret with the service account.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: build-bot
   secrets:
     - name: ssh-key

   metadata.name

   Name of the service account. In this example, build-bot.

   secrets.name

   Name of the secret containing the SSH private key. In this example, ssh-key.

5. In the run.yaml file, associate the service account with a task run or a pipeline run. Use one of the following examples:

   • To associate the service account with a task run:

     apiVersion: tekton.dev/v1
     kind: TaskRun
     metadata:
       name: build-push-task-run-2
     spec:
       taskRunTemplate:
         serviceAccountName: build-bot
       taskRef:
         name: build-push

     metadata.name

     Name of the task run. In this example, build-push-task-run-2.

     serviceAccountName

     Name of the service account. In this example, build-bot.

     taskRef.name

     Name of the task. In this example, build-push.

   • To associate the service account with a pipeline run:

     apiVersion: tekton.dev/v1
     kind: PipelineRun
     metadata:
       name: demo-pipeline
       namespace: default
     spec:
       taskRunTemplate:
         serviceAccountName: build-bot
       pipelineRef:
         name: demo-pipeline

     metadata.name

     Name of the pipeline run. In this example, demo-pipeline.

     serviceAccountName

     Name of the service account. In this example, build-bot.

     pipelineRef.name

     Name of the pipeline. In this example, demo-pipeline.

6. Apply the changes.

   $ oc apply --filename secret.yaml,serviceaccount.yaml,run.yaml

Procedure

1. Create the container registry authentication secret from an existing config.json file, which has the authentication information, by entering the following command:

   $ oc create secret generic my-registry-credentials \
     --from-file=config.json=/home/user/credentials/config.json

   my-registry-credentials

   The name of the secret.

   --from-file

   The path name of the config.json file, in this example, /home/user/credentials/config.json

2. Create the YAML manifest for the service account in the serviceaccount.yaml file. In this manifest, associate the secret with the service account.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: container-bot
   secrets:
     - name: my-registry-credentials

   metadata.name

   Name of the service account. In this example, container-bot.

   secrets.name

   Name of the secret containing the registry credentials. In this example, my-registry-credentials.

3. Create a YAML manifest for a task run or pipeline run as the run.yaml file. In this file, associate the service account with a task run or a pipeline run. Use one of the following examples:

   • To associate the service account with a task run:

     apiVersion: tekton.dev/v1
     kind: TaskRun
     metadata:
       name: build-container-task-run-2
     spec:
       taskRunTemplate:
         serviceAccountName: container-bot
       taskRef:
         name: build-container

     metadata.name

     Name of the task run. In this example, build-container-task-run-2.

     serviceAccountName

     Name of the service account. In this example, container-bot.

     taskRef.name

     Name of the task. In this example, build-container.

   • To associate the service account with a pipeline run:

     apiVersion: tekton.dev/v1
     kind: PipelineRun
     metadata:
       name: demo-pipeline
       namespace: default
     spec:
       taskRunTemplate:
         serviceAccountName: container-bot
       pipelineRef:
         name: demo-pipeline

     metadata.name

     Name of the pipeline run. In this example, demo-pipeline.

     serviceAccountName

     Name of the service account. In this example, container-bot.

     pipelineRef.name

     Name of the pipeline. In this example, demo-pipeline.

4. Apply the changes by entering the following command:

   $ oc apply --filename serviceaccount.yaml,run.yaml

Procedure

1. Create the Git SSH authentication secret from files in an existing .ssh directory by entering the following command:

   $ oc create secret generic my-github-ssh-credentials \
     --from-file=id_ed25519=/home/user/.ssh/id_ed25519 \
     --from-file=known_hosts=/home/user/.ssh/known_hosts

   my-github-ssh-credentials

   The name of the secret.

   --from-file=id_ed25519

   The name and full path name of the private key file, in this example, /home/user/.ssh/id_ed25519

   --from-file=known_hosts

   The name and full path name of the known hosts file, in this example, /home/user/.ssh/known_hosts

2. In your task definition, configure a named workspace for the Git authentication, for example, ssh-directory:

   Example definition of a workspace

   apiVersion: tekton.dev/v1
   kind: Task
   metadata:
     name: git-clone
   spec:
     workspaces:
       - name: ssh-directory
         description: |
           A .ssh directory with private key, known_hosts, config, etc.

3. In the steps of the task, access the directory using the path in the $(workspaces.<workspace_name>.path) environment variable, for example, $(workspaces.ssh-directory.path)

4. When running the task, specify the secret for the named workspace by including the --workspace argument in the tkn task start command:

   $ tkn task start <task_name>
         --workspace name=<workspace_name>,secret=<secret_name>
         # ...

   <secret_name>

   Replace <workspace_name> with the name of the workspace that you configured and <secret_name> with the name of the secret that you created.

Procedure

1. Create the container registry authentication secret from an existing config.json file, which has the authentication information, by entering the following command:

   $ oc create secret generic my-registry-credentials \
     --from-file=config.json=/home/user/credentials/config.json

   my-registry-credentials

   The name of the secret.

   --from-file

   The path name of the config.json file, in this example, /home/user/credentials/config.json

2. In your task definition, configure a named workspace for the Git authentication, for example, ssh-directory:

   Example definition of a workspace

   apiVersion: tekton.dev/v1
   kind: Task
   metadata:
     name: skopeo-copy
   spec:
     workspaces:
       - name: dockerconfig
         description: Includes a docker `config.json`
   # ...

3. In the steps of the task, access the directory by using the path in the $(workspaces.<workspace_name>.path) environment variable, for example, $(workspaces.dockerconfig.path).

4. To run the task, specify the secret for the named workspace by including the --workspace argument in the tkn task start command:

   $ tkn task start <task_name>
         --workspace name=<workspace_name>,secret=<secret_name>
         # ...

   <secret_name>

   Replace <workspace_name> with the name of the workspace that you configured and <secret_name> with the name of the secret that you created.

Procedure

1. Create a copy of the buildah task in the openshift-pipelines namespace; change the name of the copy to buildah-as-user.

   $ oc get task buildah -n openshift-pipelines -o yaml \
     | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' \
     | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f -

2. Edit the copied buildah task.

   $ oc edit task buildah-as-user

   Example: Modified Buildah task with build user

   apiVersion: tekton.dev/v1
   kind: Task
   metadata:
     name: buildah-as-user
   spec:
     description: >-
       Buildah task builds source into a container image and
       then pushes it to a container registry.
       Buildah Task builds source into a container image using Project Atomic's
       Buildah build tool.It uses Buildah's support for building from Dockerfiles,
       using its buildah bud command.This command executes the directives in the
       Dockerfile to assemble a container image, then pushes that image to a
       container registry.
     params:
     - name: IMAGE
       description: Reference of the image buildah will produce.
     - name: BUILDER_IMAGE
       description: The location of the buildah builder image.
       default: registry.redhat.io/rhel8/buildah@sha256:99cae35f40c7ec050fed3765b2b27e0b8bbea2aa2da7c16408e2ca13c60ff8ee
     - name: STORAGE_DRIVER
       description: Set buildah storage driver
       default: vfs
     - name: DOCKERFILE
       description: Path to the Dockerfile to build.
       default: ./Dockerfile
     - name: CONTEXT
       description: Path to the directory to use as context.
       default: .
     - name: TLSVERIFY
       description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)
       default: "true"
     - name: FORMAT
       description: The format of the built container, oci or docker
       default: "oci"
     - name: BUILD_EXTRA_ARGS
       description: Extra parameters passed for the build command when building images.
       default: ""
     - description: Extra parameters passed for the push command when pushing images.
       name: PUSH_EXTRA_ARGS
       type: string
       default: ""
     - description: Skip pushing the built image
       name: SKIP_PUSH
       type: string
       default: "false"
     results:
     - description: Digest of the image just built.
       name: IMAGE_DIGEST
       type: string
     workspaces:
     - name: source
     steps:
     - name: build
       securityContext:
         runAsUser: 1000
       image: $(params.BUILDER_IMAGE)
       workingDir: $(workspaces.source.path)
       script: |
         echo "Running as USER ID `id`"
         buildah --storage-driver=$(params.STORAGE_DRIVER) bud \
           $(params.BUILD_EXTRA_ARGS) --format=$(params.FORMAT) \
           --tls-verify=$(params.TLSVERIFY) --no-cache \
           -f $(params.DOCKERFILE) -t $(params.IMAGE) $(params.CONTEXT)
         [[ "$(params.SKIP_PUSH)" == "true" ]] && echo "Push skipped" && exit 0
         buildah --storage-driver=$(params.STORAGE_DRIVER) push \
           $(params.PUSH_EXTRA_ARGS) --tls-verify=$(params.TLSVERIFY) \
           --digestfile $(workspaces.source.path)/image-digest $(params.IMAGE) \
           docker://$(params.IMAGE)
         cat $(workspaces.source.path)/image-digest | tee /tekton/results/IMAGE_DIGEST
       volumeMounts:
       - name: varlibcontainers
         mountPath: /home/build/.local/share/containers
     volumes:
     - name: varlibcontainers
       emptyDir: {}

   runAsUser

   Run the container explicitly as the user id 1000, which corresponds to the build user in the Buildah image.

   echo "Running as USER ID `id`"

   Display the user id to confirm that the process is running as user id 1000.

   mountPath

   You can change the path for the volume mount as necessary.

Procedure

1. Create a TaskRun object with a custom ConfigMap and Dockerfile objects.

   Example: A task run that runs Buildah as user id 1000

   apiVersion: v1
   data:
     Dockerfile: |
       ARG BASE_IMG=registry.access.redhat.com/ubi9/ubi
       FROM $BASE_IMG AS buildah-runner
       RUN dnf -y update && \
           dnf -y install git && \
           dnf clean all
       CMD git
   kind: ConfigMap
   metadata:
     name: dockerfile
   ---
   apiVersion: tekton.dev/v1
   kind: TaskRun
   metadata:
     name: buildah-as-user-1000
   spec:
     taskRunTemplate:
       serviceAccountName: pipelines-sa-userid-1000
     params:
     - name: IMAGE
       value: image-registry.openshift-image-registry.svc:5000/test/buildahuser
     taskRef:
       kind: Task
       name: buildah-as-user
     workspaces:
     - configMap:
         name: dockerfile
       name: source

   configMapRef.name

   Use a config map because the focus is on the task run, without any prior task that fetches some sources with a Dockerfile.

   serviceAccountName

   The name of the service account that you created.

   workspaces.name

   Mount a config map as the source workspace for the buildah-as-user task.

2. (Optional) Create a pipeline and a corresponding pipeline run.

   Example: A pipeline and corresponding pipeline run

   apiVersion: tekton.dev/v1
   kind: Pipeline
   metadata:
     name: pipeline-buildah-as-user-1000
   spec:
     params:
     - name: IMAGE
     - name: URL
     workspaces:
     - name: shared-workspace
     - name: sslcertdir
       optional: true
     tasks:
     - name: fetch-repository
       taskRef:
         resolver: cluster
         params:
         - name: kind
           value: task
         - name: name
           value: git-clone
         - name: namespace
           value: openshift-pipelines
       workspaces:
       - name: output
         workspace: shared-workspace
       params:
       - name: URL
         value: $(params.URL)
       - name: SUBDIRECTORY
         value: ""
       - name: DELETE_EXISTING
         value: "true"
     - name: buildah
       taskRef:
         name: buildah-as-user
       runAfter:
       - fetch-repository
       workspaces:
       - name: source
         workspace: shared-workspace
       - name: sslcertdir
         workspace: sslcertdir
       params:
       - name: IMAGE
         value: $(params.IMAGE)
   ---
   apiVersion: tekton.dev/v1
   kind: PipelineRun
   metadata:
     name: pipelinerun-buildah-as-user-1000
   spec:
     taskRunSpecs:
       - pipelineTaskName: buildah
         taskServiceAccountName: pipelines-sa-userid-1000
     params:
     - name: URL
       value: https://github.com/openshift/pipelines-vote-api
     - name: IMAGE
       value: image-registry.openshift-image-registry.svc:5000/test/buildahuser
     pipelineRef:
       name: pipeline-buildah-as-user-1000
     workspaces:
     - name: shared-workspace
       volumeClaimTemplate:
         spec:
           accessModes:
             - ReadWriteOnce
           resources:
             requests:
               storage: 100Mi

   tasks.name: fetch-repository

   Use the git-clone task to fetch the source containing a Dockerfile and build it using the modified Buildah task.

   taskRef.name

   Refer to the modified Buildah task.

   taskServiceAccountName

   Use the service account that you created for the Buildah task.

   workspaces.name: shared-workspace

   Share data between the git-clone task and the modified Buildah task using a persistent volume claim (PVC) created automatically by the controller.

3. Start the task run or the pipeline run.