Securing OpenShift Pipelines

Procedure

1. Create the TektonChain CR for your Red Hat OpenShift Pipelines cluster.

   apiVersion: operator.tekton.dev/v1alpha1
   kind: TektonChain
   metadata:
     name: chain
   spec:
     targetNamespace: openshift-pipelines

   Copy to Clipboard Toggle word wrap

2. Apply the TektonChain CR.

   $ oc apply -f TektonChain.yaml 

   1

   Copy to Clipboard Toggle word wrap

   1

   Substitute with the file name of the TektonChain CR.

3. Check the status of the installation.

   $ oc get tektonchains.operator.tekton.dev

   Copy to Clipboard Toggle word wrap

Procedure

1. Set the namespace and name of the Kubernetes service account.

   $ export NAMESPACE=<namespace> 

   1

   
   $ export SERVICE_ACCOUNT_NAME=<service_account> 

   2

   Copy to Clipboard Toggle word wrap

   1

   The namespace associated with the service account.

   2

   The name of the service account.

2. Create a Kubernetes secret.

   $ oc create secret registry-credentials \
     --from-file=.dockerconfigjson \ 

   1

   
     --type=kubernetes.io/dockerconfigjson \
     -n $NAMESPACE

   Copy to Clipboard Toggle word wrap

   1

   Substitute with the path to your Docker config file. Default path is ~/.docker/config.json.

3. Give the service account access to the secret.

   $ oc patch serviceaccount $SERVICE_ACCOUNT_NAME \
     -p "{\"imagePullSecrets\": [{\"name\": \"registry-credentials\"}]}" -n $NAMESPACE

   Copy to Clipboard Toggle word wrap

   If you patch the default pipeline service account that Red Hat OpenShift Pipelines assigns to all task runs, the Red Hat OpenShift Pipelines Operator will override the service account. As a best practice, you can perform the following steps:

   1. Create a separate service account to assign to user’s task runs.

      $ oc create serviceaccount <service_account_name>

      Copy to Clipboard Toggle word wrap

   2. Associate the service account to the task runs by setting the value of the serviceaccountname field in the task run template.

      apiVersion: tekton.dev/v1beta1
      kind: TaskRun
      metadata:
      name: build-push-task-run-2
      spec:
      serviceAccountName: build-bot 

      1

      
      taskRef:
        name: build-push
      ...

      Copy to Clipboard Toggle word wrap

      1

      Substitute with the name of the newly created service account.

Procedure

1. Create an encrypted x509 key pair and save it as a Kubernetes secret:

   $ cosign generate-key-pair k8s://openshift-pipelines/signing-secrets

   Copy to Clipboard Toggle word wrap

   Provide a password when prompted. Cosign stores the resulting private key as part of the signing-secrets Kubernetes secret in the openshift-pipelines namespace, and writes the public key to the cosign.pub local file.

2. Configure authentication for the image registry.

   1. To configure the Tekton Chains controller for pushing signature to an OCI registry, use the credentials associated with the service account of the task run. For detailed information, see the "Authenticating to an OCI registry" section.

   2. To configure authentication for a Kaniko task that builds and pushes image to the registry, create a Kubernetes secret of the docker config.json file containing the required credentials.

      $ oc create secret generic <docker_config_secret_name> \ 

      1

      
        --from-file <path_to_config.json> 

      2

      Copy to Clipboard Toggle word wrap

      1

      Substitute with the name of the docker config secret.

      2

      Substitute with the path to docker config.json file.

3. Configure Tekton Chains by setting the artifacts.taskrun.format, artifacts.taskrun.storage, and transparency.enabled parameters in the chains-config object:

   $ oc patch configmap chains-config -n openshift-pipelines -p='{"data":{"artifacts.taskrun.format": "in-toto"}}'
   
   $ oc patch configmap chains-config -n openshift-pipelines -p='{"data":{"artifacts.taskrun.storage": "oci"}}'
   
   $ oc patch configmap chains-config -n openshift-pipelines -p='{"data":{"transparency.enabled": "true"}}'

   Copy to Clipboard Toggle word wrap

4. Start the Kaniko task.

   1. Apply the Kaniko task to the cluster.

      $ oc apply -f examples/kaniko/kaniko.yaml 

      1

      Copy to Clipboard Toggle word wrap

      1

      Substitute with the URI or file path to your Kaniko task.

   2. Set the appropriate environment variables.

      $ export REGISTRY=<url_of_registry> 

      1

      
      
      $ export DOCKERCONFIG_SECRET_NAME=<name_of_the_secret_in_docker_config_json> 

      2

      Copy to Clipboard Toggle word wrap

      1

      Substitute with the URL of the registry where you want to push the image.

      2

      Substitute with the name of the secret in the docker config.json file.

   3. Start the Kaniko task.

      $ tkn task start --param IMAGE=$REGISTRY/kaniko-chains --use-param-defaults --workspace name=source,emptyDir="" --workspace name=dockerconfig,secret=$DOCKERCONFIG_SECRET_NAME kaniko-chains

      Copy to Clipboard Toggle word wrap

      Observe the logs of this task until all steps are complete. On successful authentication, the final image will be pushed to $REGISTRY/kaniko-chains.

5. Wait for a minute to allow Tekton Chains to generate the provenance and sign it, and then check the availability of the chains.tekton.dev/signed=true annotation on the task run.

   $ oc get tr <task_run_name> \ 

   1

   
   -o json | jq -r .metadata.annotations
   
   {
     "chains.tekton.dev/signed": "true",
     ...
   }

   Copy to Clipboard Toggle word wrap

   1

   Substitute with the name of the task run.

6. Verify the image and the attestation.

   $ cosign verify --key cosign.pub $REGISTRY/kaniko-chains
   
   $ cosign verify-attestation --key cosign.pub $REGISTRY/kaniko-chains

   Copy to Clipboard Toggle word wrap

7. Find the provenance for the image in Rekor.

   1. Get the digest of the $REGISTRY/kaniko-chains image. You can search for it ing the task run, or pull the image to extract the digest.

   2. Search Rekor to find all entries that match the sha256 digest of the image.

      $ rekor-cli search --sha <image_digest> 

      1

      
      
      <uuid_1> 

      2

      
      <uuid_2> 

      3

      
      ...

      Copy to Clipboard Toggle word wrap

      1

      Substitute with the sha256 digest of the image.

      2

      The first matching universally unique identifier (UUID).

      3

      The second matching UUID.

      The search result displays UUIDs of the matching entries. One of those UUIDs holds the attestation.

   3. Check the attestation.

      $ rekor-cli get --uuid <uuid> --format json | jq -r .Attestation | base64 --decode | jq

      Copy to Clipboard Toggle word wrap

Procedure

1. Define a custom SCC with the fsGroup.type parameter set to RunAsAny:

   Example: Custom SCC

   apiVersion: security.openshift.io/v1
   kind: SecurityContextConstraints
   metadata:
     annotations:
       kubernetes.io/description: my-scc is a close replica of anyuid scc. pipelines-scc has fsGroup - RunAsAny.
     name: my-scc
   allowHostDirVolumePlugin: false
   allowHostIPC: false
   allowHostNetwork: false
   allowHostPID: false
   allowHostPorts: false
   allowPrivilegeEscalation: true
   allowPrivilegedContainer: false
   allowedCapabilities: null
   defaultAddCapabilities: null
   fsGroup:
     type: RunAsAny
   groups:
   - system:cluster-admins
   priority: 10
   readOnlyRootFilesystem: false
   requiredDropCapabilities:
   - MKNOD
   runAsUser:
     type: RunAsAny
   seLinuxContext:
     type: MustRunAs
   supplementalGroups:
     type: RunAsAny
   volumes:
   - configMap
   - downwardAPI
   - emptyDir
   - persistentVolumeClaim
   - projected
   - secret

   Copy to Clipboard Toggle word wrap

2. Create the custom SCC:

   Example: Create the my-scc SCC

   $ oc create -f my-scc.yaml

   Copy to Clipboard Toggle word wrap

3. Create a custom service account:

   Example: Create a fsgroup-runasany service account

   $ oc create serviceaccount fsgroup-runasany

   Copy to Clipboard Toggle word wrap

4. Associate the custom SCC with the custom service account:

   Example: Associate the my-scc SCC with the fsgroup-runasany service account

   $ oc adm policy add-scc-to-user my-scc -z fsgroup-runasany

   Copy to Clipboard Toggle word wrap

   If you want to use the custom service account for privileged tasks, you can associate the privileged SCC with the custom service account by running the following command:

   Example: Associate the privileged SCC with the fsgroup-runasany service account

   $ oc adm policy add-scc-to-user privileged -z fsgroup-runasany

   Copy to Clipboard Toggle word wrap

5. Use the custom service account in the pipeline run and task run:

   Example: Pipeline run YAML with fsgroup-runasany custom service account

   apiVersion: tekton.dev/v1beta1
   kind: PipelineRun
   metadata:
     name: <pipeline-run-name>
   spec:
     pipelineRef:
       name: <pipeline-cluster-task-name>
     serviceAccountName: 'fsgroup-runasany'

   Copy to Clipboard Toggle word wrap

   Example: Task run YAML with fsgroup-runasany custom service account

   apiVersion: tekton.dev/v1beta1
   kind: TaskRun
   metadata:
     name: <task-run-name>
   spec:
     taskRef:
       name: <cluster-task-name>
     serviceAccountName: 'fsgroup-runasany'

   Copy to Clipboard Toggle word wrap

Procedure

1. Create the TriggerBinding resource from the YAML file available in the pipelines-tutorial repository:

   $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/master/03_triggers/01_binding.yaml

   Copy to Clipboard Toggle word wrap

2. Create the TriggerTemplate resource from the YAML file available in the pipelines-tutorial repository:

   $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/master/03_triggers/02_template.yaml

   Copy to Clipboard Toggle word wrap

3. Create the Trigger resource directly from the pipelines-tutorial repository:

   $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/master/03_triggers/03_trigger.yaml

   Copy to Clipboard Toggle word wrap

4. Create an EventListener resource using a secure HTTPS connection:

   1. Add a label to enable the secure HTTPS connection to the Eventlistener resource:

      $ oc label namespace <ns-name> operator.tekton.dev/enable-annotation=enabled

      Copy to Clipboard Toggle word wrap

   2. Create the EventListener resource from the YAML file available in the pipelines-tutorial repository:

      $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/master/03_triggers/04_event_listener.yaml

      Copy to Clipboard Toggle word wrap

   3. Create a route with the re-encrypted TLS termination:

      $ oc create route reencrypt --service=<svc-name> --cert=tls.crt --key=tls.key --ca-cert=ca.crt --hostname=<hostname>

      Copy to Clipboard Toggle word wrap

Procedure

1. In the secret.yaml file, specify the username and password or GitHub personal access token to access the target Git repository.

   apiVersion: v1
   kind: Secret
   metadata:
     name: basic-user-pass 

   1

   
     annotations:
       tekton.dev/git-0: https://github.com
   type: kubernetes.io/basic-auth
   stringData:
     username: <username> 

   2

   
     password: <password> 

   3

   Copy to Clipboard Toggle word wrap

   1

   Name of the secret. In this example, basic-user-pass.

   2

   Username for the Git repository.

   3

   Password for the Git repository.

2. In the serviceaccount.yaml file, associate the secret with the appropriate service account.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: build-bot 

   1

   
   secrets:
     - name: basic-user-pass 

   2

   Copy to Clipboard Toggle word wrap

   1

   Name of the service account. In this example, build-bot.

   2

   Name of the secret. In this example, basic-user-pass.

3. In the run.yaml file, associate the service account with a task run or a pipeline run.

   • Associate the service account with a task run:

     apiVersion: tekton.dev/v1beta1
     kind: TaskRun
     metadata:
       name: build-push-task-run-2 

     1

     
     spec:
       serviceAccountName: build-bot 

     2

     
       taskRef:
         name: build-push 

     3

     Copy to Clipboard Toggle word wrap

     1

     Name of the task run. In this example, build-push-task-run-2.

     2

     Name of the service account. In this example, build-bot.

     3

     Name of the task. In this example, build-push.

   • Associate the service account with a PipelineRun resource:

     apiVersion: tekton.dev/v1beta1
     kind: PipelineRun
     metadata:
       name: demo-pipeline 

     1

     
       namespace: default
     spec:
       serviceAccountName: build-bot 

     2

     
       pipelineRef:
         name: demo-pipeline 

     3

     Copy to Clipboard Toggle word wrap

     1

     Name of the pipeline run. In this example, demo-pipeline.

     2

     Name of the service account. In this example, build-bot.

     3

     Name of the pipeline. In this example, demo-pipeline.

4. Apply the changes.

   $ oc apply --filename secret.yaml,serviceaccount.yaml,run.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Generate an SSH private key, or copy an existing private key, which is usually available in the ~/.ssh/id_rsa file.

2. In the secret.yaml file, set the value of ssh-privatekey to the content of the SSH private key file, and set the value of known_hosts to the content of the known hosts file.

   apiVersion: v1
   kind: Secret
   metadata:
     name: ssh-key 

   1

   
     annotations:
       tekton.dev/git-0: github.com
   type: kubernetes.io/ssh-auth
   stringData:
     ssh-privatekey: 

   2

   
     known_hosts: 

   3

   Copy to Clipboard Toggle word wrap

   1

   Name of the secret containing the SSH private key. In this example, ssh-key.

   2

   The content of the SSH private key file.

   3

   The content of the known hosts file.

   Important

   If you omit the private key, OpenShift Pipelines accepts the public key of any server.

3. Optional: To specify a custom SSH port, add :<port number> to the end of the annotation value. For example, tekton.dev/git-0: github.com:2222.

4. In the serviceaccount.yaml file, associate the ssh-key secret with the build-bot service account.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: build-bot 

   1

   
   secrets:
     - name: ssh-key 

   2

   Copy to Clipboard Toggle word wrap

   1

   Name of the service account. In this example, build-bot.

   2

   Name of the secret containing the SSH private key. In this example, ssh-key.

5. In the run.yaml file, associate the service account with a task run or a pipeline run.

   • Associate the service account with a task run:

     apiVersion: tekton.dev/v1beta1
     kind: TaskRun
     metadata:
       name: build-push-task-run-2 

     1

     
     spec:
       serviceAccountName: build-bot 

     2

     
       taskRef:
         name: build-push 

     3

     Copy to Clipboard Toggle word wrap

     1

     Name of the task run. In this example, build-push-task-run-2.

     2

     Name of the service account. In this example, build-bot.

     3

     Name of the task. In this example, build-push.

   • Associate the service account with a pipeline run:

     apiVersion: tekton.dev/v1beta1
     kind: PipelineRun
     metadata:
       name: demo-pipeline 

     1

     
       namespace: default
     spec:
       serviceAccountName: build-bot 

     2

     
       pipelineRef:
         name: demo-pipeline 

     3

     Copy to Clipboard Toggle word wrap

     1

     Name of the pipeline run. In this example, demo-pipeline.

     2

     Name of the service account. In this example, build-bot.

     3

     Name of the pipeline. In this example, demo-pipeline.

6. Apply the changes.

   $ oc apply --filename secret.yaml,serviceaccount.yaml,run.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a copy of the buildah cluster task as an ordinary task.

   $ oc get clustertask buildah -o yaml | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f -

   Copy to Clipboard Toggle word wrap

2. Edit the copied buildah task.

   $ oc edit task buildah-as-user

   Copy to Clipboard Toggle word wrap

   Example: Modified Buildah task with build user

   apiVersion: tekton.dev/v1beta1
   kind: Task
   metadata:
     name: buildah-as-user
   spec:
     description: >-
       Buildah task builds source into a container image and
       then pushes it to a container registry.
       Buildah Task builds source into a container image using Project Atomic's
       Buildah build tool.It uses Buildah's support for building from Dockerfiles,
       using its buildah bud command.This command executes the directives in the
       Dockerfile to assemble a container image, then pushes that image to a
       container registry.
     params:
     - name: IMAGE
       description: Reference of the image buildah will produce.
     - name: BUILDER_IMAGE
       description: The location of the buildah builder image.
       default: registry.redhat.io/rhel8/buildah@sha256:99cae35f40c7ec050fed3765b2b27e0b8bbea2aa2da7c16408e2ca13c60ff8ee
     - name: STORAGE_DRIVER
       description: Set buildah storage driver
       default: vfs
     - name: DOCKERFILE
       description: Path to the Dockerfile to build.
       default: ./Dockerfile
     - name: CONTEXT
       description: Path to the directory to use as context.
       default: .
     - name: TLSVERIFY
       description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)
       default: "true"
     - name: FORMAT
       description: The format of the built container, oci or docker
       default: "oci"
     - name: BUILD_EXTRA_ARGS
       description: Extra parameters passed for the build command when building images.
       default: ""
     - description: Extra parameters passed for the push command when pushing images.
       name: PUSH_EXTRA_ARGS
       type: string
       default: ""
     - description: Skip pushing the built image
       name: SKIP_PUSH
       type: string
       default: "false"
     results:
     - description: Digest of the image just built.
       name: IMAGE_DIGEST
       type: string
     workspaces:
     - name: source
     steps:
     - name: build
       securityContext:
         runAsUser: 1000 

   1

   
       image: $(params.BUILDER_IMAGE)
       workingDir: $(workspaces.source.path)
       script: |
         echo "Running as USER ID `id`" 

   2

   
         buildah --storage-driver=$(params.STORAGE_DRIVER) bud \
           $(params.BUILD_EXTRA_ARGS) --format=$(params.FORMAT) \
           --tls-verify=$(params.TLSVERIFY) --no-cache \
           -f $(params.DOCKERFILE) -t $(params.IMAGE) $(params.CONTEXT)
         [[ "$(params.SKIP_PUSH)" == "true" ]] && echo "Push skipped" && exit 0
         buildah --storage-driver=$(params.STORAGE_DRIVER) push \
           $(params.PUSH_EXTRA_ARGS) --tls-verify=$(params.TLSVERIFY) \
           --digestfile $(workspaces.source.path)/image-digest $(params.IMAGE) \
           docker://$(params.IMAGE)
         cat $(workspaces.source.path)/image-digest | tee /tekton/results/IMAGE_DIGEST
       volumeMounts:
       - name: varlibcontainers
         mountPath: /home/build/.local/share/containers 

   3

   
     volumes:
     - name: varlibcontainers
       emptyDir: {}

   Copy to Clipboard Toggle word wrap

   1

   Run the container explicitly as the user id 1000, which corresponds to the build user in the Buildah image.

   2

   Display the user id to confirm that the process is running as user id 1000.

   3

   You can change the path for the volume mount as necessary.

Procedure

1. Create a TaskRun object with a custom ConfigMap and Dockerfile objects.

   Example: A task run that runs Buildah as user id 1000

   apiVersion: v1
   data:
     Dockerfile: |
       ARG BASE_IMG=registry.access.redhat.com/ubi8/ubi
       FROM $BASE_IMG AS buildah-runner
       RUN dnf -y update && \
           dnf -y install git && \
           dnf clean all
       CMD git
   kind: ConfigMap
   metadata:
     name: dockerfile 

   1

   
   ---
   apiVersion: tekton.dev/v1beta1
   kind: TaskRun
   metadata:
     name: buildah-as-user-1000
   spec:
     serviceAccountName: pipelines-sa-userid-1000 

   2

   
     params:
     - name: IMAGE
       value: image-registry.openshift-image-registry.svc:5000/test/buildahuser
     taskRef:
       kind: Task
       name: buildah-as-user
     workspaces:
     - configMap:
         name: dockerfile 

   3

   
       name: source

   Copy to Clipboard Toggle word wrap

   1

   Use a config map because the focus is on the task run, without any prior task that fetches some sources with a Dockerfile.

   2

   The name of the service account that you created.

   3

   Mount a config map as the source workspace for the buildah-as-user task.

2. (Optional) Create a pipeline and a corresponding pipeline run.

   Example: A pipeline and corresponding pipeline run

   apiVersion: tekton.dev/v1beta1
   kind: Pipeline
   metadata:
     name: pipeline-buildah-as-user-1000
   spec:
     params:
     - name: IMAGE
     - name: URL
     workspaces:
     - name: shared-workspace
     - name: sslcertdir
       optional: true
     tasks:
     - name: fetch-repository 

   1

   
       taskRef:
         name: git-clone
         kind: ClusterTask
       workspaces:
       - name: output
         workspace: shared-workspace
       params:
       - name: url
         value: $(params.URL)
       - name: subdirectory
         value: ""
       - name: deleteExisting
         value: "true"
     - name: buildah
       taskRef:
         name: buildah-as-user 

   2

   
       runAfter:
       - fetch-repository
       workspaces:
       - name: source
         workspace: shared-workspace
       - name: sslcertdir
         workspace: sslcertdir
       params:
       - name: IMAGE
         value: $(params.IMAGE)
   ---
   apiVersion: tekton.dev/v1beta1
   kind: PipelineRun
   metadata:
     name: pipelinerun-buildah-as-user-1000
   spec:
     taskRunSpecs:
       - pipelineTaskName: buildah
         taskServiceAccountName: pipelines-sa-userid-1000 

   3

   
     params:
     - name: URL
       value: https://github.com/openshift/pipelines-vote-api
     - name: IMAGE
       value: image-registry.openshift-image-registry.svc:5000/test/buildahuser
     pipelineRef:
       name: pipeline-buildah-as-user-1000
     workspaces:
     - name: shared-workspace 

   4

   
       volumeClaimTemplate:
         spec:
           accessModes:
             - ReadWriteOnce
           resources:
             requests:
               storage: 100Mi

   Copy to Clipboard Toggle word wrap

   1

   Use the git-clone cluster task to fetch the source containing a Dockerfile and build it using the modified Buildah task.

   2

   Refer to the modified Buildah task.

   3

   Use the service account that you created for the Buildah task.

   4

   Share data between the git-clone task and the modified Buildah task using a persistent volume claim (PVC) created automatically by the controller.

3. Start the task run or the pipeline run.