About OpenShift Pipelines

You can use Task resources as the building blocks of a pipeline to define a set of sequentially executed steps. Each task functions as a reusable unit of work with specific inputs and outputs, capable of running individually or as part of a larger pipeline.

Task resources are the building blocks of a pipeline and consist of sequentially executed steps. It is essentially a function of inputs and outputs. A task can run individually or as a part of the pipeline. You can reuse tasks in many pipelines.

Steps are a series of commands that are sequentially executed by the task and achieve a specific goal, such as building an image. Every task runs as a pod, and each step runs as a container within that pod. Because steps run within the same pod, they can access the same volumes for caching files, config maps, and secrets.

The following example shows the apply-manifests task.

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: apply-manifests
spec:
  workspaces:
  - name: source
  params:
    - name: manifest_dir
      description: The directory in source that contains yaml manifests
      type: string
      default: "k8s"
  steps:
    - name: apply
      image: image-registry.openshift-image-registry.svc:5000/openshift/cli:latest
      workingDir: /workspace/source
      command: ["/bin/bash", "-c"]
      args:
        - |-
          echo Applying manifests in $(params.manifest_dir) directory
          oc apply -f $(params.manifest_dir)
          echo -----------------------------------

This task starts the pod and runs a container inside that pod by using the specified image to run the specified commands.

spec:
  pipeline:
    disable-working-directory-overwrite: false
    disable-home-env-overwrite: false

You can use 'when' expressions to guard task execution by defining specific criteria that must be met before a task runs. These expressions allow you to control the flow of your pipeline, including the execution of tasks in the finally section, based on static inputs, variables, or results from earlier tasks.

When expressions guard task execution by setting criteria for running tasks within a pipeline. They contain a list of components that allow a task to run only when certain criteria are met. You can also include when expressions in the final set of tasks that you specify by using the finally field in the pipeline YAML file.

The key components of a when expression are as follows:

The declared when expressions evaluate before the task runs. If the when expression evaluates to True, the task runs. If the when expression evaluates to False, the task skips.

You can use the when expressions in various use cases. For example, whether:

The following example shows the when expressions for a pipeline run. The pipeline run will run the create-file task only if the following criteria are met: the path parameter is README.md, and the echo-file-exists task runs only if the exists result from the check-file task is yes.

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  generateName: guarded-pr-
spec:
  taskRunTemplate:
    serviceAccountName: pipeline
  pipelineSpec:
    params:
      - name: path
        type: string
        description: The path of the file to be created
    workspaces:
      - name: source
        description: |
          This workspace is shared among all the pipeline tasks to read/write common resources
    tasks:
      - name: create-file
        when:
          - input: "$(params.path)"
            operator: in
            values: ["README.md"]
        workspaces:
          - name: source
            workspace: source
        taskSpec:
          workspaces:
            - name: source
              description: The workspace to create the readme file in
          steps:
            - name: write-new-stuff
              image: ubuntu
              script: 'touch $(workspaces.source.path)/README.md'
      - name: check-file
        params:
          - name: path
            value: "$(params.path)"
        workspaces:
          - name: source
            workspace: source
        runAfter:
          - create-file
        taskSpec:
          params:
            - name: path
          workspaces:
            - name: source
              description: The workspace to check for the file
          results:
            - name: exists
              description: indicates whether the file exists or is missing
          steps:
            - name: check-file
              image: alpine
              script: |
                if test -f $(workspaces.source.path)/$(params.path); then
                  printf yes | tee /tekton/results/exists
                else
                  printf no | tee /tekton/results/exists
                fi
      - name: echo-file-exists
        when:
          - input: "$(tasks.check-file.results.exists)"
            operator: in
            values: ["yes"]
        taskSpec:
          steps:
            - name: echo
              image: ubuntu
              script: 'echo file exists'
...
      - name: task-should-be-skipped-1
        when:
          - input: "$(params.path)"
            operator: notin
            values: ["README.md"]
        taskSpec:
          steps:
            - name: echo
              image: ubuntu
              script: exit 1
...
    finally:
      - name: finally-task-should-be-executed
        when:
          - input: "$(tasks.echo-file-exists.status)"
            operator: in
            values: ["Succeeded"]
          - input: "$(tasks.status)"
            operator: in
            values: ["Succeeded"]
          - input: "$(tasks.check-file.results.exists)"
            operator: in
            values: ["yes"]
          - input: "$(params.path)"
            operator: in
            values: ["README.md"]
        taskSpec:
          steps:
            - name: echo
              image: ubuntu
              script: 'echo finally done'
  params:
    - name: path
      value: README.md
  workspaces:
    - name: source
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 16Mi

The Pipeline Run details page of the OpenShift Container Platform web console shows the status of the tasks and when expressions as follows:

You can use finally tasks to run a final set of tasks in your pipeline regardless of whether the earlier tasks succeed or fail. These tasks run in parallel after all other pipeline tasks finish, allowing you to perform cleanup or notification actions before the pipeline exits.

The finally tasks are the final set of tasks specified using the finally field in the pipeline YAML file. A finally task always executes the tasks within the pipeline, irrespective of whether the pipeline runs succeed.

You can configure a finally task to consume the results of any task within the same pipeline. This approach does not change the order in which this final task runs.

The following example shows a code snippet of the clone-cleanup-workspace pipeline. This code clones the repository into a shared workspace and cleans up the workspace. After the pipeline tasks finish, the cleanup task specified in the finally section of the pipeline YAML file cleans up the workspace.

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: clone-cleanup-workspace
spec:
  workspaces:
    - name: git-source
  tasks:
    - name: clone-app-repo
      taskRef:
        name: git-clone-from-catalog
      params:
        - name: url
          value: https://github.com/tektoncd/community.git
        - name: subdirectory
          value: application
      workspaces:
        - name: output
          workspace: git-source
  finally:
    - name: cleanup
      taskRef:
        name: cleanup-workspace
      workspaces:
        - name: source
          workspace: git-source
    - name: check-git-commit
      params:
        - name: commit
          value: $(tasks.clone-app-repo.results.commit)
      taskSpec:
        params:
          - name: commit
        steps:
          - name: check-commit-initialized
            image: alpine
            script: |
              if [[ ! $(params.commit) ]]; then
                exit 1
              fi

You can use a TaskRun resource to instantiate and run a task with specific inputs, outputs, and execution parameters on a cluster. You can start a task run independently or as part of a pipeline run to run the steps defined in a task.

A TaskRun instantiates a task for execution with specific inputs, outputs, and execution parameters on a cluster. You can start it on its own or as part of a pipeline run for each task in a pipeline.

A task consists of one or more steps that run container images, and each container image performs a specific piece of build work. A task run starts the steps in a task in the specified order, until all steps run successfully or a failure occurs. A TaskRun is automatically created by a PipelineRun for each task in a pipeline.

The following example shows a task run that runs the apply-manifests task with the relevant input parameters:

apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
  name: apply-manifests-taskrun
spec:
  taskRunTemplate:
    serviceAccountName: pipeline
  taskRef:
    kind: Task
    name: apply-manifests
  workspaces:
  - name: source
    persistentVolumeClaim:
      claimName: source-pvc

You can use a Pipeline resource to arrange a collection of tasks in a specific order of execution. By defining a pipeline, you construct complex workflows that automate the build, deployment, and delivery of your applications.

A Pipeline is a collection of Task resources arranged in a specific order of execution. You run them to construct complex workflows that automate the build, deployment and delivery of applications by using one or more tasks to define a CI/CD workflow for your application.

A Pipeline resource definition consists of several fields or attributes, which together enable the pipeline to run a specific goal. Each Pipeline resource definition must contain at least one Task resource, which obtains specific inputs and produces specific outputs. The pipeline definition can also optionally include several Conditions, Workspaces, Parameters, or Resources depending on the application requirements.

The following example shows the build-and-deploy pipeline, which builds an application image from a Git repository by using the buildah task provided in the openshift-pipelines namespace:

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: build-and-deploy
spec:
  workspaces:
  - name: shared-workspace
  params:
  - name: deployment-name
    type: string
    description: name of the deployment to be patched
  - name: git-url
    type: string
    description: url of the git repo for the code of deployment
  - name: git-revision
    type: string
    description: revision to be used from repo of the code for deployment
    default: "pipelines-1.21"
  - name: IMAGE
    type: string
    description: image to be built from the code
  tasks:
  - name: fetch-repository
    taskRef:
      resolver: cluster
      params:
      - name: kind
        value: task
      - name: name
        value: git-clone
      - name: namespace
        value: openshift-pipelines
    workspaces:
    - name: output
      workspace: shared-workspace
    params:
    - name: URL
      value: $(params.git-url)
    - name: SUBDIRECTORY
      value: ""
    - name: DELETE_EXISTING
      value: "true"
    - name: REVISION
      value: $(params.git-revision)
  - name: build-image
    taskRef:
      resolver: cluster
      params:
      - name: kind
        value: task
      - name: name
        value: buildah
      - name: namespace
        value: openshift-pipelines
    workspaces:
    - name: source
      workspace: shared-workspace
    params:
    - name: TLSVERIFY
      value: "false"
    - name: IMAGE
      value: $(params.IMAGE)
    runAfter:
    - fetch-repository
  - name: apply-manifests
    taskRef:
      name: apply-manifests
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter:
    - build-image
  - name: update-deployment
    taskRef:
      name: update-deployment
    workspaces:
    - name: source
      workspace: shared-workspace
    params:
    - name: deployment
      value: $(params.deployment-name)
    - name: IMAGE
      value: $(params.IMAGE)
    runAfter:
    - apply-manifests

You can use a PipelineRun resource to instantiate and run a pipeline with specific inputs, outputs, and credentials. This resource binds a pipeline to a workspace and parameter values, enabling you to run your CI/CD workflow for a specific scenario.

A PipelineRun is a type of resource that binds a pipeline, workspaces, credentials, and a set of parameter values specific to a scenario to run the CI/CD workflow.

A pipeline run is the running instance of a pipeline. It also creates a task run for each task in the pipeline run.

The pipeline executes the tasks sequentially until they are complete or a task fails. The status field tracks and the progress of each task run and stores it for monitoring and auditing purposes.

The following example runs the build-and-deploy pipeline with relevant resources and parameters:

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: build-deploy-api-pipelinerun
spec:
  pipelineRef:
    name: build-and-deploy
  params:
  - name: deployment-name
    value: vote-api
  - name: git-url
    value: https://github.com/openshift-pipelines/vote-api.git
  - name: IMAGE
    value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api
  workspaces:
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 500Mi

You can use a pod template in a PipelineRun or TaskRun custom resource (CR) to configure the pods that run your tasks. This allows you to set specific parameters, such as security contexts or user IDs, for every pod created during the pipeline or task run.

Optionally, you can define a pod template in a PipelineRun or TaskRun custom resource (CR). You can use any parameters available for a Pod CR in the pod template. When creating pods for running the pipeline or task, OpenShift Pipelines sets these parameters for every pod.

For example, you can use a pod template to make the pod run as a user and not as root.

For a pipeline run, you can define a pod template in the pipelineRunTemplate.podTemplate spec, as in the following example:

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: mypipelinerun
spec:
  pipelineRef:
    name: mypipeline
  taskRunTemplate:
    podTemplate:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1001

For a task run, you can define a pod template in the podTemplate spec, as in the following example:

apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
  name: mytaskrun
  namespace: default
spec:
  taskRef:
    name: mytask
  podTemplate:
    schedulerName: volcano
    securityContext:
      runAsNonRoot: true
      runAsUser: 1001

You can use workspaces to declare shared storage volumes that tasks in a pipeline need for input or output at runtime. By separating volume declaration from runtime storage, workspaces allow you to specify the filesystem location independently, making your tasks reusable and flexible across different environments.

Instead of specifying the actual location of the volumes, workspaces allow you to declare the filesystem or parts of the filesystem that you need at runtime. A task or pipeline declares the workspace, and you must give the specific location details of the volume. It is then mounted into that workspace in a task run or a pipeline run. This separation of volume declaration from runtime storage volumes makes the tasks reusable, flexible, and independent of the user environment.

With workspaces, you can:

You can specify workspaces in the TaskRun or PipelineRun using:

The following example shows a code snippet of the build-and-deploy pipeline, which declares a shared-workspace workspace for the build-image and apply-manifests tasks you define in the pipeline.

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: build-and-deploy
spec:
  workspaces:
  - name: shared-workspace
  params:
...
  tasks:
  - name: build-image
    taskRef:
      resolver: cluster
      params:
      - name: kind
        value: task
      - name: name
        value: buildah
      - name: namespace
        value: openshift-pipelines
    workspaces:
    - name: source
      workspace: shared-workspace
    params:
    - name: TLSVERIFY
      value: "false"
    - name: IMAGE
      value: $(params.IMAGE)
    runAfter:
    - fetch-repository
  - name: apply-manifests
    taskRef:
      name: apply-manifests
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter:
      - build-image
...

Workspaces help tasks share data, and let you specify one or more volumes that each task in the pipeline requires during execution. You can create a persistent volume claim or give a volume claim template that creates a persistent volume claim for you.

The following code snippet of the build-deploy-api-pipelinerun pipeline run uses a volume claim template to create a persistent volume claim for defining the storage volume for the shared-workspace workspace used in the build-and-deploy pipeline.

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: build-deploy-api-pipelinerun
spec:
  pipelineRef:
    name: build-and-deploy
  params:
...

  workspaces:
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 500Mi

You can use a StepAction custom resource (CR) to define a reusable action that a step performs. By referencing a StepAction object from a step, you can share and reuse action definitions across many tasks or reference actions from external sources.

A step is a part of a task. If you define a step in a task, you cannot reference this step from another task.

This CR has the action that a step performs. You can reference a StepAction object from a step to create a step that performs the action. You can also use resolvers to reference a StepAction definition that is available from an external source.

The following examples shows a StepAction CR named apply-manifests-action. This step action applies manifests from a source tree to your OpenShift Container Platform environment:

apiVersion: tekton.dev/v1
kind: StepAction
metadata:
  name: apply-manifests-action
spec:
  params:
  - name: working_dir
    description: The working directory where the source is located
    type: string
    default: "/workspace/source"
  - name: manifest_dir
    description: The directory in source that contains yaml manifests
    default: "k8s"
  results:
  - name: output
    description: The output of the oc apply command
  image: image-registry.openshift-image-registry.svc:5000/openshift/cli:latest
  env:
  - name: MANIFEST_DIR
    value: $(params.manifest_dir)
  workingDir: $(params.working_dir)
  script: |
      #!/usr/bin/env bash
      oc apply -f "$MANIFEST_DIR" | tee $(results.output)

The StepAction CR does not include definitions of workspaces. Instead, the step action expects that the task that includes the action also provides the mounted source tree, typically using a workspace.

A StepAction object can define parameters and results. When you reference this object, you must specify the values for the parameters of the StepAction object in the definition of the step. The results of the StepAction object automatically become the results of the step.

The following example task includes a step that references the apply-manifests-action step action, provides the necessary parameters, and uses the result:

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: apply-manifests-with-action
spec:
  workspaces:
  - name: source
  params:
  - name: manifest_dir
    description: The directory in source that contains yaml manifests
    type: string
    default: "k8s"
  steps:
  - name: apply
    ref:
      name: apply-manifests-action
    params:
    - name: working_dir
      value: "/workspace/source"
    - name: manifest_dir
      value: $(params.manifest_dir)
  - name: display_result
    script: 'echo $(step.apply.results.output)'

You can use Triggers in conjunction with pipelines to create a comprehensive CI/CD system driven by Kubernetes resources. Triggers capture external events, such as Git pull requests, and process them to extract information, enabling you to automatically instantiate pipelines and deploy resources based on event data.

For example, you define a CI/CD workflow by using Red Hat OpenShift Pipelines for your application. The pipeline must start for any new changes to take effect in the application repository. Triggers automate this process by capturing and processing any change event and by triggering a pipeline run that deploys the new image with the latest changes.

Triggers consist of the following main resources that work together to form a reusable, decoupled, and self-sustaining CI/CD system: