Chapter 4. Creating CI/CD solutions for applications using OpenShift Pipelines
With Red Hat OpenShift Pipelines, you can create a customized CI/CD solution to build, test, and deploy your application.
To create a full-fledged, self-serving CI/CD Pipeline for an application, you must perform the following tasks:
- Create custom Tasks, or install existing reusable Tasks.
- Create and define the delivery Pipeline for your application.
- Create a PersistentVolumeClaim attached to the Workspace to provide the volume or filesystem for Pipeline execution.
- Create a PipelineRun to instantiate and invoke the Pipeline.
- Add Triggers to capture any events in the source repository.
This section uses the pipelines-tutorial
example to demonstrate the preceding tasks. The example uses a simple application which consists of:
-
A front-end interface,
vote-ui
, with the source code in theui-repo
Git repository. -
A back-end interface,
vote-api
, with the source code in theapi-repo
Git repository. -
The
apply_manifest
andupdate-deployment
Tasks in thepipelines-tutorial
Git repository.
4.1. Prerequisites
- You have access to an OpenShift Container Platform cluster.
- You have installed OpenShift Pipelines using the Red Hat OpenShift Pipelines Operator listed in the OpenShift OperatorHub. Once installed, it is applicable to the entire cluster.
- You have installed OpenShift Pipelines CLI.
-
You have forked the front-end
ui-repo
and back-endapi-repo
Git repositories using your GitHub ID, and have Administrator access to these repositories. -
Optional: You have cloned the
pipelines-tutorial
Git repository.
4.2. Creating a project and checking your Pipeline ServiceAccount
Procedure
Log in to your OpenShift Container Platform cluster:
$ oc login -u <login> -p <password> https://openshift.example.com:6443
Create a project for the sample application. For this example workflow, create the
pipelines-tutorial
project:$ oc new-project pipelines-tutorial
NoteIf you create a project with a different name, be sure to update the resource URLs used in the example with your project name.
View the
pipeline
ServiceAccount:Red Hat OpenShift Pipelines Operator adds and configures a ServiceAccount named
pipeline
that has sufficient permissions to build and push an image. This ServiceAccount is used by PipelineRun.$ oc get serviceaccount pipeline
4.3. Creating Pipeline Tasks
Procedure
Install the
apply-manifests
andupdate-deployment
Tasks from thepipelines-tutorial
repository, which contains a list of reusable Tasks for Pipelines:$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-2/01_pipeline/01_apply_manifest_task.yaml $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-2/01_pipeline/02_update_deployment_task.yaml
Use the
tkn task list
command to list the Tasks you created:$ tkn task list
The output verifies that the
apply-manifests
andupdate-deployment
Tasks were created:NAME DESCRIPTION AGE apply-manifests 1 minute ago update-deployment 48 seconds ago
Use the
tkn clustertasks list
command to list the Operator-installed additional ClusterTasks, for example --buildah
ands2i-python-3
:NoteYou must use a privileged Pod container to run the
buildah
ClusterTask because it requires a privileged security context. To learn more about security context constraints (SCC) for pods, see the Additional resources section.$ tkn clustertasks list
The output lists the Operator-installed ClusterTasks:
NAME DESCRIPTION AGE buildah 1 day ago git-clone 1 day ago s2i-php 1 day ago tkn 1 day ago
4.4. Assembling a Pipeline
A Pipeline represents a CI/CD flow and is defined by the Tasks to be executed. It is designed to be generic and reusable in multiple applications and environments.
A Pipeline specifies how the Tasks interact with each other and their order of execution using the from
and runAfter
parameters. It uses the workspaces
field to specify one or more volumes that each Task in the Pipeline requires during execution.
In this section, you will create a Pipeline that takes the source code of the application from GitHub and then builds and deploys it on OpenShift Container Platform.
The Pipeline performs the following tasks for the back-end application vote-api
and front-end application vote-ui
:
-
Clones the source code of the application from the Git repository by referring to the
git-url
andgit-revision
parameters. -
Builds the container image using the
buildah
ClusterTask. -
Pushes the image to the internal image registry by referring to the
image
parameter. -
Deploys the new image on OpenShift Container Platform by using the
apply-manifests
andupdate-deployment
Tasks.
Procedure
Copy the contents of the following sample Pipeline YAML file and save it:
apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: build-and-deploy spec: workspaces: - name: shared-workspace params: - name: deployment-name type: string description: name of the deployment to be patched - name: git-url type: string description: url of the git repo for the code of deployment - name: git-revision type: string description: revision to be used from repo of the code for deployment default: "release-tech-preview-2" - name: IMAGE type: string description: image to be built from the code tasks: - name: fetch-repository taskRef: name: git-clone kind: ClusterTask workspaces: - name: output workspace: shared-workspace params: - name: url value: $(params.git-url) - name: subdirectory value: "" - name: deleteExisting value: "true" - name: revision value: $(params.git-revision) - name: build-image taskRef: name: buildah kind: ClusterTask params: - name: TLSVERIFY value: "false" - name: IMAGE value: $(params.IMAGE) workspaces: - name: source workspace: shared-workspace runAfter: - fetch-repository - name: apply-manifests taskRef: name: apply-manifests workspaces: - name: source workspace: shared-workspace runAfter: - build-image - name: update-deployment taskRef: name: update-deployment workspaces: - name: source workspace: shared-workspace params: - name: deployment value: $(params.deployment-name) - name: IMAGE value: $(params.IMAGE) runAfter: - apply-manifests
The Pipeline definition abstracts away the specifics of the Git source repository and image registries. These details are added as
params
when a Pipeline is triggered and executed.Create the Pipeline:
$ oc create -f <pipeline-yaml-file-name.yaml>
Alternatively, you can also execute the YAML file directly from the Git repository:
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-2/01_pipeline/04_pipeline.yaml
Use the
tkn pipeline list
command to verify that the Pipeline is added to the application:$ tkn pipeline list
The output verifies that the
build-and-deploy
Pipeline was created:NAME AGE LAST RUN STARTED DURATION STATUS build-and-deploy 1 minute ago --- --- --- ---
4.5. Specifying PersistentVolumeClaims as VolumeSource in Workspaces
Workspaces help Tasks share data, and allow you to specify one or more volumes that each Task in the Pipeline requires during execution.
In this section, you will create a PersistentVolumeClaim to provide data storage and bind it to the Workspace. This PersistentVolumeClaim provides the volumes or filesystem required for the Pipeline execution.
Procedure
Copy and save the contents of the following sample PersistentVolumeClaim YAML file:
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: source-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 500Mi
Create the PersistentVolumeClaim, specifying the file you just created:
$ oc create -f <PersistentVolumeClaim-yaml-file-name.yaml>
Alternatively, you can execute the YAML file directly from the Git repository:
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-2/01_pipeline/03_persistent_volume_claim.yaml
4.6. Running a Pipeline
A PipelineRun starts a Pipeline and ties it to the Git and image resources that should be used for the specific invocation. It automatically creates and starts the TaskRuns for each Task in the Pipeline.
Procedure
Start the Pipeline for the back-end application:
$ tkn pipeline start build-and-deploy -w name=shared-workspace,claimName=source-pvc -p deployment-name=vote-api -p git-url=http://github.com/openshift-pipelines/vote-api.git -p IMAGE=image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api
Note the PipelineRun ID returned in the command output.
Track the PipelineRun progress:
$ tkn pipelinerun logs <pipelinerun ID> -f
Start the Pipeline for the front-end application:
$ tkn pipeline start build-and-deploy -w name=shared-workspace,claimName=source-pvc -p deployment-name=vote-api -p git-url=http://github.com/openshift-pipelines/vote-ui.git -p IMAGE=image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-ui
Note the PipelineRun ID returned in the command output.
Track the PipelineRun progress:
$ tkn pipelinerun logs <pipelinerun ID> -f
After a few minutes, use
tkn pipelinerun list
command to verify that the Pipeline ran successfully by listing all the PipelineRuns:$ tkn pipelinerun list
The output lists the PipelineRuns:
NAME STARTED DURATION STATUS build-and-deploy-run-xy7rw 1 hour ago 2 minutes Succeeded build-and-deploy-run-z2rz8 1 hour ago 19 minutes Succeeded
Get the application route:
$ oc get route vote-ui --template='http://{{.spec.host}}'
Note the output of the previous command. You can access the application using this route.
To rerun the last PipelineRun, using the PipelineResources and ServiceAccount of the previous Pipeline, run:
$ tkn pipeline start build-and-deploy --last
4.7. Adding Triggers to a Pipeline
Triggers enable Pipelines to respond to external GitHub events, such as push events and pull requests. After you have assembled and started the Pipeline for the application, add TriggerBindings, TriggerTemplates, and an EventListener to capture the GitHub events.
Procedure
Copy the content of the following sample
TriggerBinding
YAML file and save it:apiVersion: triggers.tekton.dev/v1alpha1 kind: TriggerBinding metadata: name: vote-app spec: params: - name: git-repo-url value: $(body.repository.url) - name: git-repo-name value: $(body.repository.name) - name: git-revision value: $(body.head_commit.id)
Create the
TriggerBinding
:$ oc create -f <triggerbinding-yaml-file-name.yaml>
Alternatively, you can create the
TriggerBinding
directly from thepipelines-tutorial
Git repository:$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-2/03_triggers/01_binding.yaml
Copy the content of the following sample
TriggerTemplate
YAML file and save it:apiVersion: triggers.tekton.dev/v1alpha1 kind: TriggerTemplate metadata: name: vote-app spec: params: - name: git-repo-url description: The git repository url - name: git-revision description: The git revision default: release-tech-preview-2 - name: git-repo-name description: The name of the deployment to be created / patched resourcetemplates: - apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: build-deploy-$(params.git-repo-name)-$(uid) spec: serviceAccountName: pipeline pipelineRef: name: build-and-deploy params: - name: deployment-name value: $(tt.params.git-repo-name) - name: git-url value: $(tt.params.git-repo-url) - name: git-revision value: $(tt.params.git-revision) - name: IMAGE value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/$(tt.params.git-repo-name) workspaces: - name: shared-workspace persistentvolumeclaim: claimName: source-pvc
Create the
TriggerTemplate
:$ oc create -f <triggertemplate-yaml-file-name.yaml>
Alternatively, you can create the
TriggerTemplate
directly from thepipelines-tutorial
Git repository:$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-2/03_triggers/02_template.yaml
Copy the contents of the following sample
EventListener
YAML file and save it:apiVersion: triggers.tekton.dev/v1alpha1 kind: EventListener metadata: name: vote-app spec: serviceAccountName: pipeline triggers: - bindings: - ref: vote-app template: name: vote-app
Create the
EventListener
:$ oc create -f <eventlistener-yaml-file-name.yaml>
Alternatively, you can create the
EvenListener
directly from thepipelines-tutorial
Git repository:$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-2/03_triggers/03_event_listener.yaml
Expose the EventListener service as an OpenShift Container Platform route to make it publicly accessible:
$ oc expose svc el-vote-app
4.8. Creating Webhooks
Webhooks are HTTP POST messages that are received by the EventListeners whenever a configured event occurs in your repository. The event payload is then mapped to TriggerBindings, and processed by TriggerTemplates. The TriggerTemplates eventually start one or more PipelineRuns, leading to the creation and deployment of Kubernetes resources.
In this section, you will configure a Webhook URL on your forked Git repositories vote-ui
and vote-api
. This URL points to the publicly accessible EventListener service route.
Adding Webhooks requires administrative privileges to the repository. If you do not have administrative access to your repository, contact your system administrator for adding Webhooks.
Procedure
Get the Webhook URL:
$ echo "URL: $(oc get route el-vote-app --template='http://{{.spec.host}}')"
Note the URL obtained in the output.
Configure Webhooks manually on the front-end repository:
-
Open the front-end Git repository
vote-ui
in your browser. -
Click Settings
Webhooks Add Webhook On the Webhooks/Add Webhook page:
- Enter the Webhook URL from step 1 in Payload URL field
- Select application/json for the Content type
- Specify the secret in the Secret field
- Ensure that the Just the push event is selected
- Select Active
- Click Add Webhook
-
Open the front-end Git repository
-
Repeat step 2 for the back-end repository
vote-api
.
4.9. Triggering a PipelineRun
Whenever a push
event occurs in the Git repository, the configured Webhook sends an event payload to the publicly exposed EventListener service route. The EventListener service of the application processes the payload, and passes it to the relevant TriggerBindings and TriggerTemplates pair. The TriggerBinding extracts the parameters and the TriggerTemplate uses these parameters to create resources. This may rebuild and redeploy the application.
In this section, you push an empty commit to the front-end vote-ui
repository, which then triggers the PipelineRun.
Procedure
From the terminal, clone your forked Git repository
vote-ui
:$ git clone git@github.com:<your GitHub ID>/vote-ui.git -b release-tech-preview-2
Push an empty commit:
$ git commit -m "empty-commit" --allow-empty && git push origin release-tech-preview-2
Check if the PipelineRun was triggered:
$ tkn pipelinerun list
Notice that a new PipelineRun was initiated.
4.10. Additional resources
- For more details on pipelines in the Developer perspective, see the working with Pipelines in the Developer perspective section.
- To learn more about Security Context Constraints (SCCs), see Managing Security Context Constraints section.
- For more examples of reusable Tasks, see the OpenShift Catalog repository. Additionally, you can also see the Tekton Catalog in the Tekton project.