Chapter 11. Automatically building Dockerfiles with Build workers

Procedure

1. Enter the following command to create a project where Builds will be run, for example, builder:

   $ oc new-project builder

2. Create a new ServiceAccount in the the builder namespace by entering the following command:

   $ oc create sa -n builder quay-builder

3. Enter the following command to grant a user the edit role within the builder namespace:

   $ oc policy add-role-to-user -n builder edit system:serviceaccount:builder:quay-builder

4. Enter the following command to retrieve a token associated with the quay-builder service account in the builder namespace. This token is used to authenticate and interact with the OpenShift Container Platform cluster’s API server.

   $ oc sa get-token -n builder quay-builder

5. Identify the URL for the OpenShift Container Platform cluster’s API server. This can be found in the OpenShift Container Platform Web Console.

6. Identify a worker node label to be used when schedule Build jobs. Because Build pods need to run on bare metal worker nodes, typically these are identified with specific labels.

   Check with your cluster administrator to determine exactly which node label should be used.

7. Optional. If the cluster is using a self-signed certificate, you must get the Kube API Server’s certificate authority (CA) to add to Red Hat Quay’s extra certificates.

   1. Enter the following command to obtain the name of the secret containing the CA:

      $ oc get sa openshift-apiserver-sa --namespace=openshift-apiserver -o json | jq '.secrets[] | select(.name | contains("openshift-apiserver-sa-token"))'.name

   2. Obtain the ca.crt key value from the secret in the OpenShift Container Platform Web Console. The value begins with "-----BEGIN CERTIFICATE-----"`.
   3. Import the CA to Red Hat Quay. Ensure that the name of this file matches K8S_API_TLS_CA.

8. Create the following SecurityContextConstraints resource for the ServiceAccount:

   apiVersion: security.openshift.io/v1
   kind: SecurityContextConstraints
   metadata:
     name: quay-builder
   priority: null
   readOnlyRootFilesystem: false
   requiredDropCapabilities: null
   runAsUser:
     type: RunAsAny
   seLinuxContext:
     type: RunAsAny
   seccompProfiles:
   - '*'
   supplementalGroups:
     type: RunAsAny
   volumes:
   - '*'
   allowHostDirVolumePlugin: true
   allowHostIPC: true
   allowHostNetwork: true
   allowHostPID: true
   allowHostPorts: true
   allowPrivilegeEscalation: true
   allowPrivilegedContainer: true
   allowedCapabilities:
   - '*'
   allowedUnsafeSysctls:
   - '*'
   defaultAddCapabilities: null
   fsGroup:
     type: RunAsAny
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: quay-builder-scc
     namespace: builder
   rules:
   - apiGroups:
     - security.openshift.io
     resourceNames:
     - quay-builder
     resources:
     - securitycontextconstraints
     verbs:
     - use
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: quay-builder-scc
     namespace: builder
   subjects:
   - kind: ServiceAccount
     name: quay-builder
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: quay-builder-scc

Procedure

1. Ensure that your Red Hat Quay config.yaml file has Builds enabled, for example:

   FEATURE_BUILD_SUPPORT: True

2. Add the following information to your Red Hat Quay config.yaml file, replacing each value with information that is relevant to your specific installation:

   BUILD_MANAGER:
   - ephemeral
   - ALLOWED_WORKER_COUNT: 1
     ORCHESTRATOR_PREFIX: buildman/production/
     ORCHESTRATOR:
       REDIS_HOST: quay-redis-host
       REDIS_PASSWORD: quay-redis-password
       REDIS_SSL: true
       REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
     EXECUTORS:
     - EXECUTOR: kubernetes
       BUILDER_NAMESPACE: builder
       K8S_API_SERVER: api.openshift.somehost.org:6443
       K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt
       VOLUME_SIZE: 8G
       KUBERNETES_DISTRIBUTION: openshift
       CONTAINER_MEMORY_LIMITS: 1G
       CONTAINER_CPU_LIMITS: 300m
       CONTAINER_MEMORY_REQUEST: 1G
       CONTAINER_CPU_REQUEST: 300m
       NODE_SELECTOR_LABEL_KEY: beta.kubernetes.io/instance-type
       NODE_SELECTOR_LABEL_VALUE: n1-standard-4
       CONTAINER_RUNTIME: podman
       SERVICE_ACCOUNT_NAME: *****
       SERVICE_ACCOUNT_TOKEN: *****
       QUAY_USERNAME: quay-username
       QUAY_PASSWORD: quay-password
       WORKER_IMAGE: <registry>/quay-quay-builder
       WORKER_TAG: some_tag
       BUILDER_VM_CONTAINER_IMAGE: <registry>/quay-quay-builder-qemu-rhcos:v3.4.0
       SETUP_TIME: 180
       MINIMUM_RETRY_THRESHOLD: 0
       SSH_AUTHORIZED_KEYS:
       - ssh-rsa 12345 someuser@email.com
       - ssh-rsa 67890 someuser2@email.com
       HTTP_PROXY: <http://10.0.0.1:80>
       HTTPS_PROXY: <http://10.0.0.1:80>
       NO_PROXY: <hostname.example.com>

   For more information about each configuration field, see

Procedure

1. Create a port forwarding tunnel between your local machine and a pod running with either an OpenShift Container Platform cluster or a Kubernetes cluster by entering the following command:

   $ oc port-forward <builder_pod> 9999:2222

2. Establish an SSH connection to the remote host using a specified SSH key and port, for example:

   $ ssh -i /path/to/ssh/key/set/in/ssh_authorized_keys -p 9999 core@localhost

3. Obtain the quay-builder service logs by entering the following commands:

   $ systemctl status quay-builder

   $ journalctl -f -u quay-builder