Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 3. Bare metal builds with Red Hat Quay on OpenShift Container Platform

Bare metal builds enable you to build container images on physical infrastructure using {product-title} with Red Hat Quay on OpenShift Container Platform. You can create an environment for bare metal builds to leverage your existing hardware resources.

To configure bare metal builds for Red Hat Quay on OpenShift Container Platform with {product-title}, you can create a build project, configure service accounts, and update your configuration file.

Note

If you are using the Red Hat Quay Operator on OpenShift Container Platform with a managed route component in your QuayRegistry CRD, see "Red Hat Quay on OpenShift Container Platform builds limitations with self-managed routes".

Prerequisites

  • You have an OpenShift Container Platform cluster provisioned with the Red Hat Quay Operator running.
  • You have set the tls component to unmanaged and uploaded custom SSL/TLS certificates to the Red Hat Quay Operator. For more information, see SSL and TLS for Red Hat Quay.
  • You are logged into OpenShift Container Platform as a cluster administrator.

Procedure

  1. Enter the following command to create a project where Builds will be run, for example, bare-metal-builder: 

    $ oc new-project bare-metal-builder
    Copy to Clipboard Toggle word wrap

  2. Create a new ServiceAccount in the bare-metal-builder namespace by entering the following command: 

    $ oc create sa -n bare-metal-builder quay-builder
    Copy to Clipboard Toggle word wrap

  3. Enter the following command to grant a user the edit role within the bare-metal-builder namespace: 

    $ oc policy add-role-to-user -n bare-metal-builder edit system:serviceaccount:bare-metal-builder:quay-builder
    Copy to Clipboard Toggle word wrap

  4. Enter the following command to retrieve a token associated with the quay-builder service account in the bare-metal-builder namespace. This token is used to authenticate and interact with the OpenShift Container Platform cluster’s API server.

    1. If your OpenShift Container Platform cluster is version 4.11+, enter the following command: 

      oc create token quay-builder  -n bare-metal-builder --duration 24h
      Copy to Clipboard Toggle word wrap

    2. If your OpenShift Container Platform cluster is earlier than version 4.11, for example, version 4.10, enter the following command: 

      $ oc sa get-token -n bare-metal-builder quay-builder
      Copy to Clipboard Toggle word wrap
  5. Identify the URL for the OpenShift Container Platform cluster’s API server. This can be found in the OpenShift Container Platform web console.

  6. Identify a worker node label to be used when scheduling build jobs. Because build pods must run on bare metal worker nodes, typically these are identified with specific labels.

    Check with your cluster administrator to determine exactly which node label should be used.

  7. Obtain the Kube API Server’s certificate authority (CA) to add to Red Hat Quay’s extra certificates.

    1. On OpenShift Container Platform versions 4.15+, enter the following commands to obtain the name of the secret containing the CA: 

      $ oc extract cm/kube-root-ca.crt -n openshift-apiserver
      Copy to Clipboard Toggle word wrap 
      $ mv ca.crt build_cluster.crt
      Copy to Clipboard Toggle word wrap

    2. On OpenShift Container Platform versions earlier than 4.15, for example, 4.14, enter the following command: 

      $ oc get sa openshift-apiserver-sa --namespace=openshift-apiserver -o json | jq '.secrets[] | select(.name | contains("openshift-apiserver-sa-token"))'.name
      Copy to Clipboard Toggle word wrap
    3. Obtain the ca.crt key value from the secret in the OpenShift Container Platform Web Console. The value begins with "-----BEGIN CERTIFICATE-----"`.
    4. Import the CA to Red Hat Quay. Ensure that the name of this file matches the K8S_API_TLS_CA field used in Step 9.

  8. Create the following SecurityContextConstraints resource for the ServiceAccount: 

    apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: quay-builder
priority: null
readOnlyRootFilesystem: false
requiredDropCapabilities: null
runAsUser:
  type: RunAsAny
seLinuxContext:
  type: RunAsAny
seccompProfiles:
- '*'
supplementalGroups:
  type: RunAsAny
volumes:
- '*'
allowHostDirVolumePlugin: true
allowHostIPC: true
allowHostNetwork: true
allowHostPID: true
allowHostPorts: true
allowPrivilegeEscalation: true
allowPrivilegedContainer: true
allowedCapabilities:
- '*'
allowedUnsafeSysctls:
- '*'
defaultAddCapabilities: null
fsGroup:
  type: RunAsAny
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: quay-builder-scc
  namespace: bare-metal-builder
rules:
- apiGroups:
  - security.openshift.io
  resourceNames:
  - quay-builder
  resources:
  - securitycontextconstraints
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: quay-builder-scc
  namespace: bare-metal-builder
subjects:
- kind: ServiceAccount
  name: quay-builder
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: quay-builder-scc
    Copy to Clipboard Toggle word wrap

  9. Update the config.yaml file of your Red Hat Quay on OpenShift Container Platform deployment to include an appropriate bare metal builds configuration by using the OpenShift Container Platform web console.

    1. Click Operators Installed Operators Red Hat Quay Quay Registry.
    2. Click the name of your registry, for example, example-registry.
    3. Under Config Bundle Secret, click the name of your configuration bundle, for example, extra-ca-certificate-config-bundle-secret.
    4. Click Actions Edit Secret.

    5. Add the following information to your Red Hat Quay config.yaml file, replacing each value with information that is relevant to your specific installation: 

      FEATURE_USER_INITIALIZE: true
BROWSER_API_CALLS_XHR_ONLY: false
SUPER_USERS:
- <superusername>
FEATURE_USER_CREATION: false
FEATURE_QUOTA_MANAGEMENT: true
FEATURE_BUILD_SUPPORT: True
BUILDMAN_HOSTNAME: ${BUILDMAN_HOSTNAME}:443
BUILD_MANAGER:
- ephemeral
- ALLOWED_WORKER_COUNT: 10
  ORCHESTRATOR_PREFIX: buildman/production/
    ORCHESTRATOR:
      REDIS_HOST: <sample_redis_hostname>
      REDIS_PASSWORD: ""
      REDIS_SSL: false
      REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
  EXECUTORS:
  - EXECUTOR: kubernetes
    BUILDER_NAMESPACE: <sample_builder_namespace>
    K8S_API_SERVER: <sample_k8s_api_server>
    K8S_API_TLS_CA: <sample_crt_file>
    VOLUME_SIZE: 8G
    KUBERNETES_DISTRIBUTION: openshift
    CONTAINER_MEMORY_LIMITS: 1G
    CONTAINER_CPU_LIMITS: 300m
    CONTAINER_MEMORY_REQUEST: 1G
    CONTAINER_CPU_REQUEST: 300m
    NODE_SELECTOR_LABEL_KEY: beta.kubernetes.io/instance-type
    NODE_SELECTOR_LABEL_VALUE: n1-standard-4
    CONTAINER_RUNTIME: podman
    SERVICE_ACCOUNT_NAME: <sample_service_account_name>
    SERVICE_ACCOUNT_TOKEN: <sample_account_token>
    QUAY_USERNAME: <quay_username>
    QUAY_PASSWORD: <quay_password>
    WORKER_IMAGE: <registry>/quay-quay-builder
    WORKER_TAG: <some_tag>
    BUILDER_VM_CONTAINER_IMAGE: registry.redhat.io/quay/quay-builder-qemu-rhcos-rhel8:v3.9.10-4
    SETUP_TIME: 180
    MINIMUM_RETRY_THRESHOLD: 0
    SSH_AUTHORIZED_KEYS:
    - <ssh-rsa 12345 someuser@email.com>
    - <ssh-rsa 67890 someuser2@email.com>
    HTTP_PROXY: <http://10.0.0.1:80>
    HTTPS_PROXY: <http://10.0.0.1:80>
    NO_PROXY: <hostname.example.com>
      Copy to Clipboard Toggle word wrap

      where:

      BUILDMAN_HOSTNAME:: Specifies the hostname of the Red Hat Quay registry. Obtain this by running the following command: $ oc get route quayregistry-quay-builder -n ${QUAY_PROJECT} -o jsonpath='{.spec.host}'.

      REDIS_HOST:: Specifies the hostname for your Redis service.

      BUILDER_NAMESPACE:: Specifies the name of your bare metal builds namespace. This example used bare-metal-builder.

      K8S_API_SERVER:: Specifies the K8S_API_SERVER is obtained by running $ oc cluster-info.

      K8S_API_TLS_CA:: Specifies the name of your custom CA cert, for example, K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build-cluster.crt.

      CONTAINER_MEMORY_LIMITS:: Specifies the memory limit for your container. Defaults to 5120Mi if left unspecified.

      CONTAINER_CPU_LIMITS:: Specifies the CPU limit for your container. Defaults to 1000m if left unspecified.

      CONTAINER_MEMORY_REQUEST:: Specifies the memory request for your container. Defaults to 3968Mi if left unspecified.

      CONTAINER_CPU_REQUEST:: Specifies the CPU request for your container. Defaults to 500m if left unspecified.

      SERVICE_ACCOUNT_TOKEN:: Specifies the token for your service account. Obtain this by running $ oc create sa.

      SSH_AUTHORIZED_KEYS:: Specifies the SSH authorized keys for your build environment. This key, or keys, should correspond to the private key that an admin or developer will use to SSH into the build worker for debugging purposes. This key can be obtained by establishing an SSH connection to the remote host using a specific SSH key and port. For example: $ ssh -i /path/to/ssh/key/set/in/ssh_authorized_keys -p 9999 core@localhost.

  10. Restart your Red Hat Quay registry to enable the builds feature.

The {product-title} builds feature has specific networking constraints when using managed routes on Red Hat Quay on OpenShift Container Platform. Understanding these limitations helps you ensure successful communication between build executors and the build manager.

OpenShift Container Platform routes are typically restricted to serving traffic to a single port. Because Red Hat Quay Builds require gRPC communication, the Operator creates a specific Route resource to direct this traffic to the build manager server.

Key considerations for this architecture include:

  • Insecure vs. Secure Traffic: OpenShift ingress must have HTTP/2 enabled to support the gRPC protocol used by the build manager.
  • Identity and Trust: The build manager requires the CA certificate of the build cluster to be present in the Red Hat Quay configuration to establish a secure connection.
  • Hostname Resolution: Build jobs must be able to resolve the build manager’s hostname. If using a custom subdomain, specific DNS mapping is required to point to the OpenShift router.

See "Configuring Red Hat Quay builds for managed routes" for more information.

To use {product-title} builds with managed routes and custom hostnames, you can configure DNS records and update your registry configuration. This enables gRPC communication between build executors and the build manager.

Prerequisites

  • The Red Hat Quay Operator is installed and a QuayRegistry exists.
  • Your kubectl or oc CLI tool is configured for the target cluster.

Procedure

  1. Enable HTTP/2 ingress on your OpenShift Container Platform cluster to support gRPC.

  2. Retrieve the host address of the generated build-manager route: 

    $ kubectl get -n <namespace> route <quayregistry-name>-quay-builder -o jsonpath={.status.ingress[0].host}
    Copy to Clipboard Toggle word wrap
  3. Create a CNAME record with your DNS provider that points your custom hostname (for example, builder-registry.example.com) to the route host retrieved in the previous step.
  4. Update the Secret referenced by spec.configBundleSecret in your QuayRegistry to include the build cluster CA certificate. The key must be named extra_ca_cert_build_cluster.cert.

  5. Add the BUILDMAN_HOSTNAME field to your config.yaml and include the port number: 

    BUILDMAN_HOSTNAME: builder-registry.example.com:443
BUILD_MANAGER:
- ephemeral
  ALLOWED_WORKER_COUNT: 1
  ...
    Copy to Clipboard Toggle word wrap
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top