Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.

Chapter 3. Bare metal builds with Red Hat Quay on OpenShift Container Platform


The procedures in this section explain how to create an environment for bare metal builds for Red Hat Quay on OpenShift Container Platform.

3.1. Configuring bare metal builds for Red Hat Quay on OpenShift Container Platform

Use the following procedure to configure bare metal builds for Red Hat Quay on OpenShift Container Platform.

Note

If you are using the Red Hat Quay Operator on OpenShift Container Platform with a managed route component in your QuayRegistry CRD, see "Red Hat Quay on OpenShift Container Platform builds limitations with self-managed routes".

Prerequisites

  • You have an OpenShift Container Platform cluster provisioned with the Red Hat Quay Operator running.
  • You have set the tls component to unmanaged and uploaded custom SSL/TLS certificates to the Red Hat Quay Operator. For more information, see SSL and TLS for Red Hat Quay.
  • You are logged into OpenShift Container Platform as a cluster administrator.

Procedure

  1. Enter the following command to create a project where Builds will be run, for example, bare-metal-builder:

    $ oc new-project bare-metal-builder
  2. Create a new ServiceAccount in the the bare-metal-builder namespace by entering the following command:

    $ oc create sa -n bare-metal-builder quay-builder
  3. Enter the following command to grant a user the edit role within the bare-metal-builder namespace:

    $ oc policy add-role-to-user -n bare-metal-builder edit system:serviceaccount:bare-metal-builder:quay-builder
  4. Enter the following command to retrieve a token associated with the quay-builder service account in the bare-metal-builder namespace. This token is used to authenticate and interact with the OpenShift Container Platform cluster’s API server.

    1. If your OpenShift Container Platform cluster is version 4.11+, enter the following command:

      oc create token quay-builder  -n bare-metal-builder --duration 24h
    2. If your OpenShift Container Platform cluster is earlier than version 4.11, for example, version 4.10, enter the following command:

      $ oc sa get-token -n bare-metal-builder quay-builder
  5. Identify the URL for the OpenShift Container Platform cluster’s API server. This can be found in the OpenShift Container Platform web console.
  6. Identify a worker node label to be used when scheduling build jobs. Because build pods must run on bare metal worker nodes, typically these are identified with specific labels.

    Check with your cluster administrator to determine exactly which node label should be used.

  7. Obtain the Kube API Server’s certificate authority (CA) to add to Red Hat Quay’s extra certificates.

    1. On OpenShift Container Platform versions 4.15+, enter the following commands to obtain the name of the secret containing the CA:

      $ oc extract cm/kube-root-ca.crt -n openshift-apiserver
      $ mv ca.crt build_cluster.crt
    2. On OpenShift Container Platform versions earlier than 4.15, for example, 4.14, enter the following command:

      $ oc get sa openshift-apiserver-sa --namespace=openshift-apiserver -o json | jq '.secrets[] | select(.name | contains("openshift-apiserver-sa-token"))'.name
    3. Obtain the ca.crt key value from the secret in the OpenShift Container Platform Web Console. The value begins with "-----BEGIN CERTIFICATE-----"`.
    4. Import the CA to Red Hat Quay. Ensure that the name of this file matches the K8S_API_TLS_CA field used in Step 9.
  8. Create the following SecurityContextConstraints resource for the ServiceAccount:

    apiVersion: security.openshift.io/v1
    kind: SecurityContextConstraints
    metadata:
      name: quay-builder
    priority: null
    readOnlyRootFilesystem: false
    requiredDropCapabilities: null
    runAsUser:
      type: RunAsAny
    seLinuxContext:
      type: RunAsAny
    seccompProfiles:
    - '*'
    supplementalGroups:
      type: RunAsAny
    volumes:
    - '*'
    allowHostDirVolumePlugin: true
    allowHostIPC: true
    allowHostNetwork: true
    allowHostPID: true
    allowHostPorts: true
    allowPrivilegeEscalation: true
    allowPrivilegedContainer: true
    allowedCapabilities:
    - '*'
    allowedUnsafeSysctls:
    - '*'
    defaultAddCapabilities: null
    fsGroup:
      type: RunAsAny
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      name: quay-builder-scc
      namespace: bare-metal-builder
    rules:
    - apiGroups:
      - security.openshift.io
      resourceNames:
      - quay-builder
      resources:
      - securitycontextconstraints
      verbs:
      - use
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: quay-builder-scc
      namespace: bare-metal-builder
    subjects:
    - kind: ServiceAccount
      name: quay-builder
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: quay-builder-scc
  9. Update the config.yaml file of your Red Hat Quay on OpenShift Container Platform deployment to include an appropriate bare metal builds configuration by using the OpenShift Container Platform web console.

    1. Click Operators Installed Operators Red Hat Quay Quay Registry.
    2. Click the name of your registry, for example, example-registry.
    3. Under Config Bundle Secret, click the name of your configuration bundle, for example, extra-ca-certificate-config-bundle-secret.
    4. Click Actions Edit Secret.
    5. Add the following information to your Red Hat Quay config.yaml file, replacing each value with information that is relevant to your specific installation:

      FEATURE_USER_INITIALIZE: true
      BROWSER_API_CALLS_XHR_ONLY: false
      SUPER_USERS:
      - <superusername>
      FEATURE_USER_CREATION: false
      FEATURE_QUOTA_MANAGEMENT: true
      FEATURE_BUILD_SUPPORT: True
      BUILDMAN_HOSTNAME: ${BUILDMAN_HOSTNAME}:443 1
      BUILD_MANAGER:
      - ephemeral
      - ALLOWED_WORKER_COUNT: 10
        ORCHESTRATOR_PREFIX: buildman/production/
          ORCHESTRATOR:
            REDIS_HOST: <sample_redis_hostname> 2
            REDIS_PASSWORD: ""
            REDIS_SSL: false
            REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
        EXECUTORS:
        - EXECUTOR: kubernetes
          BUILDER_NAMESPACE: <sample_builder_namespace> 3
          K8S_API_SERVER: <sample_k8s_api_server> 4
          K8S_API_TLS_CA: <sample_crt_file> 5
          VOLUME_SIZE: 8G
          KUBERNETES_DISTRIBUTION: openshift
          CONTAINER_MEMORY_LIMITS: 1G 6
          CONTAINER_CPU_LIMITS: 300m 7
          CONTAINER_MEMORY_REQUEST: 1G 8
          CONTAINER_CPU_REQUEST: 300m 9
          NODE_SELECTOR_LABEL_KEY: beta.kubernetes.io/instance-type
          NODE_SELECTOR_LABEL_VALUE: n1-standard-4
          CONTAINER_RUNTIME: podman
          SERVICE_ACCOUNT_NAME: <sample_service_account_name>
          SERVICE_ACCOUNT_TOKEN: <sample_account_token> 10
          QUAY_USERNAME: <quay_username>
          QUAY_PASSWORD: <quay_password>
          WORKER_IMAGE: <registry>/quay-quay-builder
          WORKER_TAG: <some_tag>
          BUILDER_VM_CONTAINER_IMAGE: quay.io/quay/quay-builder-qemu-fedoracoreos:latest
          SETUP_TIME: 180
          MINIMUM_RETRY_THRESHOLD: 0
          SSH_AUTHORIZED_KEYS: 11
          - <ssh-rsa 12345 someuser@email.com>
          - <ssh-rsa 67890 someuser2@email.com>
          HTTP_PROXY: <http://10.0.0.1:80>
          HTTPS_PROXY: <http://10.0.0.1:80>
          NO_PROXY: <hostname.example.com>
      1
      Obtained by running the following command: $ oc get route quayregistry-quay-builder -n ${QUAY_PROJECT} -o jsonpath='{.spec.host}'.
      2
      The hostname for your Redis service.
      3
      Set to match the name of your bare metal builds namespace. This example used bare-metal-builder.
      4
      The K8S_API_SERVER is obtained by running $ oc cluster-info.
      5
      You must manually create and add your custom CA cert, for example, K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build-cluster.crt.
      6
      Defaults to 5120Mi if left unspecified.
      7
      Defaults to 1000m if left unspecified.
      8
      Defaults to 3968Mi if left unspecified.
      9
      Defaults to 500m if left unspecified.
      10
      Obtained when running $ oc create sa.
      11
      Allows public SSH keys to be added to the build environment for remote troubleshooting access. This key, or keys, should correspond to the private key that an admin or developer will use to SSH into the build worker for debugging purposes. This key can be obtained by establishing an SSH connection to the remote host using a specific SSH key and port. For example: $ ssh -i /path/to/ssh/key/set/in/ssh_authorized_keys -p 9999 core@localhost.
  10. Restart your Red Hat Quay registry to enable the builds feature.

3.1.1. Red Hat Quay on OpenShift Container Platform builds limitations with self-managed routes

The following limitations apply when you are using the Red Hat Quay Operator on OpenShift Container Platform with a managed route component:

  • Currently, OpenShift Container Platform routes are only able to serve traffic to a single port. Additional steps are required to set up Red Hat Quay Builds.
  • Ensure that your kubectl or oc CLI tool is configured to work with the cluster where the Red Hat Quay Operator is installed and that your QuayRegistry exists; the QuayRegistry does not have to be on the same bare metal cluster where builders run.
  • Ensure that HTTP/2 ingress is enabled on the OpenShift cluster by following these steps.
  • The Red Hat Quay Operator creates a Route resource that directs gRPC traffic to the Build manager server running inside of the existing Quay pod, or pods. If you want to use a custom hostname, or a subdomain like <builder-registry.example.com>, ensure that you create a CNAME record with your DNS provider that points to the status.ingress[0].host of the create Route resource. For example:

    $ kubectl get -n <namespace> route <quayregistry-name>-quay-builder -o jsonpath={.status.ingress[0].host}
  • Using the OpenShift Container Platform UI or CLI, update the Secret referenced by spec.configBundleSecret of the QuayRegistry with the build cluster CA certificate. Name the key extra_ca_cert_build_cluster.cert. Update the config.yaml file entry with the correct values referenced in the build configuration that you created when you configured Red Hat Quay builds, and add the BUILDMAN_HOSTNAME CONFIGURATION FIELD:

    BUILDMAN_HOSTNAME: <build-manager-hostname> 1
    BUILD_MANAGER:
    - ephemeral
    - ALLOWED_WORKER_COUNT: 1
      ORCHESTRATOR_PREFIX: buildman/production/
      JOB_REGISTRATION_TIMEOUT: 600
      ORCHESTRATOR:
        REDIS_HOST: <quay_redis_host
        REDIS_PASSWORD: <quay_redis_password>
        REDIS_SSL: true
        REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
      EXECUTORS:
      - EXECUTOR: kubernetes
        BUILDER_NAMESPACE: builder
        ...
    1
    The externally accessible server hostname which the build jobs use to communicate back to the build manager. Default is the same as SERVER_HOSTNAME. For an OpenShift route resource, it is either status.ingress[0].host or the CNAME entry if using a custom hostname. BUILDMAN_HOSTNAME must include the port number, for example, somehost:443 for an OpenShift Container Platform route resource, as the gRPC client used to communicate with the build manager does not infer any port if omitted.
Red Hat logoGithubRedditYoutubeTwitter

Lernen

Testen, kaufen und verkaufen

Communitys

Über Red Hat Dokumentation

Wir helfen Red Hat Benutzern, mit unseren Produkten und Diensten innovativ zu sein und ihre Ziele zu erreichen – mit Inhalten, denen sie vertrauen können.

Mehr Inklusion in Open Source

Red Hat hat sich verpflichtet, problematische Sprache in unserem Code, unserer Dokumentation und unseren Web-Eigenschaften zu ersetzen. Weitere Einzelheiten finden Sie in Red Hat Blog.

Über Red Hat

Wir liefern gehärtete Lösungen, die es Unternehmen leichter machen, plattform- und umgebungsübergreifend zu arbeiten, vom zentralen Rechenzentrum bis zum Netzwerkrand.

© 2024 Red Hat, Inc.