Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.
Chapter 3. Bare metal builds with Red Hat Quay on OpenShift Container Platform
The procedures in this section explain how to create an environment for bare metal builds for Red Hat Quay on OpenShift Container Platform.
3.1. Configuring bare metal builds for Red Hat Quay on OpenShift Container Platform
Use the following procedure to configure bare metal builds for Red Hat Quay on OpenShift Container Platform.
If you are using the Red Hat Quay Operator on OpenShift Container Platform with a managed route
component in your QuayRegistry
CRD, see "Red Hat Quay on OpenShift Container Platform builds limitations with self-managed routes".
Prerequisites
- You have an OpenShift Container Platform cluster provisioned with the Red Hat Quay Operator running.
-
You have set the
tls
component tounmanaged
and uploaded custom SSL/TLS certificates to the Red Hat Quay Operator. For more information, see SSL and TLS for Red Hat Quay. - You are logged into OpenShift Container Platform as a cluster administrator.
Procedure
Enter the following command to create a project where Builds will be run, for example,
bare-metal-builder
:$ oc new-project bare-metal-builder
Create a new
ServiceAccount
in the thebare-metal-builder
namespace by entering the following command:$ oc create sa -n bare-metal-builder quay-builder
Enter the following command to grant a user the
edit
role within thebare-metal-builder
namespace:$ oc policy add-role-to-user -n bare-metal-builder edit system:serviceaccount:bare-metal-builder:quay-builder
Enter the following command to retrieve a token associated with the
quay-builder
service account in thebare-metal-builder
namespace. This token is used to authenticate and interact with the OpenShift Container Platform cluster’s API server.If your OpenShift Container Platform cluster is version 4.11+, enter the following command:
oc create token quay-builder -n bare-metal-builder --duration 24h
If your OpenShift Container Platform cluster is earlier than version 4.11, for example, version 4.10, enter the following command:
$ oc sa get-token -n bare-metal-builder quay-builder
- Identify the URL for the OpenShift Container Platform cluster’s API server. This can be found in the OpenShift Container Platform web console.
Identify a worker node label to be used when scheduling build jobs. Because build pods must run on bare metal worker nodes, typically these are identified with specific labels.
Check with your cluster administrator to determine exactly which node label should be used.
Obtain the Kube API Server’s certificate authority (CA) to add to Red Hat Quay’s extra certificates.
On OpenShift Container Platform versions 4.15+, enter the following commands to obtain the name of the secret containing the CA:
$ oc extract cm/kube-root-ca.crt -n openshift-apiserver
$ mv ca.crt build_cluster.crt
On OpenShift Container Platform versions earlier than 4.15, for example, 4.14, enter the following command:
$ oc get sa openshift-apiserver-sa --namespace=openshift-apiserver -o json | jq '.secrets[] | select(.name | contains("openshift-apiserver-sa-token"))'.name
-
Obtain the
ca.crt
key value from the secret in the OpenShift Container Platform Web Console. The value begins with "-----BEGIN CERTIFICATE-----"`. -
Import the CA to Red Hat Quay. Ensure that the name of this file matches the
K8S_API_TLS_CA
field used in Step 9.
Create the following
SecurityContextConstraints
resource for theServiceAccount
:apiVersion: security.openshift.io/v1 kind: SecurityContextConstraints metadata: name: quay-builder priority: null readOnlyRootFilesystem: false requiredDropCapabilities: null runAsUser: type: RunAsAny seLinuxContext: type: RunAsAny seccompProfiles: - '*' supplementalGroups: type: RunAsAny volumes: - '*' allowHostDirVolumePlugin: true allowHostIPC: true allowHostNetwork: true allowHostPID: true allowHostPorts: true allowPrivilegeEscalation: true allowPrivilegedContainer: true allowedCapabilities: - '*' allowedUnsafeSysctls: - '*' defaultAddCapabilities: null fsGroup: type: RunAsAny --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: quay-builder-scc namespace: bare-metal-builder rules: - apiGroups: - security.openshift.io resourceNames: - quay-builder resources: - securitycontextconstraints verbs: - use --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: quay-builder-scc namespace: bare-metal-builder subjects: - kind: ServiceAccount name: quay-builder roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: quay-builder-scc
Update the
config.yaml
file of your Red Hat Quay on OpenShift Container Platform deployment to include an appropriate bare metal builds configuration by using the OpenShift Container Platform web console.-
Click Operators
Installed Operators Red Hat Quay Quay Registry. - Click the name of your registry, for example, example-registry.
- Under Config Bundle Secret, click the name of your configuration bundle, for example, extra-ca-certificate-config-bundle-secret.
-
Click Actions
Edit Secret. Add the following information to your Red Hat Quay
config.yaml
file, replacing each value with information that is relevant to your specific installation:FEATURE_USER_INITIALIZE: true BROWSER_API_CALLS_XHR_ONLY: false SUPER_USERS: - <superusername> FEATURE_USER_CREATION: false FEATURE_QUOTA_MANAGEMENT: true FEATURE_BUILD_SUPPORT: True BUILDMAN_HOSTNAME: ${BUILDMAN_HOSTNAME}:443 1 BUILD_MANAGER: - ephemeral - ALLOWED_WORKER_COUNT: 10 ORCHESTRATOR_PREFIX: buildman/production/ ORCHESTRATOR: REDIS_HOST: <sample_redis_hostname> 2 REDIS_PASSWORD: "" REDIS_SSL: false REDIS_SKIP_KEYSPACE_EVENT_SETUP: false EXECUTORS: - EXECUTOR: kubernetes BUILDER_NAMESPACE: <sample_builder_namespace> 3 K8S_API_SERVER: <sample_k8s_api_server> 4 K8S_API_TLS_CA: <sample_crt_file> 5 VOLUME_SIZE: 8G KUBERNETES_DISTRIBUTION: openshift CONTAINER_MEMORY_LIMITS: 1G 6 CONTAINER_CPU_LIMITS: 300m 7 CONTAINER_MEMORY_REQUEST: 1G 8 CONTAINER_CPU_REQUEST: 300m 9 NODE_SELECTOR_LABEL_KEY: beta.kubernetes.io/instance-type NODE_SELECTOR_LABEL_VALUE: n1-standard-4 CONTAINER_RUNTIME: podman SERVICE_ACCOUNT_NAME: <sample_service_account_name> SERVICE_ACCOUNT_TOKEN: <sample_account_token> 10 QUAY_USERNAME: <quay_username> QUAY_PASSWORD: <quay_password> WORKER_IMAGE: <registry>/quay-quay-builder WORKER_TAG: <some_tag> BUILDER_VM_CONTAINER_IMAGE: quay.io/quay/quay-builder-qemu-fedoracoreos:latest SETUP_TIME: 180 MINIMUM_RETRY_THRESHOLD: 0 SSH_AUTHORIZED_KEYS: 11 - <ssh-rsa 12345 someuser@email.com> - <ssh-rsa 67890 someuser2@email.com> HTTP_PROXY: <http://10.0.0.1:80> HTTPS_PROXY: <http://10.0.0.1:80> NO_PROXY: <hostname.example.com>
- 1
- Obtained by running the following command:
$ oc get route quayregistry-quay-builder -n ${QUAY_PROJECT} -o jsonpath='{.spec.host}'
. - 2
- The hostname for your Redis service.
- 3
- Set to match the name of your bare metal builds namespace. This example used
bare-metal-builder
. - 4
- The
K8S_API_SERVER
is obtained by running$ oc cluster-info
. - 5
- You must manually create and add your custom CA cert, for example,
K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build-cluster.crt
. - 6
- Defaults to
5120Mi
if left unspecified. - 7
- Defaults to
1000m
if left unspecified. - 8
- Defaults to
3968Mi
if left unspecified. - 9
- Defaults to
500m
if left unspecified. - 10
- Obtained when running
$ oc create sa
. - 11
- Allows public SSH keys to be added to the build environment for remote troubleshooting access. This key, or keys, should correspond to the private key that an admin or developer will use to SSH into the build worker for debugging purposes. This key can be obtained by establishing an SSH connection to the remote host using a specific SSH key and port. For example:
$ ssh -i /path/to/ssh/key/set/in/ssh_authorized_keys -p 9999 core@localhost
.
-
Click Operators
- Restart your Red Hat Quay registry to enable the builds feature.
3.1.1. Red Hat Quay on OpenShift Container Platform builds limitations with self-managed routes
The following limitations apply when you are using the Red Hat Quay Operator on OpenShift Container Platform with a managed route
component:
- Currently, OpenShift Container Platform routes are only able to serve traffic to a single port. Additional steps are required to set up Red Hat Quay Builds.
-
Ensure that your
kubectl
oroc
CLI tool is configured to work with the cluster where the Red Hat Quay Operator is installed and that yourQuayRegistry
exists; theQuayRegistry
does not have to be on the same bare metal cluster where builders run. - Ensure that HTTP/2 ingress is enabled on the OpenShift cluster by following these steps.
The Red Hat Quay Operator creates a
Route
resource that directs gRPC traffic to the Build manager server running inside of the existingQuay
pod, or pods. If you want to use a custom hostname, or a subdomain like<builder-registry.example.com>
, ensure that you create a CNAME record with your DNS provider that points to thestatus.ingress[0].host
of the createRoute
resource. For example:$ kubectl get -n <namespace> route <quayregistry-name>-quay-builder -o jsonpath={.status.ingress[0].host}
Using the OpenShift Container Platform UI or CLI, update the
Secret
referenced byspec.configBundleSecret
of theQuayRegistry
with the build cluster CA certificate. Name the keyextra_ca_cert_build_cluster.cert
. Update theconfig.yaml
file entry with the correct values referenced in the build configuration that you created when you configured Red Hat Quay builds, and add theBUILDMAN_HOSTNAME
CONFIGURATION FIELD:BUILDMAN_HOSTNAME: <build-manager-hostname> 1 BUILD_MANAGER: - ephemeral - ALLOWED_WORKER_COUNT: 1 ORCHESTRATOR_PREFIX: buildman/production/ JOB_REGISTRATION_TIMEOUT: 600 ORCHESTRATOR: REDIS_HOST: <quay_redis_host REDIS_PASSWORD: <quay_redis_password> REDIS_SSL: true REDIS_SKIP_KEYSPACE_EVENT_SETUP: false EXECUTORS: - EXECUTOR: kubernetes BUILDER_NAMESPACE: builder ...
- 1
- The externally accessible server hostname which the build jobs use to communicate back to the build manager. Default is the same as
SERVER_HOSTNAME
. For an OpenShiftroute
resource, it is eitherstatus.ingress[0].host
or the CNAME entry if using a custom hostname.BUILDMAN_HOSTNAME
must include the port number, for example,somehost:443
for an OpenShift Container Platformroute
resource, as the gRPC client used to communicate with the build manager does not infer any port if omitted.