Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 16. Red Hat Quay build enhancements

Prior to Red Hat Quay 3.7, Quay ran podman commands in virtual machines launched by pods. Running builds on virtual platforms requires enabling nested virtualization, which is not featured in Red Hat Enterprise Linux or OpenShift Container Platform. As a result, builds had to run on bare-metal clusters, which is an inefficient use of resources.

With Red Hat Quay 3.7., the bare-metal constraint required to run builds has been removed by adding an additional build option which does not contain the virtual machine layer. As a result, builds can be run on virtualized platforms. Backwards compatibility to run previous build configurations are also available.

16.1. Red Hat Quay enhanced build architecture

The preceding image shows the expected design flow and architecture of the enhanced build features:

Enhanced Quay builds architecture

With this enhancement, the build manager first creates the Job Object. Then, the Job Object then creates a pod using the quay-builder-image. The quay-builder-image will contain the quay-builder binary and the Podman service. The created pod runs as unprivileged. The quay-builder binary then builds the image while communicating status and retrieving build information from the Build Manager.

16.2. Red Hat Quay build limitations

Running builds in Red Hat Quay in an unprivileged context might cause some commands that were working under the previous build strategy to fail. Attempts to change the build strategy could potentially cause performance issues and reliability with the build.

Running builds directly in a container will not have the same isolation as using virtual machines. Changing the build environment might also caused builds that were previously working to fail.

16.3. Creating a Red Hat Quay builders environment with OpenShift Container Platform

The procedures in this section explain how to create a Red Hat Quay virtual builders environment with OpenShift Container Platform.

16.3.1. OpenShift Container Platform TLS component

The tls component allows you to control TLS configuration.

Note

Red Hat Quay 3.8 does not support builders when the TLS component is managed by the Operator.

If you set tls to unmanaged, you supply your own ssl.cert and ssl.key files. In this instance, if you want your cluster to support builders, you must add both the Quay route and the builder route name to the SAN list in the cert, or use a wildcard.

To add the builder route, use the following format: 

[quayregistry-cr-name]-quay-builder-[ocp-namespace].[ocp-domain-name]:443
Copy to Clipboard

16.3.2. Using OpenShift Container Platform for Red Hat Quay builders

Builders require SSL/TLS certificates. For more information about SSL/TLS certificates, see Adding TLS certificates to the Red Hat Quay container.

If you are using Amazon Web Service (AWS) S3 storage, you must modify your storage bucket in the AWS console, prior to running builders. See "Modifying your AWS S3 storage bucket" in the following section for the required parameters.

16.3.2.1. Preparing OpenShift Container Platform for virtual builders

Use the following procedure to prepare OpenShift Container Platform for Red Hat Quay virtual builders.

Note
  • This procedure assumes you already have a cluster provisioned and a Quay Operator running.
  • This procedure is for setting up a virtual namespace on OpenShift Container Platform.

Procedure

  1. Log in to your Red Hat Quay cluster using a cluster administrator account.

  2. Create a new project where your virtual builders will be run, for example, virtual-builders, by running the following command: 

    $ oc new-project virtual-builders
    Copy to Clipboard

  3. Create a ServiceAccount in the project that will be used to run builds by entering the following command: 

    $ oc create sa -n virtual-builders quay-builder
    Copy to Clipboard

  4. Provide the created service account with editing permissions so that it can run the build: 

    $ oc adm policy -n virtual-builders add-role-to-user edit system:serviceaccount:virtual-builders:quay-builder
    Copy to Clipboard

  5. Grant the Quay builder anyuid scc permissions by entering the following command: 

    $ oc adm policy -n virtual-builders add-scc-to-user anyuid -z quay-builder
    Copy to Clipboard
    Note

    This action requires cluster admin privileges. This is required because builders must run as the Podman user for unprivileged or rootless builds to work.

  6. Obtain the token for the Quay builder service account.

    1. If using OpenShift Container Platform 4.10 or an earlier version, enter the following command: 

      oc sa get-token -n virtual-builders quay-builder
      Copy to Clipboard

    2. If using OpenShift Container Platform 4.11 or later, enter the following command: 

      $ oc create token quay-builder -n virtual-builders
      Copy to Clipboard

      Example output

      eyJhbGciOiJSUzI1NiIsImtpZCI6IldfQUJkaDVmb3ltTHZ0dGZMYjhIWnYxZTQzN2dJVEJxcDJscldSdEUtYWsifQ...
      Copy to Clipboard

  7. Determine the builder route by entering the following command: 

    $ oc get route -n quay-enterprise
    Copy to Clipboard

    Example output

    NAME                                  HOST/PORT                                                                    PATH   SERVICES                              PORT   TERMINATION     WILDCARD
...
example-registry-quay-builder         example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org                example-registry-quay-app             grpc   edge/Redirect   None
...
    Copy to Clipboard

  8. Generate a self-signed SSL/TlS certificate with the .crt extension by entering the following command: 

    $ oc extract cm/kube-root-ca.crt -n openshift-apiserver
    Copy to Clipboard

    Example output

    ca.crt
    Copy to Clipboard

  9. Rename the ca.crt file to extra_ca_cert_build_cluster.crt by entering the following command: 

    $ mv ca.crt extra_ca_cert_build_cluster.crt
    Copy to Clipboard

  10. Locate the secret for you configuration bundle in the Console, and select Actions Edit Secret and add the appropriate builder configuration: 

    FEATURE_USER_INITIALIZE: true
BROWSER_API_CALLS_XHR_ONLY: false
SUPER_USERS:
- <superusername>
FEATURE_USER_CREATION: false
FEATURE_QUOTA_MANAGEMENT: true
FEATURE_BUILD_SUPPORT: True
BUILDMAN_HOSTNAME: <sample_build_route>
    1 
    
BUILD_MANAGER:
  - ephemeral
  - ALLOWED_WORKER_COUNT: 1
    ORCHESTRATOR_PREFIX: buildman/production/
    JOB_REGISTRATION_TIMEOUT: 3600
    2 
    
    ORCHESTRATOR:
      REDIS_HOST: <sample_redis_hostname>
    3 
    
      REDIS_PASSWORD: ""
      REDIS_SSL: false
      REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
    EXECUTORS:
      - EXECUTOR: kubernetesPodman
        NAME: openshift
        BUILDER_NAMESPACE: <sample_builder_namespace>
    4 
    
        SETUP_TIME: 180
        MINIMUM_RETRY_THRESHOLD: 0
        BUILDER_CONTAINER_IMAGE: <sample_builder_container_image>
    5 
    
        # Kubernetes resource options
        K8S_API_SERVER: <sample_k8s_api_server>
    6 
    
        K8S_API_TLS_CA: <sample_crt_file>
    7 
    
        VOLUME_SIZE: 8G
        KUBERNETES_DISTRIBUTION: openshift
        CONTAINER_MEMORY_LIMITS: 300m
    8 
    
        CONTAINER_CPU_LIMITS: 1G
    9 
    
        CONTAINER_MEMORY_REQUEST: 300m
    10 
    
        CONTAINER_CPU_REQUEST: 1G
    11 
    
        NODE_SELECTOR_LABEL_KEY: ""
        NODE_SELECTOR_LABEL_VALUE: ""
        SERVICE_ACCOUNT_NAME: <sample_service_account_name>
        SERVICE_ACCOUNT_TOKEN: <sample_account_token>
    12
    Copy to Clipboard
    1
    The build route is obtained by running oc get route -n with the name of your OpenShift Operator’s namespace. A port must be provided at the end of the route, and it should use the following format: [quayregistry-cr-name]-quay-builder-[ocp-namespace].[ocp-domain-name]:443.
    2
    If the JOB_REGISTRATION_TIMEOUT parameter is set too low, you might receive the following error: failed to register job to build manager: rpc error: code = Unauthenticated desc = Invalid build token: Signature has expired. It is suggested that this parameter be set to at least 240.
    3
    If your Redis host has a password or SSL/TLS certificates, you must update accordingly.
    4
    Set to match the name of your virtual builders namespace, for example, virtual-builders.
    5
    For early access, the BUILDER_CONTAINER_IMAGE is currently quay.io/projectquay/quay-builder:3.7.0-rc.2. Note that this might change during the early access window. If this happens, customers are alerted.
    6
    The K8S_API_SERVER is obtained by running oc cluster-info.
    7
    You must manually create and add your custom CA cert, for example, K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt.
    8
    Defaults to 5120Mi if left unspecified.
    9
    For virtual builds, you must ensure that there are enough resources in your cluster. Defaults to 1000m if left unspecified.
    10
    Defaults to 3968Mi if left unspecified.
    11
    Defaults to 500m if left unspecified.
    12
    Obtained when running oc create sa.

    Sample configuration

    FEATURE_USER_INITIALIZE: true
BROWSER_API_CALLS_XHR_ONLY: false
SUPER_USERS:
- quayadmin
FEATURE_USER_CREATION: false
FEATURE_QUOTA_MANAGEMENT: true
FEATURE_BUILD_SUPPORT: True
BUILDMAN_HOSTNAME: example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org:443
BUILD_MANAGER:
  - ephemeral
  - ALLOWED_WORKER_COUNT: 1
    ORCHESTRATOR_PREFIX: buildman/production/
    JOB_REGISTRATION_TIMEOUT: 3600
    ORCHESTRATOR:
      REDIS_HOST: example-registry-quay-redis
      REDIS_PASSWORD: ""
      REDIS_SSL: false
      REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
    EXECUTORS:
      - EXECUTOR: kubernetesPodman
        NAME: openshift
        BUILDER_NAMESPACE: virtual-builders
        SETUP_TIME: 180
        MINIMUM_RETRY_THRESHOLD: 0
        BUILDER_CONTAINER_IMAGE: quay.io/projectquay/quay-builder:3.7.0-rc.2
        # Kubernetes resource options
        K8S_API_SERVER: api.docs.quayteam.org:6443
        K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt
        VOLUME_SIZE: 8G
        KUBERNETES_DISTRIBUTION: openshift
        CONTAINER_MEMORY_LIMITS: 1G
        CONTAINER_CPU_LIMITS: 1080m
        CONTAINER_MEMORY_REQUEST: 1G
        CONTAINER_CPU_REQUEST: 580m
        NODE_SELECTOR_LABEL_KEY: ""
        NODE_SELECTOR_LABEL_VALUE: ""
        SERVICE_ACCOUNT_NAME: quay-builder
        SERVICE_ACCOUNT_TOKEN: "eyJhbGciOiJSUzI1NiIsImtpZCI6IldfQUJkaDVmb3ltTHZ0dGZMYjhIWnYxZTQzN2dJVEJxcDJscldSdEUtYWsifQ"
    Copy to Clipboard

16.3.2.2. Manually adding SSL/TLS certificates

Due to a known issue with the configuration tool, you must manually add your custom SSL/TLS certificates to properly run builders. Use the following procedure to manually add custom SSL/TLS certificates.

For more information creating SSL/TLS certificates, see Adding TLS certificates to the Red Hat Quay container.

16.3.2.2.1. Creating and signing certificates

Use the following procedure to create and sign an SSL/TLS certificate.

Procedure

  • Create a certificate authority and sign a certificate. For more information, see Create a Certificate Authority and sign a certificate.

    openssl.cnf

    [req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = example-registry-quay-quay-enterprise.apps.docs.quayteam.org
    1 
    
DNS.2 = example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org
    2
    Copy to Clipboard

    1
    An alt_name for the URL of your Red Hat Quay registry must be included.
    2
    An alt_name for the BUILDMAN_HOSTNAME

    Sample commands

    $ openssl genrsa -out rootCA.key 2048
$ openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem
$ openssl genrsa -out ssl.key 2048
$ openssl req -new -key ssl.key -out ssl.csr
$ openssl x509 -req -in ssl.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out ssl.cert -days 356 -extensions v3_req -extfile openssl.cnf
    Copy to Clipboard

16.3.2.2.2. Setting TLS to unmanaged

Use the following procedure to set king:tls to unmanaged.

Procedure

  1. In your Red Hat Quay Registry YAML, set kind: tls to managed: false: 

      - kind: tls
    managed: false
    Copy to Clipboard

  2. On the Events page, the change is blocked until you set up the appropriate config.yaml file. For example: 

        - lastTransitionTime: '2022-03-28T12:56:49Z'
      lastUpdateTime: '2022-03-28T12:56:49Z'
      message: >-
        required component `tls` marked as unmanaged, but `configBundleSecret`
        is missing necessary fields
      reason: ConfigInvalid
      status: 'True'
    Copy to Clipboard
16.3.2.2.3. Creating temporary secrets

Use the following procedure to create temporary secrets for the CA certificate.

Procedure

  1. Create a secret in your default namespace for the CA certificate: 

    $ oc create secret generic -n quay-enterprise temp-crt --from-file extra_ca_cert_build_cluster.crt
    Copy to Clipboard

  2. Create a secret in your default namespace for the ssl.key and ssl.cert files: 

    $ oc create secret generic -n quay-enterprise quay-config-ssl --from-file ssl.cert --from-file ssl.key
    Copy to Clipboard
16.3.2.2.4. Copying secret data to the configuration YAML

Use the following procedure to copy secret data to your config.yaml file.

Procedure

  1. Locate the new secrets in the console UI at Workloads Secrets.

  2. For each secret, locate the YAML view: 

    kind: Secret
apiVersion: v1
metadata:
  name: temp-crt
  namespace: quay-enterprise
  uid: a4818adb-8e21-443a-a8db-f334ace9f6d0
  resourceVersion: '9087855'
  creationTimestamp: '2022-03-28T13:05:30Z'
...
data:
  extra_ca_cert_build_cluster.crt: >-
    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURNakNDQWhxZ0F3SUJBZ0l....
type: Opaque
    Copy to Clipboard 
    kind: Secret
apiVersion: v1
metadata:
  name: quay-config-ssl
  namespace: quay-enterprise
  uid: 4f5ae352-17d8-4e2d-89a2-143a3280783c
  resourceVersion: '9090567'
  creationTimestamp: '2022-03-28T13:10:34Z'
...
data:
  ssl.cert: >-
    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVaakNDQTA2Z0F3SUJBZ0lVT...
  ssl.key: >-
    LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NBUUVBc...
type: Opaque
    Copy to Clipboard

  3. Locate the secret for your Red Hat Quay registry configuration bundle in the UI, or through the command line by running a command like the following: 

    $ oc get quayregistries.quay.redhat.com -o jsonpath="{.items[0].spec.configBundleSecret}{'\n'}"  -n quay-enterprise
    Copy to Clipboard

  4. In the OpenShift Container Platform console, select the YAML tab for your configuration bundle secret, and add the data from the two secrets you created: 

    kind: Secret
apiVersion: v1
metadata:
  name: init-config-bundle-secret
  namespace: quay-enterprise
  uid: 4724aca5-bff0-406a-9162-ccb1972a27c1
  resourceVersion: '4383160'
  creationTimestamp: '2022-03-22T12:35:59Z'
...
data:
  config.yaml: >-
    RkVBVFVSRV9VU0VSX0lOSVRJQUxJWkU6IHRydWUKQlJ...
  extra_ca_cert_build_cluster.crt: >-
    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURNakNDQWhxZ0F3SUJBZ0ldw....
  ssl.cert: >-
    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVaakNDQTA2Z0F3SUJBZ0lVT...
  ssl.key: >-
    LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NBUUVBc...
type: Opaque
    Copy to Clipboard
  5. Click Save.

  6. Enter the following command to see if your pods are restarting: 

    $ oc get pods -n quay-enterprise
    Copy to Clipboard

    Example output

    NAME                                                   READY   STATUS              RESTARTS   AGE
...
example-registry-quay-app-6786987b99-vgg2v             0/1     ContainerCreating   0          2s
example-registry-quay-app-7975d4889f-q7tvl             1/1     Running             0          5d21h
example-registry-quay-app-7975d4889f-zn8bb             1/1     Running             0          5d21h
example-registry-quay-app-upgrade-lswsn                0/1     Completed           0          6d1h
example-registry-quay-config-editor-77847fc4f5-nsbbv   0/1     ContainerCreating   0          2s
example-registry-quay-config-editor-c6c4d9ccd-2mwg2    1/1     Running             0          5d21h
example-registry-quay-database-66969cd859-n2ssm        1/1     Running             0          6d1h
example-registry-quay-mirror-764d7b68d9-jmlkk          1/1     Terminating         0          5d21h
example-registry-quay-mirror-764d7b68d9-jqzwg          1/1     Terminating         0          5d21h
example-registry-quay-redis-7cc5f6c977-956g8           1/1     Running             0          5d21h
    Copy to Clipboard

  7. After your Red Hat Quay registry has reconfigured, enter the following command to check if the Red Hat Quay app pods are running: 

    $ oc get pods -n quay-enterprise
    Copy to Clipboard

    Example output

    example-registry-quay-app-6786987b99-sz6kb             1/1     Running            0          7m45s
example-registry-quay-app-6786987b99-vgg2v             1/1     Running            0          9m1s
example-registry-quay-app-upgrade-lswsn                0/1     Completed          0          6d1h
example-registry-quay-config-editor-77847fc4f5-nsbbv   1/1     Running            0          9m1s
example-registry-quay-database-66969cd859-n2ssm        1/1     Running            0          6d1h
example-registry-quay-mirror-758fc68ff7-5wxlp          1/1     Running            0          8m29s
example-registry-quay-mirror-758fc68ff7-lbl82          1/1     Running            0          8m29s
example-registry-quay-redis-7cc5f6c977-956g8           1/1     Running            0          5d21h
    Copy to Clipboard

  8. In your browser, access the registry endpoint and validate that the certificate has been updated appropriately. For example: 

    Common Name (CN)	example-registry-quay-quay-enterprise.apps.docs.quayteam.org
Organisation (O)	DOCS
Organisational Unit (OU)	QUAY
    Copy to Clipboard

16.3.2.3. Using the UI to create a build trigger

Use the following procedure to use the UI to create a build trigger.

Procedure

  1. Log in to your Red Hat Quay repository.
  2. Click Create New Repository and create a new registry, for example, testrepo.

  3. On the Repositories page, click the Builds tab on the navigation pane. Alternatively, use the corresponding URL directly: 

    https://example-registry-quay-quay-enterprise.apps.docs.quayteam.org/repository/quayadmin/testrepo?tab=builds
    Copy to Clipboard
    Important

    In some cases, the builder might have issues resolving hostnames. This issue might be related to the dnsPolicy being set to default on the job object. Currently, there is no workaround for this issue. It will be resolved in a future version of Red Hat Quay.

  4. Click Create Build Trigger Custom Git Repository Push.

  5. Enter the HTTPS or SSH style URL used to clone your Git repository, then click Continue. For example: 

    https://github.com/gabriel-rh/actions_test.git
    Copy to Clipboard
  6. Check Tag manifest with the branch or tag name and then click Continue.
  7. Enter the location of the Dockerfile to build when the trigger is invoked, for example, /Dockerfile and click Continue.
  8. Enter the location of the context for the Docker build, for example, /, and click Continue.
  9. If warranted, create a Robot Account. Otherwise, click Continue.
  10. Click Continue to verify the parameters.
  11. On the Builds page, click Options icon of your Trigger Name, and then click Run Trigger Now.
  12. Enter a commit SHA from the Git repository and click Start Build.

  13. You can check the status of your build by clicking the commit in the Build History page, or by running oc get pods -n virtual-builders. For example: 

    $ oc get pods -n virtual-builders
    Copy to Clipboard

    Example output

    NAME                                               READY   STATUS    RESTARTS   AGE
f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Running   0          7s
    Copy to Clipboard 

    $ oc get pods -n virtual-builders
    Copy to Clipboard

    Example output

    NAME                                               READY   STATUS        RESTARTS   AGE
f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Terminating   0          9s
    Copy to Clipboard 

    $ oc get pods -n virtual-builders
    Copy to Clipboard

    Example output

    No resources found in virtual-builders namespace.
    Copy to Clipboard

  14. When the build is finished, you can check the status of the tag under Tags on the navigation pane.

    Note

    With early access, full build logs and timestamps of builds are currently unavailable.

16.3.2.4. Modifying your AWS S3 storage bucket

If you are using AWS S3 storage, you must change your storage bucket in the AWS console, prior to running builders.

Procedure

  1. Log in to your AWS console at s3.console.aws.com.
  2. In the search bar, search for S3 and then click S3.
  3. Click the name of your bucket, for example, myawsbucket.
  4. Click the Permissions tab.

  5. Under Cross-origin resource sharing (CORS), include the following parameters: 

      [
      {
          "AllowedHeaders": [
              "Authorization"
          ],
          "AllowedMethods": [
              "GET"
          ],
          "AllowedOrigins": [
              "*"
          ],
          "ExposeHeaders": [],
          "MaxAgeSeconds": 3000
      },
      {
          "AllowedHeaders": [
              "Content-Type",
              "x-amz-acl",
              "origin"
          ],
          "AllowedMethods": [
              "PUT"
          ],
          "AllowedOrigins": [
              "*"
          ],
          "ExposeHeaders": [],
          "MaxAgeSeconds": 3000
      }
  ]
    Copy to Clipboard

16.3.2.5. Modifying your Google Cloud Platform object bucket

Use the following procedure to configure cross-origin resource sharing (CORS) for virtual builders.

Note

Without CORS configuration, uploading a build Dockerfile fails.

Procedure

  1. Use the following reference to create a JSON file for your specific CORS needs. For example: 

    $ cat gcp_cors.json
    Copy to Clipboard

    Example output

    [
    {
      "origin": ["*"],
      "method": ["GET"],
      "responseHeader": ["Authorization"],
      "maxAgeSeconds": 3600
    },
    {
      "origin": ["*"],
      "method": ["PUT"],
      "responseHeader": [
              "Content-Type",
              "x-goog-acl",
              "origin"],
      "maxAgeSeconds": 3600
    }
]
    Copy to Clipboard

  2. Enter the following command to update your GCP storage bucket: 

    $ gcloud storage buckets update gs://<bucket_name> --cors-file=./gcp_cors.json
    Copy to Clipboard

    Example output

    Updating
  Completed 1
    Copy to Clipboard

  3. You can display the updated CORS configuration of your GCP bucket by running the following command: 

    $ gcloud storage buckets describe gs://<bucket_name>  --format="default(cors)"
    Copy to Clipboard

    Example output

    cors:
- maxAgeSeconds: 3600
  method:
  - GET
  origin:
  - '*'
  responseHeader:
  - Authorization
- maxAgeSeconds: 3600
  method:
  - PUT
  origin:
  - '*'
  responseHeader:
  - Content-Type
  - x-goog-acl
  - origin
    Copy to Clipboard

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat