Red Hat Quay Operator features

Procedure

1. Create a Quay configuration bundle secret that includes the clair-config.yaml by entering the following command:

   $ oc create secret generic --from-file config.yaml=./config.yaml --from-file extra_ca_cert_rds-ca-2019-root.pem=./rds-ca-2019-root.pem --from-file clair-config.yaml=./clair-config.yaml --from-file ssl.cert=./ssl.cert --from-file ssl.key=./ssl.key config-bundle-secret

   Copy to Clipboard Toggle word wrap

   Example Clair config.yaml file

   indexer:
       connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslrootcert=/run/certs/rds-ca-2019-root.pem sslmode=verify-ca
       layer_scan_concurrency: 6
       migrations: true
       scanlock_retry: 11
   log_level: debug
   matcher:
       connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslrootcert=/run/certs/rds-ca-2019-root.pem sslmode=verify-ca
       migrations: true
   metrics:
       name: prometheus
   notifier:
       connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslrootcert=/run/certs/rds-ca-2019-root.pem sslmode=verify-ca
       migrations: true

   Copy to Clipboard Toggle word wrap

   Note
   • The database certificate is mounted under /run/certs/rds-ca-2019-root.pem on the Clair application pod in the clair-config.yaml. It must be specified when configuring your clair-config.yaml.
   • An example clair-config.yaml can be found at Clair on OpenShift config.

2. Add the clair-config.yaml file to your bundle secret, for example:

   apiVersion: v1
   kind: Secret
   metadata:
     name: config-bundle-secret
     namespace: quay-enterprise
   data:
     config.yaml: <base64 encoded Quay config>
     clair-config.yaml: <base64 encoded Clair config>
     extra_ca_cert_<name>: <base64 encoded ca cert>
     ssl.crt: <base64 encoded SSL certificate>
     ssl.key: <base64 encoded SSL private key>

   Copy to Clipboard Toggle word wrap
   Note

   When updated, the provided clair-config.yaml file is mounted into the Clair pod. Any fields not provided are automatically populated with defaults using the Clair configuration module.

3. You can check the status of your Clair pod by clicking the commit in the Build History page, or by running oc get pods -n <namespace>. For example:

   $ oc get pods -n <namespace>

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                               READY   STATUS    RESTARTS   AGE
   f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Running   0          7s

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a Quay configuration bundle secret that includes the clair-config.yaml by entering the following command:

   $ oc create secret generic --from-file config.yaml=./config.yaml --from-file extra_ca_cert_rds-ca-2019-root.pem=./rds-ca-2019-root.pem --from-file clair-config.yaml=./clair-config.yaml config-bundle-secret

   Copy to Clipboard Toggle word wrap

   Example Clair config.yaml file

   indexer:
       connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslmode=disable
       layer_scan_concurrency: 6
       migrations: true
       scanlock_retry: 11
   log_level: debug
   matcher:
       connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslmode=disable
       migrations: true
   metrics:
       name: prometheus
   notifier:
       connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslmode=disable
       migrations: true

   Copy to Clipboard Toggle word wrap

   Note
   • The database certificate is mounted under /run/certs/rds-ca-2019-root.pem on the Clair application pod in the clair-config.yaml. It must be specified when configuring your clair-config.yaml.
   • An example clair-config.yaml can be found at Clair on OpenShift config.

2. Add the clair-config.yaml file to your bundle secret, for example:

   apiVersion: v1
   kind: Secret
   metadata:
     name: config-bundle-secret
     namespace: quay-enterprise
   data:
     config.yaml: <base64 encoded Quay config>
     clair-config.yaml: <base64 encoded Clair config>

   Copy to Clipboard Toggle word wrap
   Note
   • When updated, the provided clair-config.yaml file is mounted into the Clair pod. Any fields not provided are automatically populated with defaults using the Clair configuration module.

3. You can check the status of your Clair pod by clicking the commit in the Build History page, or by running oc get pods -n <namespace>. For example:

   $ oc get pods -n <namespace>

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                               READY   STATUS    RESTARTS   AGE
   f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Running   0          7s

   Copy to Clipboard Toggle word wrap

Procedure

1. Log in to your Red Hat Quay cluster using a cluster administrator account.

2. Create a new project where your virtual builders will be run, for example, virtual-builders, by running the following command:

   $ oc new-project virtual-builders

   Copy to Clipboard Toggle word wrap

3. Create a ServiceAccount in the project that will be used to run builds by entering the following command:

   $ oc create sa -n virtual-builders quay-builder

   Copy to Clipboard Toggle word wrap

4. Provide the created service account with editing permissions so that it can run the build:

   $ oc adm policy -n virtual-builders add-role-to-user edit system:serviceaccount:virtual-builders:quay-builder

   Copy to Clipboard Toggle word wrap

5. Grant the Quay builder anyuid scc permissions by entering the following command:

   $ oc adm policy -n virtual-builders add-scc-to-user anyuid -z quay-builder

   Copy to Clipboard Toggle word wrap
   Note

   This action requires cluster admin privileges. This is required because builders must run as the Podman user for unprivileged or rootless builds to work.

6. Obtain the token for the Quay builder service account.

   1. If using OpenShift Container Platform 4.10 or an earlier version, enter the following command:

      oc sa get-token -n virtual-builders quay-builder

      Copy to Clipboard Toggle word wrap

   2. If using OpenShift Container Platform 4.11 or later, enter the following command:

      $ oc create token quay-builder -n virtual-builders

      Copy to Clipboard Toggle word wrap

      Example output

      eyJhbGciOiJSUzI1NiIsImtpZCI6IldfQUJkaDVmb3ltTHZ0dGZMYjhIWnYxZTQzN2dJVEJxcDJscldSdEUtYWsifQ...

      Copy to Clipboard Toggle word wrap

7. Determine the builder route by entering the following command:

   $ oc get route -n quay-enterprise

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                  HOST/PORT                                                                    PATH   SERVICES                              PORT   TERMINATION     WILDCARD
   ...
   example-registry-quay-builder         example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org                example-registry-quay-app             grpc   edge/Redirect   None
   ...

   Copy to Clipboard Toggle word wrap

8. Generate a self-signed SSL/TlS certificate with the .crt extension by entering the following command:

   $ oc extract cm/kube-root-ca.crt -n openshift-apiserver

   Copy to Clipboard Toggle word wrap

   Example output

   ca.crt

   Copy to Clipboard Toggle word wrap

9. Rename the ca.crt file to extra_ca_cert_build_cluster.crt by entering the following command:

   $ mv ca.crt extra_ca_cert_build_cluster.crt

   Copy to Clipboard Toggle word wrap

10. Locate the secret for you configuration bundle in the Console, and select Actions → Edit Secret and add the appropriate builder configuration:

    FEATURE_USER_INITIALIZE: true
    BROWSER_API_CALLS_XHR_ONLY: false
    SUPER_USERS:
    - <superusername>
    FEATURE_USER_CREATION: false
    FEATURE_QUOTA_MANAGEMENT: true
    FEATURE_BUILD_SUPPORT: True
    BUILDMAN_HOSTNAME: <sample_build_route> 

    1

    
    BUILD_MANAGER:
      - ephemeral
      - ALLOWED_WORKER_COUNT: 1
        ORCHESTRATOR_PREFIX: buildman/production/
        JOB_REGISTRATION_TIMEOUT: 3600 

    2

    
        ORCHESTRATOR:
          REDIS_HOST: <sample_redis_hostname> 

    3

    
          REDIS_PASSWORD: ""
          REDIS_SSL: false
          REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
        EXECUTORS:
          - EXECUTOR: kubernetesPodman
            NAME: openshift
            BUILDER_NAMESPACE: <sample_builder_namespace> 

    4

    
            SETUP_TIME: 180
            MINIMUM_RETRY_THRESHOLD: 0
            BUILDER_CONTAINER_IMAGE: <sample_builder_container_image> 

    5

    
            # Kubernetes resource options
            K8S_API_SERVER: <sample_k8s_api_server> 

    6

    
            K8S_API_TLS_CA: <sample_crt_file> 

    7

    
            VOLUME_SIZE: 8G
            KUBERNETES_DISTRIBUTION: openshift
            CONTAINER_MEMORY_LIMITS: 300m 

    8

    
            CONTAINER_CPU_LIMITS: 1G 

    9

    
            CONTAINER_MEMORY_REQUEST: 300m 

    10

    
            CONTAINER_CPU_REQUEST: 1G 

    11

    
            NODE_SELECTOR_LABEL_KEY: ""
            NODE_SELECTOR_LABEL_VALUE: ""
            SERVICE_ACCOUNT_NAME: <sample_service_account_name>
            SERVICE_ACCOUNT_TOKEN: <sample_account_token> 

    12

    Copy to Clipboard Toggle word wrap

    1

    The build route is obtained by running oc get route -n with the name of your OpenShift Operator’s namespace. A port must be provided at the end of the route, and it should use the following format: [quayregistry-cr-name]-quay-builder-[ocp-namespace].[ocp-domain-name]:443.

    2

    If the JOB_REGISTRATION_TIMEOUT parameter is set too low, you might receive the following error: failed to register job to build manager: rpc error: code = Unauthenticated desc = Invalid build token: Signature has expired. It is suggested that this parameter be set to at least 240.

    3

    If your Redis host has a password or SSL/TLS certificates, you must update accordingly.

    4

    Set to match the name of your virtual builders namespace, for example, virtual-builders.

    5

    For early access, the BUILDER_CONTAINER_IMAGE is currently quay.io/projectquay/quay-builder:3.7.0-rc.2. Note that this might change during the early access window. If this happens, customers are alerted.

    6

    The K8S_API_SERVER is obtained by running oc cluster-info.

    7

    You must manually create and add your custom CA cert, for example, K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt.

    8

    Defaults to 5120Mi if left unspecified.

    9

    For virtual builds, you must ensure that there are enough resources in your cluster. Defaults to 1000m if left unspecified.

    10

    Defaults to 3968Mi if left unspecified.

    11

    Defaults to 500m if left unspecified.

    12

    Obtained when running oc create sa.

    Sample configuration

    FEATURE_USER_INITIALIZE: true
    BROWSER_API_CALLS_XHR_ONLY: false
    SUPER_USERS:
    - quayadmin
    FEATURE_USER_CREATION: false
    FEATURE_QUOTA_MANAGEMENT: true
    FEATURE_BUILD_SUPPORT: True
    BUILDMAN_HOSTNAME: example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org:443
    BUILD_MANAGER:
      - ephemeral
      - ALLOWED_WORKER_COUNT: 1
        ORCHESTRATOR_PREFIX: buildman/production/
        JOB_REGISTRATION_TIMEOUT: 3600
        ORCHESTRATOR:
          REDIS_HOST: example-registry-quay-redis
          REDIS_PASSWORD: ""
          REDIS_SSL: false
          REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
        EXECUTORS:
          - EXECUTOR: kubernetesPodman
            NAME: openshift
            BUILDER_NAMESPACE: virtual-builders
            SETUP_TIME: 180
            MINIMUM_RETRY_THRESHOLD: 0
            BUILDER_CONTAINER_IMAGE: quay.io/projectquay/quay-builder:3.7.0-rc.2
            # Kubernetes resource options
            K8S_API_SERVER: api.docs.quayteam.org:6443
            K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt
            VOLUME_SIZE: 8G
            KUBERNETES_DISTRIBUTION: openshift
            CONTAINER_MEMORY_LIMITS: 1G
            CONTAINER_CPU_LIMITS: 1080m
            CONTAINER_MEMORY_REQUEST: 1G
            CONTAINER_CPU_REQUEST: 580m
            NODE_SELECTOR_LABEL_KEY: ""
            NODE_SELECTOR_LABEL_VALUE: ""
            SERVICE_ACCOUNT_NAME: quay-builder
            SERVICE_ACCOUNT_TOKEN: "eyJhbGciOiJSUzI1NiIsImtpZCI6IldfQUJkaDVmb3ltTHZ0dGZMYjhIWnYxZTQzN2dJVEJxcDJscldSdEUtYWsifQ"

    Copy to Clipboard Toggle word wrap

Procedure

1. Log in to your Red Hat Quay repository.
2. Click Create New Repository and create a new registry, for example, testrepo.

3. On the Repositories page, click the Builds tab on the navigation pane. Alternatively, use the corresponding URL directly:

   https://example-registry-quay-quay-enterprise.apps.docs.quayteam.org/repository/quayadmin/testrepo?tab=builds

   Copy to Clipboard Toggle word wrap
   Important

   In some cases, the builder might have issues resolving hostnames. This issue might be related to the dnsPolicy being set to default on the job object. Currently, there is no workaround for this issue. It will be resolved in a future version of Red Hat Quay.

4. Click Create Build Trigger → Custom Git Repository Push.

5. Enter the HTTPS or SSH style URL used to clone your Git repository, then click Continue. For example:

   https://github.com/gabriel-rh/actions_test.git

   Copy to Clipboard Toggle word wrap
6. Check Tag manifest with the branch or tag name and then click Continue.
7. Enter the location of the Dockerfile to build when the trigger is invoked, for example, /Dockerfile and click Continue.
8. Enter the location of the context for the Docker build, for example, /, and click Continue.
9. If warranted, create a Robot Account. Otherwise, click Continue.
10. Click Continue to verify the parameters.
11. On the Builds page, click Options icon of your Trigger Name, and then click Run Trigger Now.
12. Enter a commit SHA from the Git repository and click Start Build.

13. You can check the status of your build by clicking the commit in the Build History page, or by running oc get pods -n virtual-builders. For example:

    $ oc get pods -n virtual-builders

    Copy to Clipboard Toggle word wrap

    Example output

    NAME                                               READY   STATUS    RESTARTS   AGE
    f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Running   0          7s

    Copy to Clipboard Toggle word wrap

    $ oc get pods -n virtual-builders

    Copy to Clipboard Toggle word wrap

    Example output

    NAME                                               READY   STATUS        RESTARTS   AGE
    f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Terminating   0          9s

    Copy to Clipboard Toggle word wrap

    $ oc get pods -n virtual-builders

    Copy to Clipboard Toggle word wrap

    Example output

    No resources found in virtual-builders namespace.

    Copy to Clipboard Toggle word wrap

14. When the build is finished, you can check the status of the tag under Tags on the navigation pane.

    Note

    With early access, full build logs and timestamps of builds are currently unavailable.

Procedure

1. Log in to your AWS console at s3.console.aws.com.
2. In the search bar, search for S3 and then click S3.
3. Click the name of your bucket, for example, myawsbucket.
4. Click the Permissions tab.

5. Under Cross-origin resource sharing (CORS), include the following parameters:

     [
         {
             "AllowedHeaders": [
                 "Authorization"
             ],
             "AllowedMethods": [
                 "GET"
             ],
             "AllowedOrigins": [
                 "*"
             ],
             "ExposeHeaders": [],
             "MaxAgeSeconds": 3000
         },
         {
             "AllowedHeaders": [
                 "Content-Type",
                 "x-amz-acl",
                 "origin"
             ],
             "AllowedMethods": [
                 "PUT"
             ],
             "AllowedOrigins": [
                 "*"
             ],
             "ExposeHeaders": [],
             "MaxAgeSeconds": 3000
         }
     ]

   Copy to Clipboard Toggle word wrap

Procedure

1. Use the following reference to create a JSON file for your specific CORS needs. For example:

   $ cat gcp_cors.json

   Copy to Clipboard Toggle word wrap

   Example output

   [
       {
         "origin": ["*"],
         "method": ["GET"],
         "responseHeader": ["Authorization"],
         "maxAgeSeconds": 3600
       },
       {
         "origin": ["*"],
         "method": ["PUT"],
         "responseHeader": [
                 "Content-Type",
                 "x-goog-acl",
                 "origin"],
         "maxAgeSeconds": 3600
       }
   ]

   Copy to Clipboard Toggle word wrap

2. Enter the following command to update your GCP storage bucket:

   $ gcloud storage buckets update gs://<bucket_name> --cors-file=./gcp_cors.json

   Copy to Clipboard Toggle word wrap

   Example output

   Updating
     Completed 1

   Copy to Clipboard Toggle word wrap

3. You can display the updated CORS configuration of your GCP bucket by running the following command:

   $ gcloud storage buckets describe gs://<bucket_name>  --format="default(cors)"

   Copy to Clipboard Toggle word wrap

   Example output

   cors:
   - maxAgeSeconds: 3600
     method:
     - GET
     origin:
     - '*'
     responseHeader:
     - Authorization
   - maxAgeSeconds: 3600
     method:
     - PUT
     origin:
     - '*'
     responseHeader:
     - Content-Type
     - x-goog-acl
     - origin

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a config.yaml file that is shared between clusters. This config.yaml file contains the details for the common PostgreSQL, Redis and storage backends:

   Geo-replication config.yaml file

   SERVER_HOSTNAME: <georep.quayteam.org or any other name> 

   1

   
   DB_CONNECTION_ARGS:
     autorollback: true
     threadlocals: true
   DB_URI: postgresql://postgres:password@10.19.0.1:5432/quay 

   2

   
   BUILDLOGS_REDIS:
     host: 10.19.0.2
     port: 6379
   USER_EVENTS_REDIS:
     host: 10.19.0.2
     port: 6379
   DISTRIBUTED_STORAGE_CONFIG:
     usstorage:
       - GoogleCloudStorage
       - access_key: GOOGQGPGVMASAAMQABCDEFG
         bucket_name: georep-test-bucket-0
         secret_key: AYWfEaxX/u84XRA2vUX5C987654321
         storage_path: /quaygcp
     eustorage:
       - GoogleCloudStorage
       - access_key: GOOGQGPGVMASAAMQWERTYUIOP
         bucket_name: georep-test-bucket-1
         secret_key: AYWfEaxX/u84XRA2vUX5Cuj12345678
         storage_path: /quaygcp
   DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS:
     - usstorage
     - eustorage
   DISTRIBUTED_STORAGE_PREFERENCE:
     - usstorage
     - eustorage
   FEATURE_STORAGE_REPLICATION: true

   Copy to Clipboard Toggle word wrap

   1

   A proper SERVER_HOSTNAME must be used for the route and must match the hostname of the global load balancer.

   2

   To retrieve the configuration file for a Clair instance deployed using the OpenShift Container Platform Operator, see Retrieving the Clair config.

2. Create the configBundleSecret by entering the following command:

   $ oc create secret generic --from-file config.yaml=./config.yaml georep-config-bundle

   Copy to Clipboard Toggle word wrap

3. In each of the clusters, set the configBundleSecret and use the QUAY_DISTRIBUTED_STORAGE_PREFERENCE environmental variable override to configure the appropriate storage for that cluster. For example:

   Note

   The config.yaml file between both deployments must match. If making a change to one cluster, it must also be changed in the other.

   US cluster QuayRegistry example

   apiVersion: quay.redhat.com/v1
   kind: QuayRegistry
   metadata:
     name: example-registry
     namespace: quay-enterprise
   spec:
     configBundleSecret: georep-config-bundle
     components:
       - kind: objectstorage
         managed: false
       - kind: route
         managed: true
       - kind: tls
         managed: false
       - kind: postgres
         managed: false
       - kind: clairpostgres
         managed: false
       - kind: redis
         managed: false
       - kind: quay
         managed: true
         overrides:
           env:
           - name: QUAY_DISTRIBUTED_STORAGE_PREFERENCE
             value: usstorage
       - kind: mirror
         managed: true
         overrides:
           env:
           - name: QUAY_DISTRIBUTED_STORAGE_PREFERENCE
             value: usstorage

   Copy to Clipboard Toggle word wrap

   Note

   Because SSL/TLS is unmanaged, and the route is managed, you must supply the certificates with either with the config tool or directly in the config bundle. For more information, see Configuring TLS and routes.

   European cluster

   apiVersion: quay.redhat.com/v1
   kind: QuayRegistry
   metadata:
     name: example-registry
     namespace: quay-enterprise
   spec:
     configBundleSecret: georep-config-bundle
     components:
       - kind: objectstorage
         managed: false
       - kind: route
         managed: true
       - kind: tls
         managed: false
       - kind: postgres
         managed: false
       - kind: clairpostgres
         managed: false
       - kind: redis
         managed: false
       - kind: quay
         managed: true
         overrides:
           env:
           - name: QUAY_DISTRIBUTED_STORAGE_PREFERENCE
             value: eustorage
       - kind: mirror
         managed: true
         overrides:
           env:
           - name: QUAY_DISTRIBUTED_STORAGE_PREFERENCE
             value: eustorage

   Copy to Clipboard Toggle word wrap

   Note

   Because SSL/TLS is unmanaged, and the route is managed, you must supply the certificates with either with the config tool or directly in the config bundle. For more information, see Configuring TLS and routes.