Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 8. Deploying Red Hat Quay on infrastructure nodes

By default, all quay-related pods are scheduled on available worker nodes in your OpenShift Container Platform cluster. In some environments, you might want to dedicate certain nodes specifically for infrastructure workloads—such as registry, database, and monitoring pods—to improve performance, isolate critical components, or simplify maintenance.

OpenShift Container Platform supports this approach using infrastructure machine sets, which automatically create and manage nodes reserved for infrastructure.

As an OpenShift Container Platform administrator, you can achieve the same result by labeling and tainting worker nodes. This ensures that only infrastructure workloads, like quay pods, are scheduled on these nodes. After your infrastructure nodes are configured, you can control where quay pods run using node selectors and tolerations.

The following procedures is intended for new deployments that install the Red Hat Quay Operator in a single namespace and provide their own backend storage. The procedure shows you how to prepare nodes and deploy Red Hat Quay on dedicated infrastructure nodes. In this procedure, all quay-related pods are placed on dedicated infrastructure nodes.

Use the following procedure to label and taint nodes for infrastructure use.

Note

The following procedure labels three worker nodes with the infra label. Depending on the resources relevant to your environment, you might have to label more than three worker nodes with the infra label.

  1. Obtain a list of worker nodes in your deployment by entering the following command: 

    $ oc get nodes | grep worker
    Copy to Clipboard Toggle word wrap

    Example output

    NAME                                                              STATUS   ROLES                  AGE    VERSION
---
example-cluster-new-c5qqp-worker-b-4zxx5.c.quay-devel.internal   Ready    worker                 401d   v1.31.11
example-cluster-new-c5qqp-worker-b-kz6jn.c.quay-devel.internal   Ready    worker                 402d   v1.31.11
example-cluster-new-c5qqp-worker-b-wrhw4.c.quay-devel.internal   Ready    worker                 401d   v1.31.11
---
    Copy to Clipboard Toggle word wrap

  2. Add the node-role.kubernetes.io/infra= label to the worker nodes by entering the following command. The number of infrastructure nodes required depends on your environment. Production environments should provision enough infra nodes to ensure high availability and sufficient resources for all quay-related components. Monitor CPU, memory, and storage utilization to determine if additional infra nodes are required. 

    $ oc label node --overwrite <infra_node_one> <infra_node_two> <infra_node_three> node-role.kubernetes.io/infra=
    Copy to Clipboard Toggle word wrap

  3. Confirm that the node-role.kubernetes.io/infra= label has been added to the proper nodes by entering the following command: 

    $ oc get node | grep infra
    Copy to Clipboard Toggle word wrap 
    ---
example-cluster-new-c5qqp-worker-b-4zxx5.c.quay-devel.internal   Ready    infra,worker           405d   v1.32.8
example-cluster-new-c5qqp-worker-b-kz6jn.c.quay-devel.internal   Ready    infra,worker           406d   v1.32.8
example-cluster-new-c5qqp-worker-b-wrhw4.c.quay-devel.internal   Ready    infra,worker           405d   v1.32.8
---
    Copy to Clipboard Toggle word wrap

  4. When a worker node is assigned the infra role, there is a chance that user workloads could get inadvertently assigned to an infra node. To avoid this, you can apply a taint to the infra node, and then add tolerations for the pods that you want to control. Taint the worker nodes with the infra label by entering the following command: 

    $ oc adm taint nodes -l node-role.kubernetes.io/infra \
  node-role.kubernetes.io/infra=reserved:NoSchedule --overwrite
    Copy to Clipboard Toggle word wrap

    Example output

    node/example-cluster-new-c5qqp-worker-b-4zxx5.c.quay-devel.internal modified
node/example-cluster-new-c5qqp-worker-b-kz6jn.c.quay-devel.internal modified
node/example-cluster-new-c5qqp-worker-b-wrhw4.c.quay-devel.internal modified
    Copy to Clipboard Toggle word wrap

Use the following procedure to create a project with the node-selector and tolerations annotations.

Procedure

  1. Add the node-selector annotation to the namespace by entering the following command: 

    $ oc annotate namespace <namespace> openshift.io/node-selector='node-role.kubernetes.io/infra='
    Copy to Clipboard Toggle word wrap

    Example output

    namespace/<namespace> annotated
    Copy to Clipboard Toggle word wrap

  2. Add the tolerations annotation to the namespace by entering the following command: 

    $ oc annotate namespace <namespace> scheduler.alpha.kubernetes.io/defaultTolerations='[{"operator":"Equal","value":"reserved","effect":"NoSchedule","key":"node-role.kubernetes.io/infra"},{"operator":"Equal","value":"reserved","effect":"NoExecute","key":"node-role.kubernetes.io/infra"}]' --overwrite
    Copy to Clipboard Toggle word wrap

    Example output

    namespace/<namespace> annotated
    Copy to Clipboard Toggle word wrap

    Important

    The tolerations in this example are specific to two taints commonly applied to infra nodes. The taints configured in your environment might differ. You must set the tolerations accordingly to match the taints applied to your infra nodes.

After you have added the node-role.kubernetes.io/infra= label to worker nodes and added the node-selector and tolerations annotations to the namespace, you must download the Red Hat Quay Operator in that namespace.

The following procedure shows you how to download the Red Hat Quay Operator on the annotated namespace and how to update the subscription to ensure successful installation.

Procedure

  1. On the OpenShift Container Platform web console, click Operators OperatorHub.
  2. In the search box, type Red Hat Quay.
  3. Click Red Hat Quay Install.
  4. Select the update channel, for example, stable-3.14 and the version.
  5. Click A specific namespace on the cluster for the installation mode, and then select the namespace that you applied the node-selector and tolerations annotations to.
  6. Click Install.

  1. Confirm that the Operator is installed by entering the following command: 

    $ oc get pods -n <annotated_namespace> -o wide | grep quay-operator
    Copy to Clipboard Toggle word wrap

    Example output

    quay-operator.v3.15.1-858b5c5fdc-lf5kj   1/1     Running   0          29m   10.130.6.18   example-cluster-new-c5qqp-worker-f-mhngl.c.quay-devel.internal   <none>           <none>
    Copy to Clipboard Toggle word wrap

8.4. Creating the Red Hat Quay registry
Copy link

After you have downloaded the Red Hat Quay Operator, you must create the Red Hat Quay registry. The registry’s components, for example, clair, postgres, redis, and so on, must be patched with the toleration annotation so that they can schedule onto the infra worker nodes.

The following procedure shows you how to create a Red Hat Quay registry that runs on infrastructure nodes.

Procedure

  1. On the OpenShift Container Platform web console, click Operators Installed Operators Red Hat Quay.
  2. On the Red Hat Quay Operator details page, click Quay Registry Create QuayRegistry.

  3. On the Create QuayRegistry page, set the monitoring and objectstorage fields to false. The monitoring component cannot be enabled when Red Hat Quay is installed in a single namespace. For example: 

    # ...
    - kind: monitoring
      managed: false
    - kind: objectstorage
      managed: false
# ...
    Copy to Clipboard Toggle word wrap
  4. Click Create.

  1. Optional: Confirm that the pods are running on infra nodes.

    1. List all Quay-related pods along with the nodes that they are scheduled on by entering the following command: 

      $ oc get pods -n <annotated_namespace> -o wide | grep example-registry
      Copy to Clipboard Toggle word wrap

      Example output

      ...
NAME                                               READY   STATUS      RESTARTS   AGE   IP             NODE                                                              NOMINATED NODE   READINESS GATES
example-registry-clair-app-5f95d685bd-dgjf6        1/1     Running     0          52m   10.128.4.12    example-cluster-new-c5qqp-worker-b-wrhw4.c.quay-devel.internal   <none>           <none>
...
      Copy to Clipboard Toggle word wrap

    2. Confirm that the nodes listed include only nodes labeled infra by running the following command: 

      $ oc get nodes -l node-role.kubernetes.io/infra -o name
      Copy to Clipboard Toggle word wrap

      Example output

      node/example-cluster-new-c5qqp-worker-b-4zxx5.c.quay-devel.internal modified
node/example-cluster-new-c5qqp-worker-b-kz6jn.c.quay-devel.internal modified
node/example-cluster-new-c5qqp-worker-b-wrhw4.c.quay-devel.internal modified
      Copy to Clipboard Toggle word wrap

      Note

      If any pod appears on a non-infra node, revisit your namespace annotations and deployment patching.

  2. Restart all pods for the Red Hat Quay registry by entering the following command: 

    $ oc delete pod -n <annotated_namespace> --all
    Copy to Clipboard Toggle word wrap

  3. Check the status of the pods by entering the following command: 

    $ oc get pods -n <annotated_namespace>
    Copy to Clipboard Toggle word wrap

    Example output

    ...
NAME                                               READY   STATUS      RESTARTS   AGE
example-registry-clair-app-5f95d685bd-dgjf6        1/1     Running     0          5m4s
...
    Copy to Clipboard Toggle word wrap

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat