Chapter 13. Open Container Initiative support and Red Hat Quay
Container registries such as Red Hat Quay were originally designed to support container images in the Docker image format. To promote the use of additional runtimes apart from Docker, the Open Container Initiative (OCI) was created to provide a standardization surrounding container runtimes and image formats. Most container registries support the OCI standardization as it is based on the Docker image manifest V2, Schema 2 format.
In addition to container images, a variety of artifacts have emerged that support not just individual applications, but also the Kubernetes platform as a whole. These range from Open Policy Agent (OPA) policies for security and governance to Helm charts and Operators that aid in application deployment.
Red Hat Quay is a private container registry that not only stores container images, but supports an entire ecosystem of tooling to aid in the management of containers. Prior to version 3.6, Red Hat Quay only supported Helm, which is considered to be the de facto package manager for Kubernetes.
Helm simplifies how applications are packaged and deployed. Helm uses a packaging format called Charts which contain the Kubernetes resources representing an application. Charts can be made available for general distribution and consumption in repositories. A Helm repository is an HTTP server that serves an index.yaml
metadata file and, optionally, a set of packaged charts. Beginning with Helm version 3, support was made available for distributing charts in OCI registries as an alternative to a traditional repository.
As an enhance to Helm support, Red Hat Quay introduced support for OCI-based artifacts from version 3.6 to include support for cosign, the ZStandard compression scheme, and other OCI media types. Support for Helm and other OCI artifacts are now enabled by default under the FEATURE_GENERAL_OCI_SUPPORT
configuration field, and can be expanded to other artifact types using the ALLOWED_OCI_ARTIFACT_TYPES
and IGNORE_UNKNOWN_MEDIATYPES
fields.
Because of the addition of FEATURE_GENERAL_OCI_SUPPORT
, ALLOWED_OCI_ARTIFACT_TYPES
, and IGNORE_UNKNOWN_MEDIATYPES
, the FEATURE_HELM_OCI_SUPPORT
configuration field has been deprecated. This configuration field is no longer supported and will be removed in a future version of Red Hat Quay.
13.1. Helm and OCI prerequisites
Prior to enabling Helm and other Open Container Initiative (OCI) artifact types, you must meet the following prerequisites.
13.1.1. Installing Helm
Use the following procedure to install the Helm client.
Procedure
- Download the latest version of Helm from the Helm releases page.
Enter the following command to unpack the Helm binary:
$ tar -zxvf helm-v3.8.2-linux-amd64.tar.gz
Move the Helm binary to the desired location:
$ mv linux-amd64/helm /usr/local/bin/helm
For more information about installing Helm, see the Installing Helm documentation.
13.1.2. Upgrading to Helm 3.8
Support for OCI registry charts requires that Helm has been upgraded to at least 3.8. If you have already downloaded Helm and need to upgrade to Helm 3.8, see the Helm Upgrade documentation.
13.1.3. Enabling your system to trust SSL/TLS certificates used by Red Hat Quay
Communication between the Helm client and Red Hat Quay is facilitated over HTTPS. As of Helm 3.5, support is only available for registries communicating over HTTPS with trusted certificates. In addition, the operating system must trust the certificates exposed by the registry. You must ensure that your operating system has been configured to trust the certificates used by Red Hat Quay. Use the following procedure to enable your system to trust the custom certificates.
Procedure
Enter the following command to copy the
rootCA.pem
file to the/etc/pki/ca-trust/source/anchors/
folder:$ sudo cp rootCA.pem /etc/pki/ca-trust/source/anchors/
Enter the following command to update the CA trust store:
$ sudo update-ca-trust extract
13.1.4. Creating an organization for Helm in Red Hat Quay
It is recommended that you create a new organization for storing Helm charts in Red Hat Quay after you have downloaded the Helm client. Use the following procedure to create a new organization using the Red Hat Quay UI.
Procedure
- Log in to your Red Hat Quay deployment.
- Click Create New Organization.
- Enter a name for the organization, for example, helm. Then, click Create Organization.
13.2. Using Helm charts with Red Hat Quay
Use the following example to download and push an etherpad chart from the Red Hat Community of Practice (CoP) repository.
Procedure
As a Red Hat Quay administrators, enable support for Helm by setting
FEATURE_GENERAL_OCI_SUPPORT
totrue
in yourconfig.yaml
file:FEATURE_GENERAL_OCI_SUPPORT: true
Add a chart repository:
$ helm repo add redhat-cop https://redhat-cop.github.io/helm-charts
Update the information of available charts locally from the chart repository:
$ helm repo update
Download a chart from a repository:
$ helm pull redhat-cop/etherpad --version=0.0.4 --untar
Package the chart into a chart archive:
$ helm package ./etherpad
Example output
Successfully packaged chart and saved it to: /home/user/linux-amd64/etherpad-0.0.4.tgz
Log in to your Quay repository using
helm registry login
:$ helm registry login quay370.apps.quayperf370.perfscale.devcluster.openshift.com
Push the chart to your Quay repository using the
helm push
command:$ helm push etherpad-0.0.4.tgz oci://quay370.apps.quayperf370.perfscale.devcluster.openshift.com
Example output:
Pushed: quay370.apps.quayperf370.perfscale.devcluster.openshift.com/etherpad:0.0.4 Digest: sha256:a6667ff2a0e2bd7aa4813db9ac854b5124ff1c458d170b70c2d2375325f2451b
Ensure that the push worked by deleting the local copy, and then pulling the chart from the repository:
$ rm -rf etherpad-0.0.4.tgz
$ helm pull oci://quay370.apps.quayperf370.perfscale.devcluster.openshift.com/etherpad --version 0.0.4
Example output:
Pulled: quay370.apps.quayperf370.perfscale.devcluster.openshift.com/etherpad:0.0.4 Digest: sha256:4f627399685880daf30cf77b6026dc129034d68c7676c7e07020b70cf7130902
13.3. Cosign OCI support with Red Hat Quay
Cosign is a tool that can be used to sign and verify container images. It uses the ECDSA-P256 signature algorithm and Red Hat’s Simple Signing payload format to create public keys that are stored in PKIX files. Private keys are stored as encrypted PEM files.
Cosign currently supports the following:
- Hardware and KMS Signing
- Bring-your-own PKI
- OIDC PKI
- Built-in binary transparency and timestamping service
13.4. Installing and using Cosign for Red Hat Quay
Use the following procedure to directly install Cosign.
Prerequisites
- You have installed Go version 1.16 or later.
-
You have set
FEATURE_GENERAL_OCI_SUPPORT
totrue
in yourconfig.yaml
file.
Procedure
Enter the following
go
command to directly install Cosign:$ go install github.com/sigstore/cosign/cmd/cosign@v1.0.0
Example output
go: downloading github.com/sigstore/cosign v1.0.0 go: downloading github.com/peterbourgon/ff/v3 v3.1.0
Generate a keypair for Cosign by entering the following command:
$ cosign generate-key-pair
Example output
Enter password for private key: Enter again: Private key written to cosign.key Public key written to cosign.pub
Sign the keypair by entering the following command:
$ cosign sign -key cosign.key quay-server.example.com/user1/busybox:test
Example output
Enter password for private key: Pushing signature to: quay-server.example.com/user1/busybox:sha256-ff13b8f6f289b92ec2913fa57c5dd0a874c3a7f8f149aabee50e3d01546473e3.sig
If you experience the
error: signing quay-server.example.com/user1/busybox:test: getting remote image: GET https://quay-server.example.com/v2/user1/busybox/manifests/test: UNAUTHORIZED: access to the requested resource is not authorized; map[]
error, which occurs because Cosign relies on~./docker/config.json
for authorization, you might need to execute the following command:$ podman login --authfile ~/.docker/config.json quay-server.example.com
Example output
Username: Password: Login Succeeded!
Enter the following command to see the updated authorization configuration:
$ cat ~/.docker/config.json { "auths": { "quay-server.example.com": { "auth": "cXVheWFkbWluOnBhc3N3b3Jk" } }
13.5. Using other artifact types with Red Hat Quay
Other artifact types that are not supported by default can be added to your Red Hat Quay deployment by using the ALLOWED_OCI_ARTIFACT_TYPES
configuration field.
Use the following procdure to add additional OCI media types.
Prerequisites
-
You have set
FEATURE_GENERAL_OCI_SUPPORT
totrue
in yourconfig.yaml
file.
Procedure
In your
config.yaml
file, add theALLOWED_OCI_ARTIFACT_TYPES
configuration field. For example:FEATURE_GENERAL_OCI_SUPPORT: true ALLOWED_OCI_ARTIFACT_TYPES: <oci config type 1>: - <oci layer type 1> - <oci layer type 2> <oci config type 2>: - <oci layer type 3> - <oci layer type 4>
Add support for your desired artifact type, for example, Singularity Image Format (SIF), by adding the following to your
config.yaml
file:ALLOWED_OCI_ARTIFACT_TYPES: application/vnd.oci.image.config.v1+json: - application/vnd.dev.cosign.simplesigning.v1+json application/vnd.cncf.helm.config.v1+json: - application/tar+gzip application/vnd.sylabs.sif.config.v1+json: - application/vnd.sylabs.sif.layer.v1+tar
ImportantWhen adding artifact types that are not configured by default, Red Hat Quay administrators will also need to manually add support for Cosign and Helm if desired.
Now, users can tag SIF images for their Red Hat Quay registry.
13.6. Disabling OCI artifacts in Red Hat Quay
Use the following procedure to disable support for OCI artifacts.
Procedure
Disable OCI artifact support by setting
FEATURE_GENERAL_OCI_SUPPORT
tofalse
in yourconfig.yaml
file. For example:FEATURE_GENERAL_OCI_SUPPORT = false