Chapter 3. Configuring Red Hat Quay before deployment
The Red Hat Quay Operator can manage all of the Red Hat Quay components when deployed on OpenShift Container Platform. This is the default configuration, however, you can manage one or more components externally when you want more control over the set up.
Use the following pattern to configure unmanaged Red Hat Quay components.
Procedure
Create a
config.yamlconfiguration file with the appropriate settings. Use the following reference for a minimal configuration:$ touch config.yamlAUTHENTICATION_TYPE: Database BUILDLOGS_REDIS: host: <quay-server.example.com> password: <strongpassword> port: 6379 ssl: false DATABASE_SECRET_KEY: <0ce4f796-c295-415b-bf9d-b315114704b8> DB_URI: <postgresql://quayuser:quaypass@quay-server.example.com:5432/quay> DEFAULT_TAG_EXPIRATION: 2w DISTRIBUTED_STORAGE_CONFIG: default: - LocalStorage - storage_path: /datastorage/registry DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: [] DISTRIBUTED_STORAGE_PREFERENCE: - default PREFERRED_URL_SCHEME: http SECRET_KEY: <e8f9fe68-1f84-48a8-a05f-02d72e6eccba> SERVER_HOSTNAME: <quay-server.example.com> SETUP_COMPLETE: true TAG_EXPIRATION_OPTIONS: - 0s - 1d - 1w - 2w - 4w - 3y USER_EVENTS_REDIS: host: <quay-server.example.com> port: 6379 ssl: falseCreate a
Secretusing the configuration file by entering the following command:$ oc create secret generic --from-file config.yaml=./config.yaml config-bundle-secretCreate a
quayregistry.yamlfile, identifying the unmanaged components and also referencing the createdSecret, for example:Example
QuayRegistryYAML fileapiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: example-registry namespace: quay-enterprise spec: configBundleSecret: <config_bundle_secret> components: - kind: objectstorage managed: false # ...Enter the following command to deploy the registry by using the
quayregistry.yamlfile:$ oc create -n quay-enterprise -f quayregistry.yaml
3.1. Pre-configuration options for automation Copy linkLink copied to clipboard!
Red Hat Quay provides configuration options that enable registry administrators to automate early setup tasks and API accessibility. These options are useful for new deployments and controlling how API calls can be made. The following options support automation and administrative control.
| Field | Type | Description |
|---|---|---|
| FEATURE_USER_INITIALIZE | Boolean |
Enables initial user bootstrapping in a newly deployed Red Hat Quay registry. When this field is set to Note
Unlike all other registry API calls that require an OAuth 2 access token generated by an OAuth application in an existing organization, the |
| BROWSER_API_CALLS_XHR_ONLY | Boolean |
Controls whether the registry API only accepts calls from browsers. To allow general browser-based access to the API, administrators must set this field to |
| SUPER_USERS | String |
Defines a list of administrative users, or superusers, who have full privileges and unrestricted access to the registry. Red Hat Quay administrators should configure |
| FEATURE_USER_CREATION | Boolean |
Relegates the creation of new users to only superusers when this field is set to |
The following YAML shows you the suggested configuration for automation:
Suggested configuration for automation
# ...
FEATURE_USER_INITIALIZE: true
BROWSER_API_CALLS_XHR_ONLY: false
SUPER_USERS:
- quayadmin
FEATURE_USER_CREATION: false
# ...
3.2. Configuring object storage Copy linkLink copied to clipboard!
You need to configure object storage before installing Red Hat Quay, irrespective of whether you are allowing the Red Hat Quay Operator to manage the storage or managing it yourself.
If you want the Red Hat Quay Operator to be responsible for managing storage, see the section on Managed storage for information on installing and configuring NooBaa and the Red Hat OpenShift Data Foundations Operator.
If you are using a separate storage solution, set objectstorage as unmanaged when configuring the Operator. See the following section. Unmanaged storage, for details of configuring existing storage.
3.2.1. Using unmanaged storage Copy linkLink copied to clipboard!
This section provides configuration examples for unmanaged storage for your convenience. Refer to the Red Hat Quay configuration guide for complete instructions on how to set up object storage.
3.2.1.1. AWS S3 storage Copy linkLink copied to clipboard!
Use the following example when configuring AWS S3 storage for your Red Hat Quay deployment.
DISTRIBUTED_STORAGE_CONFIG:
s3Storage:
- S3Storage
- host: s3.us-east-2.amazonaws.com
s3_access_key: ABCDEFGHIJKLMN
s3_secret_key: OL3ABCDEFGHIJKLMN
s3_bucket: quay_bucket
s3_region: <region>
storage_path: /datastorage/registry
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: []
DISTRIBUTED_STORAGE_PREFERENCE:
- s3Storage
3.2.1.2. AWS Cloudfront storage Copy linkLink copied to clipboard!
Use the following example when configuring AWS Cloudfront for your Red Hat Quay deployment.
When configuring AWS Cloudfront storage, the following conditions must be met for proper use with Red Hat Quay:
-
You must set an Origin path that is consistent with Red Hat Quay’s storage path as defined in your
config.yamlfile. Failure to meet this require results in a403error when pulling an image. For more information, see Origin path. - You must configure a Bucket policy and a Cross-origin resource sharing (CORS) policy.
-
You must set an Origin path that is consistent with Red Hat Quay’s storage path as defined in your
Cloudfront S3 example YAML
DISTRIBUTED_STORAGE_CONFIG:
default:
- CloudFrontedS3Storage
- cloudfront_distribution_domain: <CLOUDFRONT_DISTRIBUTION_DOMAIN>
cloudfront_key_id: <CLOUDFRONT_KEY_ID>
cloudfront_privatekey_filename: <CLOUDFRONT_PRIVATE_KEY_FILENAME>
host: <S3_HOST>
s3_access_key: <S3_ACCESS_KEY>
s3_bucket: <S3_BUCKET_NAME>
s3_secret_key: <S3_SECRET_KEY>
storage_path: <STORAGE_PATH>
s3_region: <S3_REGION>
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS:
- default
DISTRIBUTED_STORAGE_PREFERENCE:
- default
Bucket policy example
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<AWS_ACCOUNT_ID>:user/CloudFront Origin Access Identity <CLOUDFRONT_OAI_ID>"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<S3_BUCKET_NAME>/*"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<AWS_ACCOUNT_ID>:user/CloudFront Origin Access Identity <CLOUDFRONT_OAI_ID>"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<S3_BUCKET_NAME>"
}
]
}
3.2.1.3. Google Cloud storage Copy linkLink copied to clipboard!
Use the following example when configuring Google Cloud storage for your Red Hat Quay deployment.
DISTRIBUTED_STORAGE_CONFIG:
googleCloudStorage:
- GoogleCloudStorage
- access_key: GOOGQIMFB3ABCDEFGHIJKLMN
bucket_name: quay-bucket
secret_key: FhDAYe2HeuAKfvZCAGyOioNaaRABCDEFGHIJKLMN
storage_path: /datastorage/registry
boto_timeout: 120
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: []
DISTRIBUTED_STORAGE_PREFERENCE:
- googleCloudStorage
- 1
- Optional. The time, in seconds, until a timeout exception is thrown when attempting to read from a connection. The default is
60seconds. Also encompasses the time, in seconds, until a timeout exception is thrown when attempting to make a connection. The default is60seconds.
3.2.1.4. Microsoft Azure storage Copy linkLink copied to clipboard!
Use the following example when configuring Microsoft Azure storage for your Red Hat Quay deployment.
DISTRIBUTED_STORAGE_CONFIG:
azureStorage:
- AzureStorage
- azure_account_name: azure_account_name_here
azure_container: azure_container_here
storage_path: /datastorage/registry
azure_account_key: azure_account_key_here
sas_token: some/path/
endpoint_url: https://[account-name].blob.core.usgovcloudapi.net
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: []
DISTRIBUTED_STORAGE_PREFERENCE:
- azureStorage
- 1
- The
endpoint_urlparameter for Microsoft Azure storage is optional and can be used with Microsoft Azure Government (MAG) endpoints. If left blank, theendpoint_urlwill connect to the normal Microsoft Azure region.As of Red Hat Quay 3.7, you must use the Primary endpoint of your MAG Blob service. Using the Secondary endpoint of your MAG Blob service will result in the following error:
AuthenticationErrorDetail:Cannot find the claimed account when trying to GetProperties for the account whusc8-secondary.
3.2.1.5. Ceph/RadosGW Storage Copy linkLink copied to clipboard!
Use the following example when configuring Ceph/RadosGW storage for your Red Hat Quay deployment.
DISTRIBUTED_STORAGE_CONFIG:
radosGWStorage: #storage config name
- RadosGWStorage #actual driver
- access_key: access_key_here #parameters
secret_key: secret_key_here
bucket_name: bucket_name_here
hostname: hostname_here
is_secure: 'true'
port: '443'
storage_path: /datastorage/registry
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: []
DISTRIBUTED_STORAGE_PREFERENCE: #must contain name of the storage config
- radosGWStorage
3.2.1.6. Swift storage Copy linkLink copied to clipboard!
Use the following example when configuring Swift storage for your Red Hat Quay deployment.
DISTRIBUTED_STORAGE_CONFIG:
swiftStorage:
- SwiftStorage
- swift_user: swift_user_here
swift_password: swift_password_here
swift_container: swift_container_here
auth_url: https://example.org/swift/v1/quay
auth_version: 3
os_options:
tenant_id: <osp_tenant_id_here>
user_domain_name: <osp_domain_name_here>
ca_cert_path: /conf/stack/swift.cert"
storage_path: /datastorage/registry
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: []
DISTRIBUTED_STORAGE_PREFERENCE:
- swiftStorage
3.2.1.7. NooBaa unmanaged storage Copy linkLink copied to clipboard!
Use the following procedure to deploy NooBaa as your unmanaged storage configuration.
Procedure
-
Create a NooBaa Object Bucket Claim in the Red Hat Quay console by navigating to Storage
Object Bucket Claims. - Retrieve the Object Bucket Claim Data details, including the Access Key, Bucket Name, Endpoint (hostname), and Secret Key.
Create a
config.yamlconfiguration file that uses the information for the Object Bucket Claim:DISTRIBUTED_STORAGE_CONFIG: default: - RHOCSStorage - access_key: WmrXtSGk8B3nABCDEFGH bucket_name: my-noobaa-bucket-claim-8b844191-dc6c-444e-9ea4-87ece0abcdef hostname: s3.openshift-storage.svc.cluster.local is_secure: true port: "443" secret_key: X9P5SDGJtmSuHFCMSLMbdNCMfUABCDEFGH+C5QD storage_path: /datastorage/registry DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: [] DISTRIBUTED_STORAGE_PREFERENCE: - default
For more information about configuring an Object Bucket Claim, see Object Bucket Claim.
3.2.2. Using an unmanaged NooBaa instance Copy linkLink copied to clipboard!
Use the following procedure to use an unmanaged NooBaa instance for your Red Hat Quay deployment.
Procedure
-
Create a NooBaa Object Bucket Claim in the console at Storage
Object Bucket Claims. -
Retrieve the Object Bucket Claim Data details including the
Access Key,Bucket Name,Endpoint (hostname), andSecret Key. Create a
config.yamlconfiguration file using the information for the Object Bucket Claim. For example:DISTRIBUTED_STORAGE_CONFIG: default: - RHOCSStorage - access_key: WmrXtSGk8B3nABCDEFGH bucket_name: my-noobaa-bucket-claim-8b844191-dc6c-444e-9ea4-87ece0abcdef hostname: s3.openshift-storage.svc.cluster.local is_secure: true port: "443" secret_key: X9P5SDGJtmSuHFCMSLMbdNCMfUABCDEFGH+C5QD storage_path: /datastorage/registry DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: [] DISTRIBUTED_STORAGE_PREFERENCE: - default
3.2.3. Managed storage Copy linkLink copied to clipboard!
If you want the Red Hat Quay Operator to manage object storage for Red Hat Quay, your cluster needs to be capable of providing object storage through the ObjectBucketClaim API. Using the Red Hat OpenShift Data Foundation Operator, there are two supported options available:
A standalone instance of the Multi-Cloud Object Gateway backed by a local Kubernetes
PersistentVolumestorage- Not highly available
- Included in the Red Hat Quay subscription
- Does not require a separate subscription for Red Hat OpenShift Data Foundation
A production deployment of Red Hat OpenShift Data Foundation with scale-out Object Service and Ceph
- Highly available
- Requires a separate subscription for Red Hat OpenShift Data Foundation
To use the standalone instance option, continue reading below. For production deployment of Red Hat OpenShift Data Foundation, please refer to the official documentation.
Object storage disk space is allocated automatically by the Red Hat Quay Operator with 50 GiB. This number represents a usable amount of storage for most small to medium Red Hat Quay installations but might not be sufficient for your use cases. Resizing the Red Hat OpenShift Data Foundation volume is currently not handled by the Red Hat Quay Operator. See the section below about resizing managed storage for more details.
3.2.3.1. Leveraging the Multicloud Object Gateway Component in the Red Hat OpenShift Data Foundation Operator for Red Hat Quay Copy linkLink copied to clipboard!
As part of a Red Hat Quay subscription, users are entitled to use the Multicloud Object Gateway component of the Red Hat OpenShift Data Foundation Operator (formerly known as OpenShift Container Storage Operator). This gateway component allows you to provide an S3-compatible object storage interface to Red Hat Quay backed by Kubernetes PersistentVolume-based block storage. The usage is limited to a Red Hat Quay deployment managed by the Operator and to the exact specifications of the multicloud Object Gateway instance as documented below.
Since Red Hat Quay does not support local filesystem storage, users can leverage the gateway in combination with Kubernetes PersistentVolume storage instead, to provide a supported deployment. A PersistentVolume is directly mounted on the gateway instance as a backing store for object storage and any block-based StorageClass is supported.
By the nature of PersistentVolume, this is not a scale-out, highly available solution and does not replace a scale-out storage system like Red Hat OpenShift Data Foundation. Only a single instance of the gateway is running. If the pod running the gateway becomes unavailable due to rescheduling, updates or unplanned downtime, this will cause temporary degradation of the connected Red Hat Quay instances.
Deploying Red Hat Quay on OpenShift Container Platform using Red Hat OpenShift Data Foundation requires you to download the Local Storage Operator, the Red Hat OpenShift Data Foundation Operator, and then deploy a standalone Multicloud Object Gateway using the OpenShift Container Platform UI. See the following Red Hat OpenShift Data Foundation documentation for these steps: