Chapter 3. Setting up and configuring the registry
3.1. Configuring the registry for AWS user-provisioned infrastructure
3.1.1. Configuring a secret for the Image Registry Operator
In addition to the configs.imageregistry.operator.openshift.io
and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry
namespace.
The image-registry-private-configuration-user
secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.
For S3 on AWS storage the secret is expected to contain two keys:
- REGISTRY_STORAGE_S3_ACCESSKEY
- REGISTRY_STORAGE_S3_SECRETKEY
Procedure
Create an OpenShift Container Platform secret that contains the required keys.
$ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=myaccesskey --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=mysecretkey --namespace openshift-image-registry
3.1.2. Configuring registry storage for AWS with user-provisioned infrastructure
During installation, your cloud credentials are sufficient to create an S3 bucket and the Registry Operator will automatically configure storage.
If the Registry Operator cannot create an S3 bucket, and automatically configure storage, you can create an S3 bucket and configure storage with the following procedure.
Prerequisites
- A cluster on AWS with user-provisioned infrastructure.
For S3 on AWS storage the secret is expected to contain two keys:
-
REGISTRY_STORAGE_S3_ACCESSKEY
-
REGISTRY_STORAGE_S3_SECRETKEY
-
Procedure
Use the following procedure if the Registry Operator cannot create an S3 bucket and automatically configure storage.
- Set up a Bucket Lifecycle Policy to abort incomplete multipart uploads that are one day old.
Fill in the storage configuration in
configs.imageregistry.operator.openshift.io/cluster
:$ oc edit configs.imageregistry.operator.openshift.io/cluster storage: s3: bucket: <bucket-name> region: <region-name>
To secure your registry images in AWS, block public access to the S3 bucket.
3.1.3. Image Registry Operator configuration parameters for AWS S3
The following configuration parameters are available for AWS S3 registry storage.
Parameter | Description |
---|---|
| Bucket is the bucket name in which you want to store the registry’s data. It is optional and is generated if not provided. |
| Region is the AWS region in which your bucket exists. It is optional and is set based on the installed AWS Region. |
| RegionEndpoint is the endpoint for S3 compatible storage services. It is optional and defaults based on the Region that is provided. |
| Encrypt specifies whether or not the registry stores the image in encrypted format. It is optional and defaults to false. |
| KeyID is the KMS key ID to use for encryption. It is optional. Encrypt must be true, or this parameter is ignored. |
| CloudFront configures Amazon Cloudfront as the storage middleware in a registry. It is optional. |
3.2. Configuring the registry for GCP user-provisioned infrastructure
3.2.1. Configuring a secret for the Image Registry Operator
In addition to the configs.imageregistry.operator.openshift.io
and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry
namespace.
The image-registry-private-configuration-user
secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.
For GCS on GCP storage the secret is expected to contain one key whose value is the contents of a credentials file provided by GCP:
- REGISTRY_STORAGE_GCS_KEYFILE
Procedure
Create an OpenShift Container Platform secret that contains the required keys.
$ oc create secret generic image-registry-private-configuration-user --from-file=REGISTRY_STORAGE_GCS_KEYFILE=<path_to_keyfile> --namespace openshift-image-registry
3.2.2. Registry storage for GCP with user-provisioned infrastructure
You must set up the storage medium manually and configure the settings in the registry CRD.
Prerequisites
- A cluster on GCP with user-provisioned infrastructure.
- To configure registry storage for GCP, you need to provide Registry Operator cloud credentials.
For GCS on GCP storage the secret is expected to contain one key whose value is the contents of a credentials file provided by GCP:
-
REGISTRY_STORAGE_GCS_KEYFILE
-
3.2.3. Image Registry Operator configuration parameters for GCP GCS
Procedure
The following configuration parameters are available for GCP GCS registry storage.
Parameter | Description |
---|---|
| Bucket is the bucket name in which you want to store the registry’s data. It is optional and is generated if not provided. |
| Region is the GCS location in which your bucket exists. It is optional and is set based on the installed GCS Region. |
| ProjectID is the Project ID of the GCP project that this bucket should be associated with. It is optional. |
| KeyID is the KMS key ID to use for encryption. It is optional because buckets are encrypted by default on GCP. This allows for the use of a custom encryption key. |
3.3. Configuring the registry for bare metal
3.3.1. Image registry removed during installation
On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as Removed
. This allows openshift-installer
to complete installations on these platform types.
After installation, you must edit the Image Registry Operator configuration to switch the managementState
from Removed
to Managed
.
The Prometheus console provides an ImageRegistryRemoved
alert, for example:
"Image Registry has been removed. ImageStreamTags
, BuildConfigs
and DeploymentConfigs
which reference ImageStreamTags
may not work as expected. Please configure storage and update the config to Managed
state by editing configs.imageregistry.operator.openshift.io."
3.3.2. Changing the image registry’s management state
To start the image registry, you must change the Image Registry Operator configuration’s managementState
from Removed
to Managed
.
Procedure
Change
managementState
Image Registry Operator configuration fromRemoved
toManaged
. For example:$ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'
3.3.3. Image registry storage configuration
The image-registry
Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so the Registry Operator is made available.
Instructions for both configuring a PersistentVolume, which is required for production clusters, and for configuring an empty directory as the storage location, which is available for only non-production clusters, are shown.
3.3.4. Configuring registry storage for bare metal
As a cluster administrator, following installation you must configure your registry to use storage.
Prerequisites
- Cluster administrator permissions.
- A cluster on bare metal.
- Provision persistent storage for your cluster, such as Red Hat OpenShift Container Storage. To deploy a private image registry, your storage must provide ReadWriteMany access mode.
- Must have "100Gi" capacity.
Procedure
To configure your registry to use storage, change the
spec.storage.pvc
in theconfigs.imageregistry/cluster
resource.NoteWhen using shared storage such as NFS, it is strongly recommended to use the
supplementalGroups
strategy, which dictates the allowable supplemental groups for the Security Context, rather than thefsGroup
ID. Refer to the NFS Group IDs documentation for details.Verify you do not have a registry Pod:
$ oc get pod -n openshift-image-registry
Note-
If the storage type is
emptyDIR
, the replica number cannot be greater than1
. If the storage type is
NFS
, you must enable theno_wdelay
androot_squash
mount options. For example:# cat /etc/exports /mnt/data *(rw,sync,no_wdelay,root_squash,insecure,fsid=0) sh-4.3# exportfs -rv exporting *:/mnt/data
-
If the storage type is
Check the registry configuration:
$ oc edit configs.imageregistry.operator.openshift.io storage: pvc: claim:
Leave the
claim
field blank to allow the automatic creation of animage-registry-storage
PVC.Check the
clusteroperator
status:$ oc get clusteroperator image-registry
See Group IDs for additional details about using supplemental groups to handle NFS access.
3.3.5. Configuring storage for the image registry in non-production clusters
You must configure storage for the image registry Operator. For non-production clusters, you can set the image registry to an empty directory. If you do so, all images are lost if you restart the registry.
Procedure
To set the image registry storage to an empty directory:
$ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'
WarningConfigure this option for only non-production clusters.
If you run this command before the Image Registry Operator initializes its components, the
oc patch
command fails with the following error:Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found
Wait a few minutes and run the command again.
3.3.6. Additional resources
For more details about configuring registry storage for bare metal, see Recommended configurable storage technology.
3.4. Configuring the registry for vSphere
3.4.1. Image registry removed during installation
On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as Removed
. This allows openshift-installer
to complete installations on these platform types.
After installation, you must edit the Image Registry Operator configuration to switch the managementState
from Removed
to Managed
.
The Prometheus console provides an ImageRegistryRemoved
alert, for example:
"Image Registry has been removed. ImageStreamTags
, BuildConfigs
and DeploymentConfigs
which reference ImageStreamTags
may not work as expected. Please configure storage and update the config to Managed
state by editing configs.imageregistry.operator.openshift.io."
3.4.2. Changing the image registry’s management state
To start the image registry, you must change the Image Registry Operator configuration’s managementState
from Removed
to Managed
.
Procedure
Change
managementState
Image Registry Operator configuration fromRemoved
toManaged
. For example:$ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'
3.4.2.1. Image registry storage configuration
The image-registry
Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so the Registry Operator is made available.
Instructions for both configuring a PersistentVolume, which is required for production clusters, and for configuring an empty directory as the storage location, which is available for only non-production clusters, are shown.
3.4.3. Configuring registry storage for VMware vSphere
As a cluster administrator, following installation you must configure your registry to use storage.
Prerequisites
- Cluster administrator permissions.
- A cluster on VMware vSphere.
Provision persistent storage for your cluster. To deploy a private image registry, your storage must provide ReadWriteMany access mode.
ImportantvSphere volumes do not support the
ReadWriteMany
access mode. You must use a different storage backend, such asNFS
, to configure the registry storage.- Must have "100Gi" capacity.
Testing shows issues with using the NFS server on RHEL as storage backend for core services. This includes the OpenShift Container Registry and Quay, Prometheus for monitoring storage, and Elasticsearch for logging storage. Therefore, using RHEL NFS to back PVs used by core services is not recommended.
Other NFS implementations on the marketplace might not have these issues. Contact the individual NFS implementation vendor for more information on any testing that was possibly completed against these OpenShift Container Platform core components.
Procedure
To configure your registry to use storage, change the
spec.storage.pvc
in theconfigs.imageregistry/cluster
resource.NoteWhen using shared storage such as NFS, it is strongly recommended to use the
supplementalGroups
strategy, which dictates the allowable supplemental groups for the Security Context, rather than thefsGroup
ID. Refer to the NFS Group IDs documentation for details.Verify you do not have a registry Pod:
$ oc get pod -n openshift-image-registry
Note-
If the storage type is
emptyDIR
, the replica number cannot be greater than1
. If the storage type is
NFS
, you must enable theno_wdelay
androot_squash
mount options. For example:# cat /etc/exports /mnt/data *(rw,sync,no_wdelay,root_squash,insecure,fsid=0) sh-4.3# exportfs -rv exporting *:/mnt/data
-
If the storage type is
Check the registry configuration:
$ oc edit configs.imageregistry.operator.openshift.io storage: pvc: claim:
Leave the
claim
field blank to allow the automatic creation of animage-registry-storage
PVC.Optional: Add a new storage class to a PV:
Create the PV:
$ oc create -f -
apiVersion: v1 kind: PersistentVolume metadata: name: image-registry-pv spec: accessModes: - ReadWriteMany capacity: storage: 100Gi nfs: path: /registry server: 172.16.231.181 persistentVolumeReclaimPolicy: Retain storageClassName: nfs01
$ oc get pv
Create the PVC:
$ oc create -n openshift-image-registry -f -
apiVersion: "v1" kind: "PersistentVolumeClaim" metadata: name: "image-registry-pvc" spec: accessModes: - ReadWriteMany resources: requests: storage: 100Gi storageClassName: nfs01 volumeMode: Filesystem
$ oc get pvc -n openshift-image-registry
Finally, add the name of your PVC:
$ oc edit configs.imageregistry.operator.openshift.io -o yaml
storage: pvc: claim: image-registry-pvc 1
- 1
- Creating a custom PVC allows you to leave the
claim
field blank for default automatic creation of animage-registry-storage
PVC.
Check the
clusteroperator
status:$ oc get clusteroperator image-registry
See Group IDs for additional details about using supplemental groups to handle NFS access.
3.4.4. Configuring storage for the image registry in non-production clusters
You must configure storage for the image registry Operator. For non-production clusters, you can set the image registry to an empty directory. If you do so, all images are lost if you restart the registry.
Procedure
To set the image registry storage to an empty directory:
$ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'
WarningConfigure this option for only non-production clusters.
If you run this command before the Image Registry Operator initializes its components, the
oc patch
command fails with the following error:Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found
Wait a few minutes and run the command again.
3.4.5. Additional resources
For more details about configuring registry storage for vSphere, see Recommended configurable storage technology.