Questo contenuto non è disponibile nella lingua selezionata.
Chapter 16. Backing up and restoring Red Hat Quay managed by the Red Hat Quay Operator
Use the content within this section to back up and restore Red Hat Quay when managed by the Red Hat Quay Operator on OpenShift Container Platform.
16.1. Backing up Red Hat Quay Copia collegamentoCollegamento copiato negli appunti!
This procedure describes how to create a backup of Red Hat Quay deployed on OpenShift Container Platform using the Red Hat Quay Operator
Prerequisites
-
A healthy Red Hat Quay deployment on OpenShift Container Platform using the Red Hat Quay Operator (status condition
Availableis set totrue) -
The components
quay,postgresandobjectstorageare set tomanaged: true -
If the component
clairis set tomanaged: truethe componentclairpostgresis also set tomanaged: true(starting with Red Hat Quay Operator v3.7 or later)
If your deployment contains partially unmanaged database or storage components and you are using external services for Postgres or S3-compatible object storage to run your Red Hat Quay deployment, you must refer to the service provider or vendor documentation to create a backup of the data. You can refer to the tools described in this guide as a starting point on how to backup your external Postgres database or object storage.
16.1.1. Red Hat Quay configuration backup Copia collegamentoCollegamento copiato negli appunti!
Backup the
QuayRegistrycustom resource by exporting it:$ oc get quayregistry <quay-registry-name> -n <quay-namespace> -o yaml > quay-registry.yamlEdit the resulting
quayregistry.yamland remove the status section and the following metadata fields:metadata.creationTimestamp metadata.finalizers metadata.generation metadata.resourceVersion metadata.uidBackup the managed keys secret:
NoteIf you are running a version older than Red Hat Quay 3.7.0, this step can be skipped. Some secrets are automatically generated while deploying Quay for the first time. These are stored in a secret called
<quay-registry-name>-quay-registry-managed-secret-keysin the namespace of theQuayRegistryresource.$ oc get secret -n <quay-namespace> <quay-registry-name>-quay-registry-managed-secret-keys -o yaml > managed-secret-keys.yamlEdit the the resulting
managed-secret-keys.yamlfile and remove the entrymetadata.ownerReferences. Yourmanaged-secret-keys.yamlfile should look similar to the following:apiVersion: v1 kind: Secret type: Opaque metadata: name: <quayname>-quay-registry-managed-secret-keys namespace: <quay-namespace> data: CONFIG_EDITOR_PW: <redacted> DATABASE_SECRET_KEY: <redacted> DB_ROOT_PW: <redacted> DB_URI: <redacted> SECRET_KEY: <redacted> SECURITY_SCANNER_V4_PSK: <redacted>All information under the
dataproperty should remain the same.Backup the current Quay configuration:
$ oc get secret -n <quay-namespace> $(oc get quayregistry <quay-registry-name> -n <quay-namespace> -o jsonpath='{.spec.configBundleSecret}') -o yaml > config-bundle.yamlBackup the
/conf/stack/config.yamlfile mounted inside of the Quay pods:$ oc exec -it quay-pod-name -- cat /conf/stack/config.yaml > quay-config.yaml
16.1.2. Scale down your Red Hat Quay deployment Copia collegamentoCollegamento copiato negli appunti!
This step is needed to create a consistent backup of the state of your Red Hat Quay deployment. Do not omit this step, including in setups where Postgres databases and/or S3-compatible object storage are provided by external services (unmanaged by the Operator).
For Operator version 3.7 and newer: Scale down the Red Hat Quay deployment by disabling auto scaling and overriding the replica count for Red Hat Quay, mirror workers, and Clair (if managed). Your
QuayRegistryresource should look similar to the following:apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: registry namespace: ns spec: components: … - kind: horizontalpodautoscaler managed: false1 - kind: quay managed: true overrides:2 replicas: 0 - kind: clair managed: true overrides: replicas: 0 - kind: mirror managed: true overrides: replicas: 0 …For Operator version 3.6 and earlier: Scale down the Red Hat Quay deployment by scaling down the Red Hat Quay Operator first and then the managed Red Hat Quay resources:
$ oc scale --replicas=0 deployment $(oc get deployment -n <quay-operator-namespace>|awk '/^quay-operator/ {print $1}') -n <quay-operator-namespace> $ oc scale --replicas=0 deployment $(oc get deployment -n <quay-namespace>|awk '/quay-app/ {print $1}') -n <quay-namespace> $ oc scale --replicas=0 deployment $(oc get deployment -n <quay-namespace>|awk '/quay-mirror/ {print $1}') -n <quay-namespace> $ oc scale --replicas=0 deployment $(oc get deployment -n <quay-namespace>|awk '/clair-app/ {print $1}') -n <quay-namespace>Wait for the
registry-quay-app,registry-quay-mirrorandregistry-clair-apppods (depending on which components you set to be managed by the Red Hat Quay Operator) to disappear. You can check their status by running the following command:$ oc get pods -n <quay-namespace>Example output:
$ oc get pod quay-operator.v3.7.1-6f9d859bd-p5ftc 1/1 Running 0 12m quayregistry-clair-postgres-7487f5bd86-xnxpr 1/1 Running 1 (12m ago) 12m quayregistry-quay-app-upgrade-xq2v6 0/1 Completed 0 12m quayregistry-quay-config-editor-6dfdcfc44f-hlvwm 1/1 Running 0 73s quayregistry-quay-database-859d5445ff-cqthr 1/1 Running 0 12m quayregistry-quay-redis-84f888776f-hhgms 1/1 Running 0 12m
16.1.3. Red Hat Quay managed database backup Copia collegamentoCollegamento copiato negli appunti!
If your Red Hat Quay deployment is configured with external (unmanged) Postgres database(s), refer to your vendor’s documentation on how to create a consistent backup of these databases.
Identify the Quay PostgreSQL pod name:
$ oc get pod -l quay-component=postgres -n <quay-namespace> -o jsonpath='{.items[0].metadata.name}'Example output:
quayregistry-quay-database-59f54bb7-58xs7Obtain the Quay database name:
$ oc -n <quay-namespace> rsh $(oc get pod -l app=quay -o NAME -n <quay-namespace> |head -n 1) cat /conf/stack/config.yaml|awk -F"/" '/^DB_URI/ {print $4}' quayregistry-quay-databaseDownload a backup database:
$ oc exec quayregistry-quay-database-59f54bb7-58xs7 -- /usr/bin/pg_dump -C quayregistry-quay-database > backup.sql
16.1.3.1. Red Hat Quay managed object storage backup Copia collegamentoCollegamento copiato negli appunti!
The instructions in this section apply to the following configurations:
- Standalone, multi-cloud object gateway configurations
- OpenShift Data Foundations storage requires that the Red Hat Quay Operator provisioned an S3 object storage bucket from, through the ObjectStorageBucketClaim API
If your Red Hat Quay deployment is configured with external (unmanged) object storage, refer to your vendor’s documentation on how to create a copy of the content of Quay’s storage bucket.
Decode and export the
AWS_ACCESS_KEY_ID:$ export AWS_ACCESS_KEY_ID=$(oc get secret -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.AWS_ACCESS_KEY_ID}' |base64 -d)Decode and export the
AWS_SECRET_ACCESS_KEY_ID:$ export AWS_SECRET_ACCESS_KEY=$(oc get secret -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.AWS_SECRET_ACCESS_KEY}' |base64 -d)Create a new directory and copy all blobs to it:
$ mkdir blobs $ aws s3 sync --no-verify-ssl --endpoint https://$(oc get route s3 -n openshift-storage -o jsonpath='{.spec.host}') s3://$(oc get cm -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.BUCKET_NAME}') ./blobs
16.1.4. Scale the Red Hat Quay deployment back up Copia collegamentoCollegamento copiato negli appunti!
For Operator version 3.7 and newer: Scale up the Red Hat Quay deployment by re-enabling auto scaling, if desired, and removing the replica overrides for Quay, mirror workers and Clair as applicable. Your
QuayRegistryresource should look similar to the following:apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: registry namespace: ns spec: components: … - kind: horizontalpodautoscaler managed: true1 - kind: quay2 managed: true - kind: clair managed: true - kind: mirror managed: true …For Operator version 3.6 and earlier: Scale up the Red Hat Quay deployment by scaling up the Red Hat Quay Operator again:
$ oc scale --replicas=1 deployment $(oc get deployment -n <quay-operator-namespace> | awk '/^quay-operator/ {print $1}') -n <quay-operator-namespace>Check the status of the Red Hat Quay deployment:
$ oc wait quayregistry registry --for=condition=Available=true -n <quay-namespace>Example output:
apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: ... name: registry namespace: <quay-namespace> ... spec: ... status: - lastTransitionTime: '2022-06-20T05:31:17Z' lastUpdateTime: '2022-06-20T17:31:13Z' message: All components reporting as healthy reason: HealthChecksPassing status: 'True' type: Available
16.2. Restoring Red Hat Quay Copia collegamentoCollegamento copiato negli appunti!
This procedure is used to restore Red Hat Quay when the Red Hat Quay Operator manages the database. It should be performed after a backup of your Red Hat Quay registry has been performed. See Backing up Red Hat Quay for more information.
Prerequisites
- Red Hat Quay is deployed on OpenShift Container Platform using the Red Hat Quay Operator.
- A backup of the Red Hat Quay configuration managed by the Red Hat Quay Operator has been created following the instructions in the Backing up Red Hat Quay section
- Your Red Hat Quay database has been backed up.
- The object storage bucket used by Red Hat Quay has been backed up.
-
The components
quay,postgresandobjectstorageare set tomanaged: true -
If the component
clairis set tomanaged: true, the componentclairpostgresis also set tomanaged: true(starting with Red Hat Quay Operator v3.7 or later) - There is no running Red Hat Quay deployment managed by the Red Hat Quay Operator in the target namespace on your OpenShift Container Platform cluster
If your deployment contains partially unmanaged database or storage components and you are using external services for Postgres or S3-compatible object storage to run your Red Hat Quay deployment, you must refer to the service provider or vendor documentation to restore their data from a backup prior to restore Red Hat Quay
16.2.1. Restoring Red Hat Quay and its configuration from a backup Copia collegamentoCollegamento copiato negli appunti!
These instructions assume you have followed the process in the Backing up Red Hat Quay guide and create the backup files with the same names.
Restore the backed up Red Hat Quay configuration and the generated keys from the backup:
$ oc create -f ./config-bundle.yaml $ oc create -f ./managed-secret-keys.yamlImportantIf you receive the error
Error from server (AlreadyExists): error when creating "./config-bundle.yaml": secrets "config-bundle-secret" already exists, you must delete your existing resource with$ oc delete Secret config-bundle-secret -n <quay-namespace>and recreate it with$ oc create -f ./config-bundle.yaml.Restore the
QuayRegistrycustom resource:$ oc create -f ./quay-registry.yamlCheck the status of the Red Hat Quay deployment and wait for it to be available:
$ oc wait quayregistry registry --for=condition=Available=true -n <quay-namespace>
16.2.2. Scale down your Red Hat Quay deployment Copia collegamentoCollegamento copiato negli appunti!
For Operator version 3.7 and newer: Scale down the Red Hat Quay deployment by disabling auto scaling and overriding the replica count for Quay, mirror workers and Clair (if managed). Your
QuayRegistryresource should look similar to the following:apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: registry namespace: ns spec: components: … - kind: horizontalpodautoscaler managed: false1 - kind: quay managed: true overrides:2 replicas: 0 - kind: clair managed: true overrides: replicas: 0 - kind: mirror managed: true overrides: replicas: 0 …For Operator version 3.6 and earlier: Scale down the Red Hat Quay deployment by scaling down the Red Hat Quay Operator first and then the managed Red Hat Quay resources:
$ oc scale --replicas=0 deployment $(oc get deployment -n <quay-operator-namespace>|awk '/^quay-operator/ {print $1}') -n <quay-operator-namespace> $ oc scale --replicas=0 deployment $(oc get deployment -n <quay-namespace>|awk '/quay-app/ {print $1}') -n <quay-namespace> $ oc scale --replicas=0 deployment $(oc get deployment -n <quay-namespace>|awk '/quay-mirror/ {print $1}') -n <quay-namespace> $ oc scale --replicas=0 deployment $(oc get deployment -n <quay-namespace>|awk '/clair-app/ {print $1}') -n <quay-namespace>Wait for the
registry-quay-app,registry-quay-mirrorandregistry-clair-apppods (depending on which components you set to be managed by Operator) to disappear. You can check their status by running the following command:$ oc get pods -n <quay-namespace>Example output:
registry-quay-config-editor-77847fc4f5-nsbbv 1/1 Running 0 9m1s registry-quay-database-66969cd859-n2ssm 1/1 Running 0 6d1h registry-quay-redis-7cc5f6c977-956g8 1/1 Running 0 5d21h
16.2.3. Restore your Red Hat Quay database Copia collegamentoCollegamento copiato negli appunti!
Identify your Quay database pod:
$ oc get pod -l quay-component=postgres -n <quay-namespace> -o jsonpath='{.items[0].metadata.name}'Example output:
quayregistry-quay-database-59f54bb7-58xs7Upload the backup by copying it from the local environment and into the pod:
$ oc cp ./backup.sql -n <quay-namespace> registry-quay-database-66969cd859-n2ssm:/tmp/backup.sqlOpen a remote terminal to the database:
$ oc rsh -n <quay-namespace> registry-quay-database-66969cd859-n2ssmEnter psql:
bash-4.4$ psqlYou can list the database by running the following command:
postgres=# \lExample output:
List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges ----------------------------+----------------------------+----------+------------+------------+----------------------- postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | quayregistry-quay-database | quayregistry-quay-database | UTF8 | en_US.utf8 | en_US.utf8 |Drop the database:
postgres=# DROP DATABASE "quayregistry-quay-database";Example output:
DROP DATABASEExit the postgres CLI to re-enter bash-4.4:
\qRedirect your PostgreSQL database to your backup database:
sh-4.4$ psql < /tmp/backup.sqlExit bash:
sh-4.4$ exit
16.2.4. Restore your Red Hat Quay object storage data Copia collegamentoCollegamento copiato negli appunti!
Export the
AWS_ACCESS_KEY_ID:$ export AWS_ACCESS_KEY_ID=$(oc get secret -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.AWS_ACCESS_KEY_ID}' |base64 -d)Export the
AWS_SECRET_ACCESS_KEY:$ export AWS_SECRET_ACCESS_KEY=$(oc get secret -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.AWS_SECRET_ACCESS_KEY}' |base64 -d)Upload all blobs to the bucket by running the following command:
$ aws s3 sync --no-verify-ssl --endpoint https://$(oc get route s3 -n openshift-storage -o jsonpath='{.spec.host}') ./blobs s3://$(oc get cm -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.BUCKET_NAME}')
16.2.5. Scale up your Red Hat Quay deployment Copia collegamentoCollegamento copiato negli appunti!
For Operator version 3.7 and newer: Scale up the Red Hat Quay deployment by re-enabling auto scaling, if desired, and removing the replica overrides for Quay, mirror workers and Clair as applicable. Your
QuayRegistryresource should look similar to the following:apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: registry namespace: ns spec: components: … - kind: horizontalpodautoscaler managed: true1 - kind: quay2 managed: true - kind: clair managed: true - kind: mirror managed: true …For Operator version 3.6 and earlier: Scale up the Red Hat Quay deployment by scaling up the Red Hat Quay Operator again:
$ oc scale --replicas=1 deployment $(oc get deployment -n <quay-operator-namespace> | awk '/^quay-operator/ {print $1}') -n <quay-operator-namespace>Check the status of the Red Hat Quay deployment:
$ oc wait quayregistry registry --for=condition=Available=true -n <quay-namespace>Example output:
apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: ... name: registry namespace: <quay-namespace> ... spec: ... status: - lastTransitionTime: '2022-06-20T05:31:17Z' lastUpdateTime: '2022-06-20T17:31:13Z' message: All components reporting as healthy reason: HealthChecksPassing status: 'True' type: Available