Backup and restore
Backing up and restoring Red Hat Advanced Cluster Security for Kubernetes
Abstract
Chapter 1. Backing up Red Hat Advanced Cluster Security for Kubernetes Copy linkLink copied to clipboard!
You can perform data backups for Red Hat Advanced Cluster Security for Kubernetes and use these for data restoration in case of an infrastructure disaster or corrupt data.
You can configure automatic backups for the Central database by integrating with Amazon S3, S3 API compatible services, or Google Cloud Storage. You can perform on-demand backups of the Central database by using the roxctl CLI. You can also back up your Central deployment using RHACS Operator or Helm Chart installation methods.
Depending on your requirements, you can create two types of backups:
- A backup of the Central database: It includes RHACS configurations, resources, events, and certificates. In an unforeseen incident, such as database failure or data corruption, you can use the backup to recover and restore the Central database to its earlier functional state. Doing this ensures the availability and integrity of essential data, allowing you to continue normal operations without significant disruptions or loss of critical information.
- A backup of all custom deployment configurations: If you installed RHACS by using Helm charts or the RHACS Operator, you can back up settings, parameters, and customizations specific to your installation. When the RHACS installation gets accidentally deleted, or you need to migrate it to another cluster or namespace, having a backup of the deployment configurations enables a seamless recovery process. In addition, by restoring the custom settings from the backup, you can efficiently reinstate your Central installation’s unique requirements and configurations, ensuring consistent and exact deployment of the system.
1.1. Backup considerations for external databases and cloud users Copy linkLink copied to clipboard!
You must manage your backups differently if you use an external database or if you are a cloud user.
Back up with an external database
If you use an external database, you cannot use the automatic backup option or start a backup process from within Red Hat Advanced Cluster Security for Kubernetes (RHACS).
- When you upgrade your external database, you must scale down Central before you start the backup. Central attempts to connect to the database until it is successful, which can cause issues during the upgrade process.
- For a database that you manage, you must use the backup procedures that your database vendor recommends.
Back up for cloud users
If you are a cloud user, you cannot use the automatic backup option or start the backup process from an integration. Red Hat is responsible for backing up your data.
1.1.1. Backing up your PostgreSQL database and certificates Copy linkLink copied to clipboard!
By backing up your Red Hat Advanced Cluster Security for Kubernetes (RHACS) instance with an external PostgreSQL database, you can ensure the security and integrity of your data by following a vendor-recommended procedure. In this process, you back up the PostgreSQL database first, and then back up the RHACS certificates.
Procedure
To back up the PostgreSQL database, run the following command:
$ pg_dump -U <username> -d <database_name> -f <output_file_path>where:
<username>- Specifies the name of the user who has the necessary permissions to perform the dump.
<database_name>- Specifies the actual name of the database.
<output_file_path>- Specifies the desired location and name for your backup file.
To back up the RHACS certificates, run the following command:
$ roxctl central backup --certs-only=true
Because backup files include secrets and certificates, you must securely store the backup files.
1.2. Backing up Central database by using the roxctl CLI Copy linkLink copied to clipboard!
Backing up the Central database is critical to ensure data integrity and system reliability. Regular backups of the database, containing necessary configurations, resources, events, and certificates, protect against database failures, corruption, and accidental data loss.
You can use the roxctl CLI to take the backups by using the backup command. You require an API token or your administrator password to run this command.
Red Hat supports backups for the Central database through integration with Amazon S3 or Google Cloud Storage.
Backing up to S3 API compatible storage is not guaranteed to work. Red Hat does not test and support every S3 API compatible provider for backing up RHACS.
1.2.1. On-demand backups by using an API token Copy linkLink copied to clipboard!
You can back up the entire database of RHACS by using an API token.
Prerequisites
-
You have an API token with the
Adminrole. -
You have installed the
roxctlCLI.
Procedure
Set the
ROX_API_TOKENand theROX_ENDPOINTenvironment variables by running the following commands:$ export ROX_API_TOKEN=<api_token>$ export ROX_ENDPOINT=<address>:<port_number>Initiate a backup for Central by running the following command:
$ roxctl central backup1 - 1
- You can use the
--outputoption to specify the backup file location.
By default, the
roxctlCLI saves the backup file in the directory where you run the command.
1.2.2. On-demand backups by using the administrator password Copy linkLink copied to clipboard!
You can back up the entire database of RHACS by using your administrator password.
Prerequisites
- You have the administrator password.
-
You have installed the
roxctlCLI.
Procedure
Set the
ROX_ENDPOINTenvironment variable by running the following command:$ export ROX_ENDPOINT=<address>:<port_number>Initiate a backup for Central by running the following command:
$ roxctl -p <admin_password> central backup1 - 1
- For
<admin_password>, specify the administrator password.
By default, the
roxctlCLI saves the backup file in the directory in which you run the command. You can use the--outputoption to specify the backup file location.
1.3. Backing up Central deployment Copy linkLink copied to clipboard!
You can back up the deployment of a Central instance. This can be useful if you want to migrate central to another namespace or cluster by using the same configuration values.
Red Hat does not support backing up deployment configurations by using the roxctl CLI. You can use the oc or kubectl CLI to back up manifests related to your Central instance and restore the configuration.
1.3.1. Backing up deployment using the RHACS Operator Copy linkLink copied to clipboard!
When you use the RHACS Operator to instal RHACS, OpenShift Container Platform stores all the custom configuration for your Central deployment within the Central custom resource. You can backup the Central custom resource, the central-tls secret, and the administrator password. The central-tls secret includes the certificates for authenticating with Secured clusters and signing API tokens.
Procedure
Run the following command to save the Central custom resource in a YAML file:
$ oc get central -n _<central-namespace>_ _<central-name>_ -o yaml > central-cr.yamlRun the following command to save
central-tlsin a JSON file:$ oc get secret -n _<central-namespace>_ central-tls -o json | jq 'del(.metadata.ownerReferences)' > central-tls.jsonRun the following command to the administrator password in a JSON file:
$ oc get secret -n _<central-namespace>_ central-htpasswd -o json | jq 'del(.metadata.ownerReferences)' > central-htpasswd.json
1.3.2. Backing up deployment using Helm Copy linkLink copied to clipboard!
When you use the Helm chart to install RHACS, you store all the custom configuration for your Central deployment within the custom values that you apply to the Helm chart.
You can back up the custom values and save it in a YAML file.
Procedure
Run the following command to back up custom Helm chart values in a YAML file:
$ helm get values --all -n _<central-namespace>_ _<central-helm-release>_ -o yaml > central-values-backup.yaml
Chapter 2. Restoring from a backup Copy linkLink copied to clipboard!
You can restore Red Hat Advanced Cluster Security for Kubernetes (RHACS) from an existing backup by using the roxctl command-line interface (CLI).
Depending upon your requirements and the data you have backed up, you can restore from the following types of backups:
- Restore an external PostgreSQL database: Use this to restore your RHACS instance when you use an external PostgreSQL database. You can recover your system data while maintaining your existing database setup.
- Restore the Central certificates and redeploy Central: Use this to restore the Central certificates and redeploy Central after a database restoration. This ensures that authentication certificates for secured clusters and API tokens remain valid.
- Restore Central database from the Central database backup: Use this to recover from a database failure or data corruption event. You can restore and recover the Central database to its earlier functional state.
- Restore Central from the Central deployment backup: Use this if you are migrating Central to another cluster or namespace. This option restores the configurations of your Central installation.
2.1. Restoring an external PostgreSQL database Copy linkLink copied to clipboard!
By restoring an external PostgreSQL database, you can recover your Red Hat Advanced Cluster Security for Kubernetes (RHACS) instance from a backup while maintaining your existing database setup.
This procedure focuses on restoring the database that contains your RHACS system data. You can perform a PostgreSQL database restore by using the vendor-recommended pg_restore command.
Prerequisites
-
You have scaled down Central to
0.
Procedure
To perform a PostgreSQL restore, run the following command:
$ pg_restore -U <username> -d <database_name> <path_to_backup_file>where:
<username>- Specifies the actual user name.
<database_name>- Specifies the name of the target database.
<path_to_backup_file>-
Specifies the location and name of the file you want to restore. You can use
.tar,.gz, or another custom file format.
After you restore the external PostgreSQL database, you must restore the RHACS Central certificates.
2.2. Restoring the Central certificates and redeploying Central Copy linkLink copied to clipboard!
You must restore the Central certificates and redeploy Central to ensure that authentication certificates for secured clusters and API tokens remain valid for the redeployed Central instance.
Procedure
Run the
roxctl central generate interactivecommand and provide the path to your backup file, which creates a central-bundle folder with the necessary manifests and scripts.You can then use these files to install Central, ensuring that all your authentication certificates and API tokens remain valid.
For more information, see "Restore certificates using the roxctl CLI".
2.3. Restoring Central database by using the roxctl CLI Copy linkLink copied to clipboard!
You can use the roxctl CLI to restore Red Hat Advanced Cluster Security for Kubernetes by using the restore command. You require an API token or your administrator password to run this command.
2.3.1. Restoring by using an API token Copy linkLink copied to clipboard!
You can restore the entire database of RHACS by using an API token.
Prerequisites
- You have a RHACS backup file.
- You have an API token with the administrator role.
-
You have installed the
roxctlCLI.
Procedure
Set the
ROX_API_TOKENand theROX_ENDPOINTenvironment variables by running the following commands:$ export ROX_API_TOKEN=<api_token>$ export ROX_ENDPOINT=<address>:<port_number>Restore the Central database by running the following command:
$ roxctl central db restore <backup_file>1 - 1
- For
<backup_file>, specify the name of the backup file that you want to restore.
2.3.2. Restoring by using the administrator password Copy linkLink copied to clipboard!
You can restore the entire database of RHACS by using your administrator password.
Prerequisites
- You have a RHACS backup file.
- You have the administrator password.
-
You have installed the
roxctlCLI.
Procedure
Set the
ROX_ENDPOINTenvironment variable by running the following command:$ export ROX_ENDPOINT=<address>:<port_number>Restore the Central database by running the following command:
$ roxctl -p <admin_password> \1 central db restore <backup_file>2
2.3.3. Resuming the restore operation Copy linkLink copied to clipboard!
If your connection is interrupted during a restore operation or you need to go offline, you can resume the restore operation.
-
If you do not have access to the machine running the resume operation, you can use the
roxctl central db restore statuscommand to check the status of an ongoing restore operation. -
If the connection is interrupted, the
roxctlCLI automatically attempts to restore a task as soon as the connection is available again. The automatic connection retries depend on the duration specified by thetimeoutoption. -
Use the
--timeoutoption to specify the time in seconds, minutes or hours after which theroxctlCLI stops trying to resume a restore operation. If the option is not specified, the default timeout is 10 minutes. -
If a restore operation gets stuck or you want to cancel it, use the
roxctl central db restore cancelcommand to cancel a running restore operation. - If a restore operation is stuck, you have canceled it, or the time has expired, you can resume the previous restore by running the original command again.
- During interruptions, RHACS caches an ongoing restore operation for 24 hours. You can resume this operation by executing the original restore command again.
-
The
--timeoutoption only controls the client-side connection retries and has no effect on the server-side restore cache of 24 hours. - You cannot resume restores across Central pod restarts.
- If a restore operation is interrupted, you must restart it within 24 hours and before restarting Central, otherwise RHACS cancels the restore operation.
2.4. Restoring Central deployment using the roxctl CLI Copy linkLink copied to clipboard!
You can restore your Central deployment to its original configuration by using the backups you made.
You must first restore certificates by using the roxctl CLI, and then restore the Central deployment by running the Central installation scripts.
2.4.1. Restore certificates using the roxctl CLI Copy linkLink copied to clipboard!
Use the roxctl CLI to generate Kubernetes manifests to install the RHACS Central component to your cluster. Doing this allows you to ensure that authentication certificates for Secured clusters and the API tokens remain valid for the restored version. If you backed up another instance of RHACS Central, you can use the certificate files from that backup.
With the roxctl CLI, you can not restore the entire Central deployment. Instead, first you use the roxctl CLI to generate new manifests using the certificates in your central data backup. Afterwards, you use those manifests to install Central.
Prerequisites
- You must have the Red Hat Advanced Cluster Security for Kubernetes backup file.
-
You must have installed the
roxctlCLI.
Procedure
Run the interactive install command:
$ roxctl central generate interactiveFor the following prompt, enter the path of the Red Hat Advanced Cluster Security for Kubernetes backup file:
Enter path to the backup bundle from which to restore keys and certificates (optional): _<backup-file-path>_- For other following prompts, press Enter to accept the default value or enter custom values as required.
On completion, the interactive install command creates a folder named central-bundle, which has the necessary YAML manifests and scripts to deploy Central.
2.4.2. Running the Central installation scripts Copy linkLink copied to clipboard!
After you run the interactive installer, you can run the setup.sh script to install Central.
Procedure
Run the
setup.shscript to configure image registry access:$ ./central-bundle/central/scripts/setup.shTo enable the policy as code feature (Technology Preview), manually apply the
config.stackrox.ioCRD that is located in the .zip file athelm/chart/crds/config.stackrox.io_securitypolicies.yaml.ImportantPolicy as code is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
To apply the CRD, run the following command:
- Create the necessary resources:
- Check the deployment progress:
After Central is running, find the RHACS portal IP address and open it in your browser. Depending on the exposure method you selected when answering the prompts, use one of the following methods to get the IP address.
Expand Exposure method Command Address Example Route
oc -n stackrox get route centralThe address under the
HOST/PORTcolumn in the outputhttps://central-stackrox.example.routeNode Port
oc get node -owide && oc -n stackrox get svc central-loadbalancerIP or hostname of any node, on the port shown for the service
https://198.51.100.0:31489Load Balancer
oc -n stackrox get svc central-loadbalancerEXTERNAL-IP or hostname shown for the service, on port 443
https://192.0.2.0None
central-bundle/central/scripts/port-forward.sh 8443https://localhost:8443https://localhost:8443
If you have selected autogenerated password during the interactive install, you can run the following command to see it for logging into Central:
$ cat central-bundle/password
2.5. Restore Central deployment using the RHACS Operator Copy linkLink copied to clipboard!
You can restore your Central deployment to its original configuration by using the RHACS Operator. To successfully restore, you need the backup of your Central custom resource, central-tls, and the administrator password.
Prerequisites
-
You must have the
central-tlsbackup file. - You must have the Central custom resource backup file.
- You must have the administrator password backup file.
Procedure
Use the
central-tlsbackup file to create resources:$ oc apply -f central-tls.jsonUse the
central-htpasswdbackup file to create secrets:$ oc apply -f central-htpasswd.jsonUse the
central-cr.yamlfile to create the Central deployment:$ oc apply -f central-cr.yaml
2.6. Restore Central deployment using Helm Copy linkLink copied to clipboard!
You can restore your Central deployment to its original configuration by using Helm. To successfully restore, you need the backup of your Central custom resource, the central-tls secret, and the administrator password.
Prerequisites
- You must have the Helm values backup file.
- You must have a Red Hat Advanced Cluster Security for Kubernetes backup file.
-
You must have installed the
roxctlCLI.
Procedure
Generate
values-private.yamlfrom the RHACS database backup file:$ roxctl central generate k8s pvc --backup-bundle _<path-to-backup-file>_ --output-format "helm-values"Run the
helm installcommand and specify your backup files:$ helm install -n stackrox --create-namespace stackrox-central-services rhacs/central-services -f central-values-backup.yaml -f central-bundle/values-private.yaml
2.7. Restoring central to another cluster or namespace Copy linkLink copied to clipboard!
You can use the backups of the RHACS Central database and the deployment to restore Central to another cluster or namespace.
The following list provides a high-level overview of installation steps:
Depending upon your installation method, you must first restore Central deployment by following the instructions in the following topics:
Important- Make sure to use the backed-up Central certificates so that secured clusters and API tokens issued by the old Central instance remain valid.
- If you are deploying to another namespace, you must change the namespace in backed-up resources or commands.
- Restore Central database by following the instruction in the Restoring Central database by using the roxctl CLI topic.
- If you have an external DNS entry pointing to your old RHACS Central instance, you must reconfigure it to point to the new RHACS Central instance that you create.