Backup and recovery for operator environments
Safeguard against data loss with backup and recovery of Ansible Automation Platform operator on OpenShift Container Platform
Abstract
Preface Copy linkLink copied to clipboard!
Thank you for your interest in Red Hat Ansible Automation Platform. Ansible Automation Platform is a commercial offering that helps teams manage complex multi-tier deployments by adding control, knowledge, and delegation to Ansible-powered environments.
Use the procedures in this guide to create backup resources that can be used for recovering your Red Hat Ansible Automation Platform deployment in the event of a failure.
Providing feedback on Red Hat documentation Copy linkLink copied to clipboard!
If you have a suggestion to improve this documentation, or find an error, you can contact technical support at https://access.redhat.com to open a request.
Chapter 1. Backup and recovery of Red Hat Ansible Automation Platform Copy linkLink copied to clipboard!
To safeguard against unexpected data loss and application errors, it is critical that you perform periodic backups of your Red Hat Ansible Automation Platform deployment. In addition to data loss prevention, backups allow you to fall back to a different deployment state.
1.1. About backup and recovery Copy linkLink copied to clipboard!
Red Hat recommends backing up deployments of Red Hat Ansible Automation Platform in your Red Hat OpenShift Container Platform environment to prevent data loss.
A backup resource of your Red Hat Ansible Automation Platform deployment includes the following:
-
Custom deployment of specific values in the
specsection of the Ansible Automation Platform custom resource object. -
Back up of the
postgresqldatabase. -
secret_key,admin_password, andbroadcast_websocketsecrets. - Database configuration.
Be sure to secure your backup resources because they can include sensitive information.
1.1.1. Backup recommendations Copy linkLink copied to clipboard!
Recovering from data loss requires that you plan for and create backup resources of your Red Hat Ansible Automation Platform deployments on a regular basis. At a minimum, Red Hat recommends backing up deployments of Red Hat Ansible Automation Platform under the following circumstances:
- Before upgrading your Red Hat Ansible Automation Platform deployments.
- Before upgrading your OpenShift cluster.
- Once per week. This is particularly important if your environment is configured for automatic upgrades.
Chapter 2. Creating Red Hat Ansible Automation Platform backup resources Copy linkLink copied to clipboard!
Backing up your Red Hat Ansible Automation Platform deployment involves creating backup resources for your deployed instances. Use the following procedures to create backup resources for your Red Hat Ansible Automation Platform deployment. We recommend taking backups before upgrading the Ansible Automation Platform Operator. Take a backup regularly in case you want to restore the platform to a previous state.
2.1. Backing up your Ansible Automation Platform deployment Copy linkLink copied to clipboard!
Regularly backing up your Ansible Automation Platform deployment is vital to protect against unexpected data loss and application errors. Ansible Automation Platform hosts any enabled components (such as, automation controller, automation hub, and Event-Driven Ansible), when you back up Ansible Automation Platform the operator will also back up these components.
Ansible Automation Platform Operator creates a PersistentVolumeClaim (PVC) for your Ansible Automation Platform Backup automatically. You can use your own pre-created PVC by using the backup_pvc spec and specifying your PVC.
Prerequisites
- You must be authenticated on OpenShift cluster.
- You have installed Ansible Automation Platform Operator on the cluster.
- You have deployed a Ansible Automation Platform instance using the Ansible Automation Platform Operator.
Procedure
- Log in to Red Hat OpenShift Container Platform.
- Navigate to → .
- Select your Ansible Automation Platform Operator deployment.
- Go to your All Instances tab, and click .
Select Ansible Automation Platform Backup from the list.
NoteWhen creating the Ansible Automation Platform Backup resource it also creates backup resources for each of the nested components that are enabled.
- In the Name field, enter a name for the backup.
- In the Deployment name field, enter the name of the deployed Ansible Automation Platform instance being backed up. For example if your Ansible Automation Platform deployment must be backed up and the deployment name is aap, enter 'aap' in the Deployment name field.
Click . This results in an AnsibleAutomationPlatformBackup resource similar to the following:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verification
To verify that your backup was successful you can:
- Log in to Red Hat OpenShift Container Platform.
- Navigate to → .
- Select your Ansible Automation Platform Operator deployment.
- Click All Instances.
The All Instances page displays the main backup and the backups for each component with the name you specified when creating your backup resource. The status for the following instances must be either Running or Successful:
- AnsibleAutomationPlatformBackup
- AutomationControllerBackup
- EDABackup
- AutomationHubBackup
Chapter 3. Recovering a Red Hat Ansible Automation Platform deployment Copy linkLink copied to clipboard!
If you lose information on your system or experience issues with an upgrade, you can use the backup resources of your deployment instances. Use the following procedures to recover your Ansible Automation Platform deployment files.
3.1. Recovering your Ansible Automation Platform deployment Copy linkLink copied to clipboard!
Ansible Automation Platform manages any enabled components (such as, automation controller, automation hub, and Event-Driven Ansible), when you recover Ansible Automation Platform you also restore these components.
In previous versions of the Ansible Automation Platform Operator, it was necessary to create a restore object for each component of the platform. Now, you create a single AnsibleAutomationPlatformRestore resource, which creates and manages the other restore objects:
- AutomationControllerRestore
- AutomationHubRestore
- EDARestore
Prerequisites
- You must be authenticated with an OpenShift cluster.
- You have installed the Ansible Automation Platform Operator on the cluster.
- The AnsibleAutomationPlatformBackups deployment is available in your cluster.
Procedure
- Log in to Red Hat OpenShift Container Platform.
- Navigate to → .
- Select your Ansible Automation Platform Operator deployment.
- Go to your All Instances tab, and click .
- Select Ansible Automation Platform Restore from the list.
- For Name enter the name for the recovery deployment.
- For New Ansible Automation Platform Name enter the new name for your Ansible Automation Platform instance.
- Backup Source defaults to CR.
- For Backup name enter the name you chose when creating the backup.
- Click .
Verification
Your backups starts restoring under the AnsibleAutomationPlatformRestores tab.
The recovery is not complete until all the resources are successfully restored. Depending on the size of your database this this can take some time.
To verify that your recovery was successful you can:
- Go to → .
- Confirm that all pods are in a Running or Completed state.
3.2. Recovering your Ansible Automation Platform deployment from a PVC Copy linkLink copied to clipboard!
A persistent volume claim (PVC) is a storage volume that stores data for automation hub and automation controller applications. These PVCs are independent of the applications and persist even if an application is deleted. You can restore data from a PVC as an alternative to recovering from an Ansible Automation Platform backup.
For more information see the Finding and deleting PVCs section of the Installing on OpenShift Container Platform guide.
Prerequisites
- You have an existing PVC containing a backup.
- You have installed the Ansible Automation Platform Operator on Red Hat OpenShift Container Platform.
Procedure
- Log in to Red Hat OpenShift Container Platform.
- Navigate to → .
- Select your Ansible Automation Platform Operator deployment.
- Go to your All Instances tab, and click .
- Select Ansible Automation Platform Restore from the list.
- For Name enter the name for the recovery deployment.
- For New Ansible Automation Platform Name enter the new name for your Ansible Automation Platform instance.
For Backup Source select PVC.
- Backup PVC: Enter the name of your PVC.
- Backup Directory: Enter the path to your backup directory on your PVC.
- For Backup name enter the name you chose when creating the backup.
Under YAML view paste in the following example:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Click .
Verification
Your backups restore under the AnsibleAutomationPlatformRestores tab.
The recovery is not complete until all the resources are successfully restored. Depending on the size of your database this this can take some time.
- Go to → .
- Confirm that all pods are in a Running or Completed state.
3.3. Recovering your Ansible Automation Platform deployment from an external database Copy linkLink copied to clipboard!
You can restore an external database on Red Hat OpenShift Container Platform using the Operator. Use the following procedure to restore from an external database.
Restoring from an external database force drops the database, which overrides your existing external database.
Prerequisites
- You have an external database.
- You have installed the Ansible Automation Platform Operator on OpenShift Container Platform.
Procedure
- Log in to Red Hat OpenShift Container Platform.
- Navigate to → .
- Select your Ansible Automation Platform Operator deployment.
- Go to your All Instances tab, and click .
- Select Ansible Automation Platform Restore from the list.
- For Name enter the name for the recovery deployment.
For New Ansible Automation Platform Name enter the new name for your Ansible Automation Platform instance.
-
If restoring to the same name Ansible Automation Platform then you must add
force_drop_db: trueto drop the database on restore.
-
If restoring to the same name Ansible Automation Platform then you must add
- Backup Source defaults to CR.
For Backup name enter the name you chose when creating the backup. Under YAML view paste in the following example:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Click .
Verification
Your backups starts restoring under the AnsibleAutomationPlatformRestores tab.
The recovery is not complete until all the resources are successfully restored. Depending on the size of your database this this can take some time.
To verify that your recovery was successful you can:
- Go to → .
- Confirm that all pods are in a Running or Completed state.
Chapter 4. Troubleshooting Copy linkLink copied to clipboard!
Use this information to diagnose and resolve issues during backup and recovery.
4.1. Automation controller custom resource has the same name as an existing deployment Copy linkLink copied to clipboard!
The name specified for the new AutomationController custom resource must not match an existing deployment or the recovery process will fail.
If your AutomationController customer resource matches an existing deployment, perform the following steps to resolve the issue.
Procedure
Delete the existing AutomationController and the associated postgres PVC:
oc delete automationcontroller <YOUR_DEPLOYMENT_NAME> -n <YOUR_NAMESPACE> oc delete pvc postgres-13-<YOUR_DEPLOYMENT_NAME>-13-0 -n <YOUR_NAMESPACE>
oc delete automationcontroller <YOUR_DEPLOYMENT_NAME> -n <YOUR_NAMESPACE> oc delete pvc postgres-13-<YOUR_DEPLOYMENT_NAME>-13-0 -n <YOUR_NAMESPACE>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use AutomationControllerRestore with the same deployment_name in it:
oc apply -f restore.yaml
oc apply -f restore.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow