Chapter 10. Upgrading your deployment
You can upgrade your existing Ansible Automation Platform from GCP Marketplace deployment to the newer version. The upgrade process covers upgrading of automation hub and automation controller. The upgrade process takes roughly the same amount of time as a Ansible Automation Platform from GCP Marketplace deployment install.
Ansible Automation Platform from GCP Marketplace supports sequential upgrades. All upgrades should be no more than one major version behind the version you are currently upgrading to. For example, to upgrade to Ansible Automation Platform from GCP Marketplace to 2.3.20230221-00, you must be on version 2.2.20230215-00.
Prerequisites
- Docker must be installed to run the upgrade playbook.
- The upgrade process requires several volumes to be mounted. Prepare a fresh directory to be used for this process.
The following procedures form the upgrade process:
Backup your Ansible Automation Platform 2.2 stack.
-
Pull the
ansible-on-clouds-ops
2.2 container image - Prepare the environment
-
Run the
ansible-on-clouds-ops
2.2 container to backup the stack
-
Pull the
Upgrade Ansible Automation Platform.
-
Pull the
ansible-on-clouds-ops
2.3 container image -
Generate data files by running the
ansible-on-clouds-ops
container - Update the data file
-
Run the
ansible-on-clouds-ops
2.3 container to upgrade your Ansible Automation Platform from GCP Marketplace deployment
-
Pull the
(Optional) Restore the stack from backup.
-
Pull the
ansible-on-clouds-ops
2.2 container image - Prepare the environment
-
Run the
ansible-on-clouds-ops
2.2 container to restore the stack
-
Pull the
10.1. Backing up before the upgrade
The following procedures backup the deployment before the upgrade process begins.
10.1.1. Pulling the ansible-on-clouds-ops 2.2 container image
Procedure
Pull the
ansible-on-clouds-ops 2.2
container image with the same tag version as the foundation deployment.NoteBefore pulling the docker image, make sure you are logged in to registry.redhat.com using docker. Use the following command to login to registry.redhat.com.
$ docker login registry.redhat.io
For more information about registry login, see Registry Authentication
$ export IMAGE=registry.redhat.io/ansible-on-clouds/ansible-on-clouds-ops-rhel8:2.2.20230215 $ docker pull $IMAGE --platform=linux/amd64
For EMEA regions (Europe, Middle East, Africa) run the following command instead:
$ export IMAGE=registry.redhat.io/ansible-on-clouds/ansible-on-clouds-ops-emea-rhel8:2.2.20230215 $ docker pull $IMAGE --platform=linux/amd64
10.1.2. Preparing the backup environment
Procedure
Create the directories used to configure the upgrade process using the following commands:
# Secrets directory. Store the GCP credentials file. $ mkdir secrets # Extra vars directory. All the configuration variables for the upgrade process. $ mkdir extra_vars
10.1.3. Backing up your Ansible Automation Platform stack
Procedure
Store your service account credential JSON file in the
/secrets
directory.NoteTo create the GCP credentials file, see, Create credentials and Create and delete service account keys
The
extra_vars.yml
file contains the following input variables within theextra_vars
directory:gcp_service_account_credentials_json_path
is the service account credential JSON file path for the Google cloud account. This is the path within anansible-on-clouds-ops
container where the credentials JSON file is mounted to the/secrets
directory. The path should be/secrets/<service-account-creds-json-filename>
.Replace
<service-account-creds-json-filename>
with your Google service account credential filename.-
gcp_deployment_name
is the deployment name that you want to upgrade. -
gcp_compute_region
is the GCP region that you provided when deploying foundation deployment. If you have not provided a region when deploying the foundation, use the default regionus-east1
here. gcp_compute_zone
is the GCP zone that you provided when deploying foundation deployment. If you have not provided a zone when deploying the foundation, use the defaultus-east1-b
here.NoteYou can find the region and zone from the filestore instance deployed with the foundation deployment. Look for the filestore with the name
<deployment-name>-filestore
.-
Make note of the location field that is your
gcp_compute_zone
. For example, the filestore instance location might beus-east1-d
-
Remove the last two characters from the location that is your
gcp_compute_region
. For example, if yourgcp_compute_zone
location isus-east1-d
, your region isus-east1
.
gcp_service_account_credentials_json_path: "/secrets/<service-account-creds-json-filename>" gcp_deployment_name: "" gcp_compute_region: "" gcp_compute_zone: ""
-
Make note of the location field that is your
The final result should look similar to the following:
$ tree tree . ├── extra_vars │ └── extra_vars.yaml └── secrets └── gc-ansible-cloud-123434.json
10.1.4. Running the backup playbook as a container
Procedure
-
Replace
<absolute path to Google application credentials dir>
with the path to the service account credential JSON file for the Google cloud. -
Replace
<absolute path of extra vars dir>
with the path to the directory where theextra_vars.yml
file you created in step 1 of the backup process. Use the following command to run the playbook:
$ docker run --rm \ -v <absolute path to google application credentials dir>:/secrets:ro \ -v <absolute path of extra vars dir>:/extra_vars:ro \ $IMAGE \ redhat.ansible_on_clouds.gcp_backup_deployment \ -e @/extra_vars/extra_vars.yaml
Note the output of the playbook.
The playbook output resembles the following:
TASK [redhat.ansible_on_clouds.standalone_gcp_backup : [backup_deployment] Print vars required for restore process] *** ok: [localhost] => { "msg": [ "AAP on GCP Backup successful. Please note below vars which are required for restore process.", "{ gcp_sql_backup_id: 1677552759614 , gcp_sql_backup_db_name: test-bnr-aap-db ,gcp_filestore_backup_name: test-bnr-filestore-iygs }" ] }
ImportantMake a note of the SQL database backup ID, SQL database backup name and the filestore backup name. You will require these for the restore playbook.
10.2. Upgrading Ansible Automation Platform
The following procedures form the upgrading process for Ansible Automation Platform from GCP Marketplace.
10.2.1. Pulling the ansible-on-clouds-ops 2.3 container image
Procedure
Pull the docker image for the
ansible-on-clouds-ops
2.3 container with the same tag as the version you want to upgrade to.NoteBefore pulling the docker image, make sure you are logged in to registry.redhat.com using docker. Use the following command to login to registry.redhat.com.
$ docker login registry.redhat.io
For more information about registry login, see Registry Authentication
NoteThe
ansible-on-clouds-ops
container image tag must match the version that you want to upgrade to. For example, if your foundation deployment version is 2.2.20230215-00, pull theansible-on-clouds-ops
image with tag 2.3.20230221 to upgrade to version 2.3.20230221.$ export IMAGE=registry.redhat.io/ansible-on-clouds/ansible-on-clouds-ops-rhel8:2.3.20230221 $ docker pull $IMAGE --platform=linux/amd64
For EMEA regions (Europe, Middle East, Africa) run the following command instead:
$ export IMAGE=registry.redhat.io/ansible-on-clouds/ansible-on-clouds-ops-emea-rhel8:2.3.20230221 $ docker pull $IMAGE --platform=linux/amd64
10.2.2. Generating the data file
Procedure
Run the following commands to generate the required data file. These commands create a directory, and populate it with an empty data template that, when populated, is used during the upgrade.
# Create a folder to hold the configuration files $ mkdir command_generator_data # Populate command_generator_data folder with configuration file template $ docker run --rm -v $(pwd)/command_generator_data:/data $IMAGE \ command_generator_vars gcp_upgrade \ --output-data-file /data/extra_vars.yml
After running the previous commands, a
command_generator_data/extra_vars.yml
template is created. This template file should resemble:gcp_upgrade: ansible_config_path: cloud_credentials_path: deployment_name: extra_vars: gcp_backup_taken: gcp_compute_region: gcp_compute_zone:
10.2.3. Updating the data file
You must populate the data file before triggering the upgrade. The following variables are parameters listed in the data file.
-
ansible_config_path
(Optional) Only use if overriding with a custom ansible_config. -
cloud_credentials_path
is the path to your GCP credentials file. -
deployment_name
is the name of the foundation deployment. This is the same name you used when you deployed the foundation. -
gcp_backup_taken
is the verification that a manual backup of the current deployment was recently created prior to running this upgrade. Usetrue
here to verify a recent backup was created. -
gcp_compute_region
is the GCP region that you provided when deploying foundation deployment. If you have not provided a region when deploying the foundation, use the default regionus-east1
here. -
gcp_compute_zone
is the GCP zone that you provided when deploying foundation deployment. If you have not provided a zone when deploying the foundation, use the defaultus-east1-b
here.
After populating the data file, it should resemble the following.
The following values are provided as examples:
gcp_upgrade: ansible_config_path: cloud_credentials_path: ~/secrets/GCP-secrets.json deployment_name: AnsibleAutomationPlatform extra_vars: gcp_backup_taken: true gcp_compute_region: us-east1 gcp_compute_zone: us-east1-b
10.2.4. Running the upgrade playbook
To run the upgrade, run the command generator to generate the upgrade CLI command:
$ docker run --rm -v $(pwd)/command_generator_data:/data $IMAGE command_generator --data-file /data/extra_vars.yml
This generates the following command:
----------------------------------------------- docker run --rm --env PLATFORM=GCP -v ~/secrets/GCP-secrets.json:/home/runner/.gcp/credentials:ro --env ANSIBLE_CONFIG=../gcp-ansible.cfg --env DEPLOYMENT_NAME=AnsibleAutomationPlatform --env GENERATE_INVENTORY=true $IMAGE redhat.ansible_on_clouds.gcp_upgrade \ -e 'gcp_deployment_name=AnsibleAutomationPlatform \ gcp_service_account_credentials_json_path=/home/runner/.gcp/credentials \ gcp_compute_region=us-east1 gcp_compute_zone=us-east1-b gcp_backup_taken=True' ===============================================
Run the given upgrade command to trigger the upgrade.
$ docker run --rm --env PLATFORM=GCP -v ~/secrets/GCP-secrets.json:/home/runner/.gcp/credentials:ro \ --env ANSIBLE_CONFIG=../gcp-ansible.cfg \ --env DEPLOYMENT_NAME=AnsibleAutomationPlatform \ --env GENERATE_INVENTORY=true $IMAGE redhat.ansible_on_clouds.gcp_upgrade \ -e 'gcp_deployment_name=AnsibleAutomationPlatform \ gcp_service_account_credentials_json_path=/home/runner/.gcp/credentials \ gcp_compute_region=us-east1 gcp_compute_zone=us-east1-b gcp_backup_taken=True'
The upgrade can take some time to complete, but can take longer depending on the number of extension nodes on the system. A successful upgrade is marked by the log below.
TASK [redhat.ansible_on_clouds.standalone_gcp_upgrade : [upgrade] Show GCP current version] *** ok: [localhost] => { "msg": "gcp_current_version: 2.3.20230221-00" }
- Your Ansible Automation Platform from GCP Marketplace deployment is now upgraded to a newer version and you can log in to Red Hat Ansible Automation Platform automation controller and automation hub using your deployment credentials.
10.3. Restoring the stack from backups
If, for some reason, the upgrade process fails, you can use the backup files from Backing up your Ansible Automation Platform stack to restore your stack.
The following procedures are optional.
If the upgrade process fails, you can use the backup files from Backing up before the upgrade to restore your stack. The restore process installs a new deployment, and restores the filestore and SQL database instance to the specified backup.
The restored filestore instance uses CIDR network range 192.168.244.0/29, which is different from the default network filestore network range of the backup deployment.
The following instructions describe how to run the restore playbook.
10.3.1. Pulling the ansible-on-clouds-ops 2.2 container image to restore the stack
Procedure
Pull the
ansible-on-clouds-ops 2.2
container image with the same tag version as the foundation deployment.NoteBefore pulling the docker image, make sure you are logged in to registry.redhat.com using docker. Use the following command to login to registry.redhat.com.
$ docker login registry.redhat.io
For more information about registry login, see Registry Authentication
$ export IMAGE=registry.redhat.io/ansible-on-clouds/ansible-on-clouds-ops-rhel8:2.2.20230215 $ docker pull $IMAGE --platform=linux/amd64
For EMEA regions (Europe, Middle East, Africa) run the following command instead:
$ export IMAGE=registry.redhat.io/ansible-on-clouds/ansible-on-clouds-ops-emea-rhel8:2.2.20230215 $ docker pull $IMAGE --platform=linux/amd64
10.3.2. Preparing the backup environment
Procedure
Create the directories used to configure the upgrade process using the following commands:
# Secrets directory. Store the GCP credentials file. $ mkdir secrets # Extra vars directory. All the configuration variables for the upgrade process. $ mkdir extra_vars
10.3.3. Running the restore playbook as a container
Procedure
-
Replace
<absolute path to Google application credentials dir>
with the path to the service account credential JSON file for the Google cloud. -
Replace
<absolute path of extra vars dir>
with the path to theextra_vars.yml
file you created in Preparing the backup environment. Use the following command to run the playbook.
$ docker run --rm \ -v <absolute path to Google application credentials dir>:/secrets:ro \ -v <absolute path of extra vars dir>:/extra_vars:ro \ $IMAGE \ redhat.ansible_on_clouds.gcp_restore_deployment \ -e @/extra_vars/extra_vars.yaml
When you have run the playbook, a new restored deployment is visible in the GCP deployment. It can take a few minutes for the deployment to finish and for all the containers to run.
NoteAccess to the restored deployment must be configured through either an external load balancer or VPN. When a connection method is configured you can log in to Red Hat Ansible Automation Platform automation controller and automation hub using your old deployment credentials. In addition, all job history, uploaded collections and other records must be in the same state as the restored deployment.