Ansible Automation Platform migration

Procedure

1. Document the full topology of your current RPM deployment:

   1. Map out all servers, nodes, and their roles (for example control nodes, execution nodes, database servers).
   2. Note the hostname, IP address, and function of each server in your deployment.
   3. Document the network configuration between components.

2. Ansible Automation Platform version information:

   1. Record the exact Ansible Automation Platform version (X.Y) currently deployed.

3. Document the specific version of each component:

   1. Automation controller version
   2. Automation hub version
   3. Platform gateway version

4. Database configuration:

   1. Database names for each component
   2. Database users and roles
   3. Connection parameters and authentication methods
   4. Any custom PostgreSQL configurations or optimizations

Procedure

1. Verify the PostgreSQL database version is PostgreSQL version 15.

   You can verify your current PostgreSQL version by connecting to your database server and running the following command as the postgres user:

   $ psql -c 'SELECT version();'

   Important

   PostgreSQL version 15 is a strict requirement for the migration process to succeed. If running PostgreSQL 13 or earlier, upgrade to version 15 before proceeding with the migration.

   If using an Ansible Automation Platform managed database, re-run the installation program to upgrade the PostgreSQL version. If using a customer provided (external) database, contact your database administrator or service provider to confirm the version and arrange for an upgrade if required.

2. Create a complete backup of the source environment:

   $ ./setup.sh -e 'backup_dest=/path/to/backup_dir/' -b

3. Get the connection settings from one node from each of the component groups.

   For each command, access the host and become the root user.

   • Access the automation controller node and run:

     # awx-manage print_settings | grep '^DATABASES'

   • Access the automation hub node and run:

     # grep '^DATABASES' /etc/pulp/settings.py

   • Access the platform gateway node and run:

     # aap-gateway-manage print_settings | grep '^DATABASES'

4. Stage the manually created artifact on the platform gateway node.

   # mkdir -p /tmp/backups/artifact/{controller,gateway,hub}

   # mkdir -p /tmp/backups/artifact/controller/custom_configs

   # touch /tmp/backups/artifact/secrets.yml

   # cd /tmp/backups/artifact/

5. Validate the database size and make sure you have enough space on the filesystem for the pg_dump.

   You can verify the database sizes by connecting to your database server and running the following command as the postgres user:

   $ psql -c '\l+'

   Adjust the filesystem size or mount an external filesystem as needed before performing the next step.

   Note

   These commands send all target files to the /tmp filesystem. Adjust the commands to match your environment’s needs.

6. Perform database dumps of all components on the platform gateway node within the artifact you created.

   # psql -h <pg_hostname> -U <component_pg_user> -d <database_name> -t -c 'SHOW server_version;' # ensure connectivity to the database

   # pg_dump -h <pg_hostname> -U <component_pg_user> -d <component_pg_name> --clean --create -Fc -f <component>/<component>.pgc

   # ls -ld <component>/<component>.pgc

   # echo "<component>_pg_database: <database_name>" >> secrets.yml ## Add the database name for the component to the secrets file

7. Export secrets from the RPM environment from one node of each component group.

   For each of the following steps, use the root user to run the commands.

   • Access the automation controller node, gather the secret key, and add it to the controller_secret_key value in the secrets.yml file.

     # cat /etc/tower/SECRET_KEY

   • Access the automation hub node, gather the secret key, and add it to the hub_secret_key value in the secrets.yml file.

     # grep '^SECRET_KEY' /etc/pulp/settings.py | awk -F'=' '{ print $2 }'

   • Access the automation hub node, gather the database_fields.symmetric.key value, and add it to the hub_db_fields_encryption_key value in the secrets.yml file.

     # cat /etc/pulp/certs/database_fields.symmetric.key

   • Access the platform gateway node, gather the secret key, and add it to the gateway_secret_key value in the secrets.yml file.

     # cat /etc/ansible-automation-platform/gateway/SECRET_KEY

8. Export automation controller custom configurations.

   If any custom settings exist on the /etc/tower/conf.d, copy them to /tmp/backups/artifact/controller/custom_configs.

   Configuration files on automation controller that are managed by the installation program and not considered custom:

   • /etc/tower/conf.d/postgres.py
   • /etc/tower/conf.d/channels.py
   • /etc/tower/conf.d/caching.py
   • /etc/tower/conf.d/cluster_host_id.py

9. Package the artifact.

   # cd /tmp/backups/artifact/

   # [ -f sha256sum.txt ] && rm -f sha256sum.txt; find . -type f -name "*.pgc" -exec sha256sum {} \; >> sha256sum.txt

   # cat sha256sum.txt

   # cd ..

   # tar cf artifact.tar artifact

   # sha256sum artifact.tar > artifact.tar.sha256

   # sha256sum --check artifact.tar.sha256

   # tar tvf artifact.tar

   Example output of tar tvf artifact.tar:

   drwxr-xr-x ansible/ansible     0 2025-05-08 16:48 artifact/
   drwxr-xr-x ansible/ansible     0 2025-05-08 16:33 artifact/controller/
   -rw-r--r-- ansible/ansible 732615 2025-05-08 16:26 artifact/controller/controller.pgc
   drwxr-xr-x ansible/ansible      0 2025-05-08 16:33 artifact/controller/custom_configs/
   drwxr-xr-x ansible/ansible      0 2025-05-08 16:11 artifact/gateway/
   -rw-r--r-- ansible/ansible 231155 2025-05-08 16:28 artifact/gateway/gateway.pgc
   drwxr-xr-x ansible/ansible      0 2025-05-08 16:26 artifact/hub/
   -rw-r--r-- ansible/ansible 29252002 2025-05-08 16:26 artifact/hub/hub.pgc
   -rw-r--r-- ansible/ansible      614 2025-05-08 16:24 artifact/secrets.yml
   -rw-r--r-- ansible/ansible      338 2025-05-08 16:48 artifact/sha256sum.txt

10. Download the artifact.tar and artifact.tar.sha256 to your local machine or transfer to the target node with the scp command.

Procedure

1. Document the full topology of your current containerized deployment:

   1. Map out all servers, nodes, and their roles (for example control nodes, execution nodes, database servers).
   2. Note the hostname, IP address, and function of each server in your deployment.
   3. Document the network configuration between components.

2. Ansible Automation Platform version information:

   1. Record the exact Ansible Automation Platform version (X.Y) currently deployed.

3. Document the specific version of each component:

   1. Automation controller version
   2. Automation hub version
   3. Platform gateway version

4. Database configuration:

   1. Database names for each component
   2. Database users and roles
   3. Connection parameters and authentication methods
   4. Any custom PostgreSQL configurations or optimizations
5. Identify all custom configurations and settings
6. Document container resource allocations and volumes

Procedure

1. Verify the PostgreSQL database version is PostgreSQL version 15.

   You can verify your current PostgreSQL version by connecting to your database server and running the following command as the postgres user:

   $ podman exec -it postgresql bash -c 'psql -c "SELECT version();"'

   Important

   PostgreSQL version 15 is a strict requirement for the migration process to succeed. If running PostgreSQL 13 or earlier, upgrade to version 15 before proceeding with the migration.

   If using an Ansible Automation Platform managed database, re-run the installation program to upgrade the PostgreSQL version. If using a customer provided (external) database, contact your database administrator or service provider to confirm the version and arrange for an upgrade if required.

2. Create a complete backup of the source environment:

   $ ansible-playbook -i <path_to_inventory> ansible.containerized_installer.backup

3. Get the connection settings from one node in each of the component groups.

   • Access the automation controller node and run:

     $ podman exec -it automation-controller-task bash -c 'awx-manage print_settings | grep '^DATABASES'

   • Access the automation hub node and run:

     $ podman exec -it automation-hub-api bash -c "pulpcore-manager diffsettings | grep '^DATABASES'"

   • Access the platform gateway node and run:

     $ podman exec -it automation-gateway bash -c "aap-gateway-manage print_settings | grep '^DATABASES'"

4. Validate the database size and make sure you have enough space on the filesystem for the pg_dump.

   You can verify the database sizes by connecting to your database server and running the following command as the postgres user:

   $ podman exec -it postgresql bash -c 'psql -c "\l+"'

   Adjust the filesystem size or mount an external filesystem as needed before performing the next step.

   Note

   These commands send all target files to the /tmp filesystem. Adjust the commands to match your environment’s needs.

5. Stage the manually created artifact on the platform gateway node.

   # mkdir -p /tmp/backups/artifact/{controller,gateway,hub}

   # mkdir -p /tmp/backups/artifact/controller/custom_configs

   # touch /tmp/backups/artifact/secrets.yml

   # cd /tmp/backups/artifact/

6. Perform database dumps of all components on the platform gateway node within the artifact created previously.

   To run the psql and pg_restore commands, you must create a temporary container and run the commands inside of it. This command must be run from the database node.

   $ podman run -it --rm --name postgresql_restore_temp --network host --volume ~/aap/tls/extracted:/etc/pki/ca-trust/extracted:z --volume ~/aap/postgresql/server.crt:/var/lib/pgsql/server.crt:ro,z --volume ~/aap/postgresql/server.key:/var/lib/pgsql/server.key:ro,z --volume /tmp/backups/artifact:/var/lib/pgsql/backups:ro,z registry.redhat.io/rhel8/postgresql-15:latest bash

   Note

   This command assumes the image registry.redhat.io/rhel8/postgresql-15:latest. If you are missing the image, check the available images for the user with podman images ls.

   The command above opens a shell inside the container named postgresql_restore_temp and has the artifact mounted into /var/lib/pgsql/backups. Also, this command is mounting the PostgreSQL certificates to ensure that you can resolve the correct certificates.

   bash-4.4$ cd /var/lib/pgsql/backups
   bash-4.4$ psql -h <pg_hostname> -U <component_pg_user> -d <database_name> -t -c 'SHOW server_version;' # ensure connectivity to db
   bash-4.4$ pg_dump -h <pg_hostname> -U <component_pg_user> -d <component_pg_name> --clean --create -Fc -f <component>/<component>.pgc
   bash-4.4$ ls -ld <component>/<component>.pgc
   bash-4.4$ echo "<component>_pg_database: <database_name>" >> secrets.yml ## Add the DB name for the component to the secrets file

   After collecting this data, exit from this temporary container.

7. Export the secrets from the containerized environment from one node of each component group.

   For each step below, use the root user to run the commands.

   1. Access the automation controller node and gather the secret key and add to the controller_secret_key value in secrets.yaml file.

      $ podman secret inspect --showsecret --format "{{.SecretData}}" controller_secret_key

   2. Access the automation hub node and gather the secret key and add to the hub_secret_key value in secrets.yaml file.

      $ podman secret inspect --showsecret --format "{{.SecretData}}" hub_secret_key

   3. Access the automation hub node and gather the database_fields.symmetric.key value and add to the hub_db_fields_encryption_key value in secrets.yaml file.

      $ podman secret inspect --showsecret --format "{{.SecretData}}" hub_database_fields

   4. Access the platform gateway node and gather the secret key and add to the gateway_secret_key value in secrets.yaml file.

      $ podman secret inspect --showsecret --format "{{.SecretData}}" gateway_secret_key

8. Export automation controller custom configurations.

   If any extra_settings exist in your containerized installation inventory, copy them into a new file and saving them under /tmp/backups/artifact/controller/custom_configs.

9. Package the artifact.

   # cd /tmp/backups/artifact/
   # [ -f sha256sum.txt ] && rm -f sha256sum.txt; find . -type f -name "*.pgc" -exec sha256sum {} \; >> sha256sum.txt
   # cat sha256sum.txt
   # cd ..
   # tar cf artifact.tar artifact
   # sha256sum artifact.tar > artifact.tar.sha256
   # sha256sum --check artifact.tar.sha256
   # tar tvf artifact.tar

   Example output of tar tvf artifact.tar:

   drwxr-xr-x ansible/ansible     0 2025-05-08 16:48 artifact/
   drwxr-xr-x ansible/ansible     0 2025-05-08 16:33 artifact/controller/
   -rw-r--r-- ansible/ansible 732615 2025-05-08 16:26 artifact/controller/controller.pgc
   drwxr-xr-x ansible/ansible      0 2025-05-08 16:33 artifact/controller/custom_configs/
   drwxr-xr-x ansible/ansible      0 2025-05-08 16:11 artifact/gateway/
   -rw-r--r-- ansible/ansible 231155 2025-05-08 16:28 artifact/gateway/gateway.pgc
   drwxr-xr-x ansible/ansible      0 2025-05-08 16:26 artifact/hub/
   -rw-r--r-- ansible/ansible 29252002 2025-05-08 16:26 artifact/hub/hub.pgc
   -rw-r--r-- ansible/ansible      614 2025-05-08 16:24 artifact/secrets.yml
   -rw-r--r-- ansible/ansible      338 2025-05-08 16:48 artifact/sha256sum.txt

10. Download the artifact.tar and artifact.tar.sha256 to your local machine or transfer to the target node with the scp command.

Procedure

1. Validate the file system home folder size and make sure it has enough space to transfer the artifact.
2. Transfer the artifact to the nodes where you will be working by using scp or any preferred file transfer method. It is recommended that you work from the platform gateway node as it has access to most systems. However, if you have access or file system space limitations due to the PostgreSQL dumps, work from the database node instead.
3. Download the latest version of containerized Ansible Automation Platform from the Ansible Automation Platform download page.
4. Validate the artifact checksum.

5. Extract the artifact on the home folder for the user running the containers.

   $ cd ~

   $ sha256sum --check artifact.tar.sha256

   $ tar xf artifact.tar

   $ cd artifact

   $ sha256sum --check sha256sum.txt

6. Generate an inventory file for your containerized deployment.

   Configure the inventory file to match the same topology as the source environment. Configure the component database names and the secret_key values from the artifact’s secrets.yml file.

   You can do this in two ways:

   • Set the extra variables in the inventory file.

   • Use the secrets.yml file as an additional variables file when running the installation program.

     1. Option 1: Extra variables in the inventory file

        $ egrep 'pg_database|_key' inventory
        controller_pg_database=<redacted>
        controller_secret_key=<redacted>
        gateway_pg_database=<redacted>
        gateway_secret_key=<redacted>
        hub_pg_database=<redacted>
        hub_secret_key=<redacted>
        __hub_database_fields=<redacted>

        Note

        The __hub_database_fields value comes from the hub_db_fields_encryption_key value in your secret.

     2. Option 2: Additional variables file

        $ ansible-playbook -i inventory ansible.containerized_installer.install -e @~/artifact/secrets.yml -e "__hub_database_fields='{{ hub_db_fields_encryption_key }}'"

7. Install and configure the containerized target environment.
8. Verify PostgreSQL database version is on version 15.

9. Create a backup of the initial containerized environment.

   $ ansible-playbook -i <path_to_inventory> ansible.containerized_installer.backup

10. Verify the fresh installation functions correctly.

Procedure

1. Stop the containerized services, except the database.

   1. In all nodes, if Performance Co-Pilot is configured, run the following command:

      $ systemctl --user stop pcp

   2. Access the automation controller node and run:

      $ systemctl --user stop automation-controller-task automation-controller-web automation-controller-rsyslog
      $ systemctl --user stop receptor

   3. Access the automation hub node and run:

      $ systemctl --user stop automation-hub-api automation-hub-content automation-hub-web automation-hub-worker-1 automation-hub-worker-2

   4. Access the Event-Driven Ansible node and run:

      $ systemctl --user stop automation-eda-scheduler automation-eda-daphne automation-eda-web automation-eda-api automation-eda-worker-1 automation-eda-worker-2 automation-eda-activation-worker-1 automation-eda-activation-worker-2

   5. Access the platform gateway node and run:

      $ systemctl --user stop automation-gateway automation-gateway-proxy

   6. Access the platform gateway node when using standalone Redis, or all nodes from the Redis group in your inventory file when using clustered Redis, and run:

      $ systemctl --user stop redis-unix redis-tcp

      Note

      In an enterprise deployment, the components run on different nodes. Run the commands on each component node.

2. Import database dumps to the containerized environment.

   1. If you are using an Ansible Automation Platform managed database, you must create a temporary container to run the psql and pg_restore commands. Run this command from the database node:

      $ podman run -it --rm --name postgresql_restore_temp --network host --volume ~/aap/tls/extracted:/etc/pki/ca-trust/extracted:z --volume ~/aap/postgresql/server.crt:/var/lib/pgsql/server.crt:ro,z --volume ~/aap/postgresql/server.key:/var/lib/pgsql/server.key:ro,z --volume ~/artifact:/var/lib/pgsql/backups:ro,z registry.redhat.io/rhel8/postgresql-15:latest bash

      Note

      The command above opens a shell inside the container named postgresql_restore_temp with the artifact mounted at /var/lib/pgsql/backups. Additionally, it mounts the PostgreSQL certificates to ensure that you can resolve the correct certificates.

      The command assumes the image registry.redhat.io/rhel8/postgresql-15:latest is available. If you are missing the image, check the available images for the user with podman images ls.

      It also assumes that the artifact is located in the current user’s home folder. If the artifact is located elsewhere, change the ~/artifact with the required path.

   2. If you are using a customer-provided (external) database, you can run the psql and pg_restore commands from any node that has these commands installed and that has access to the database. Reach out to your database administrator if you are unsure.

   3. From inside the container, access the database and ensure the users have the CREATEDB role.

      bash-4.4$ psql -h <pg_hostname> -U postgres
      postgres=# \l
                Name           |     Owner     | Encoding |   Collate   |    Ctype    | ICU Locale | Locale Provider |   Access privileg
      es
      -------------------------+---------------+----------+-------------+-------------+------------+-----------------+------------------
      -----
       automationedacontroller | eda           | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
       automationhub           | automationhub | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
       awx                     | awx           | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
       gateway                 | gateway       | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
      ...

   4. For each component name, add the CREATEDB role to the Owner. For example:

      postgres=# ALTER ROLE awx WITH CREATEDB;
      postgres=# \q

      Replace awx with the database owner.

   5. With the CREATEDB in place, access the path where the artifact is mounted, and run the pg_restore commands.

      bash$ cd /var/lib/pgsql/backups
      bash$ pg_restore --clean --create --no-owner -h <pg_hostname> -U <component_pg_user> -d template1 <component>/<component>.pgc

   6. After the restore, remove the permissions from the user. For example:

      postgres=# ALTER ROLE awx WITH NOCREATEDB;
      postgres=# \q

      Replace awx with each user containing the role.

3. Start the containerized services, except the database.

   Note

   In an enterprise deployment, the components run on different nodes. Run the commands on each component node.

   1. In all nodes, if Performance Co-Pilot is configured, run the following command:

      $ systemctl --user start pcp

   2. Access the automation controller node and run:

      $ systemctl --user start automation-controller-task automation-controller-web automation-controller-rsyslog
      $ systemctl --user start receptor

   3. Access the automation hub node and run:

      $ systemctl --user start automation-hub-api automation-hub-content automation-hub-web automation-hub-worker-1 automation-hub-worker-2

   4. Access the Event-Driven Ansible node and run:

      $ systemctl --user start automation-eda-scheduler automation-eda-daphne automation-eda-web automation-eda-api automation-eda-worker-1 automation-eda-worker-2  automation-eda-activation-worker-1 automation-eda-activation-worker-2

   5. Access the platform gateway node and run:

      $ systemctl --user start automation-gateway automation-gateway-proxy

   6. Access the platform gateway node when using standalone Redis, or all nodes from the Redis group in your inventory when using clustered Redis, and run:

      $ systemctl --user start redis-unix redis-tcp

Procedure

1. Deprovision the platform gateway configuration.

   • To deprovision platform gateway configuration, SSH to the host serving an automation-gateway container as the same rootless user from 4.2.6 and run the following to remove the platform gateway proxy configuration:

     $ podman exec -it automation-gateway bash
     $ aap-gateway-manage migrate
     $ aap-gateway-manage shell_plus
     >>> HTTPPort.objects.all().delete(); ServiceNode.objects.all().delete(); ServiceCluster.objects.all().delete()

2. Transfer custom configurations and settings.

   • Edit the inventory file and apply any relevant extra_settings to each component by using the component_extra_settings.

3. Remove all resource server key secrets to be repopulated by the installation program:

   $ for i in `podman secret ls | egrep 'resource_server' | awk '{print $2}'`; do podman secret rm $i; done

4. Re-run the installation program on the target environment by using the same inventory from the installation.

5. Sync platform gateway resources if Event-Driven Ansible is present:

   $ podman exec -it automation-eda-api bash
   $ aap-eda-manage resource_sync

6. Validate instances for automation execution.

   1. SSH to the host serving an automation-controller-task container as the rootless user, and run the following commands to validate and remove instances that are orphaned from the source artifact:

      $ podman exec -it automation-controller-task bash

      $ awx-manage list_instances

   2. Find nodes that are no longer part of this cluster. A good indicator is nodes with 0 capacity as they have failed their health checks:

      [ungrouped capacity=0]
      	[DISABLED] node1.example.org capacity=0 node_type=hybrid version=X.Y.Z heartbeat="..."
      	[DISABLED] node2.example.org capacity=0 node_type=execution version=ansible-runner-X.Y.Z heartbeat="..."

   3. Remove those nodes with awx-manage, leaving only the aap-controller-task instance:

      awx-manage deprovision_instance --host=node1.example.org
      awx-manage deprovision_instance --host=node2.example.org

7. Repair orphaned automation hub content links for Pulp.

   • Run the following command from any host that has direct access to the automation hub address:

     $ curl -d '{\"verify_checksums\": true }' -X POST -k https://<gateway url>/api/galaxy/pulp/api/v3/repair/ -u <gateway_admin_user>:<gateway_admin_password>

8. Reconcile instance groups configuration:

   1. Go to Automation Execution → Infrastructure → Instance Groups.
   2. Select the Instance Group and then select the Instances tab.
   3. Associate or disassociate instances as required.

9. Reconcile decision environments and credentials:

   1. Go to Automation Decisions → Decision Environments.
   2. Edit each decision environment which references a registry URL either unrelated or no longer accessible to this new environment. For example, the automation hub decision environment might require modification for the target automation hub environment.
   3. Select each associated credential to these decision environments and ensure their addresses align with the new environment.

10. Reconcile execution environments and credentials:

    1. Go to Automation Execution → Infrastructure → Execution Environments.
    2. Check each execution environment image and verify their addresses against the new environment.
    3. Go to Automation Execution → Infrastructure → Credentials.
    4. Edit each credential and ensure that all environment specific information aligns with the new environment.
11. Verify any further customizations or configurations after the migration, such as RBAC rules with instance groups.

Procedure

1. Verify all migrated components function correctly.

   1. Platform gateway: Access the Ansible Automation Platform URL at https://<gateway_hostname>/ and verify that the dashboard loads correctly. Check that the platform gateway service is running and connected to automation controller.
   2. Automation controller: Under Automation Execution, check that projects, inventories, and job templates are present and configured.
   3. Automation hub: Under Automation Content, verify that collections, namespaces, and their contents are visible.
   4. Event-Driven Ansible (if applicable): Under Automation Execution Decisions, verify that rule audits, rulebook activations, and projects are accessible.

   5. For each component, check the logs to ensure there are no startup errors or warnings:

      podman logs <container_name>

2. Test workflows and automation processes.

   1. Run job templates: Run several key job templates, including those with dependencies on various credential types.
   2. Test workflow templates: Run workflow templates to ensure that workflow nodes run in the correct order and that the workflow completes successfully.
   3. Verify execution environments: Ensure that jobs run in the appropriate execution environments and can access required dependencies.
   4. Check job artifacts: Verify that job artifacts are properly stored and accessible.
   5. Validate job scheduling: Test scheduled jobs to ensure they run at the expected times.

3. Validate user access and permissions.

   1. User authentication: Test login functionality with various user accounts to ensure authentication works correctly.
   2. Role-based access controls: Verify that users have appropriate permissions for organizations, projects, inventories, and job templates.
   3. Team memberships: Confirm that team memberships and team-based permissions are intact.
   4. API access: Test API tokens and ensure that API access is functioning properly.
   5. SSO integration (if applicable): Verify that Single Sign-On authentication is working correctly.

4. Confirm content synchronization and availability.

   1. Collection synchronization: Check that you can synchronize collections from a remote.
   2. Collection Upload: Check that you can upload collections.
   3. Collection repositories: Verify that automation hub makes collections available and that execution environments can use them.
   4. Project synchronization: Check that projects can sync content from source control repositories.
   5. External content sources: Test synchronization from automation hub and Ansible Galaxy (if configured).
   6. Execution environment availability: Confirm that all required execution environments exist and that execution nodes can access them.
   7. Content dependencies: Verify that the system correctly resolves content dependencies when running jobs.

Procedure

1. Configure Ansible Automation Platform Operator for an Ansible Automation Platform deployment.
2. Set up the database configuration (internal or external).
3. Set up the Redis configuration (internal or external).
4. Install Ansible Automation Platform using Ansible Automation Platform Operator.
5. Create a backup of the initial OpenShift Container Platform deployment.
6. Verify the fresh installation functions correctly.

Procedure

1. Scale down Ansible Automation Platform components.

   1. Begin by scaling down the Ansible Automation Platform deployment by using idle_aap:

      oc patch ansibleautomationplatform aap --type merge -p '{"spec":{"idle_aap":true}}'

   2. Wait for component pods to stop. Only the 6 Operator pods will remain running.

      NAME                                                                  READY   STATUS      RESTARTS   AGE
      pod/aap-controller-migration-4.6.13-5swc6                             0/1     Completed   0          160m
      pod/aap-gateway-operator-controller-manager-6b75c95458-4zrxv          2/2     Running     0          26h
      pod/ansible-lightspeed-operator-controller-manager-b674c55b8-qncjp    2/2     Running     0          45h
      pod/automation-controller-operator-controller-manager-6b79d48d4cchn   2/2     Running     0          45h
      pod/automation-hub-operator-controller-manager-5cd674c984-5njfj       2/2     Running     0          45h
      pod/eda-server-operator-controller-manager-645f4db5-d2flt             2/2     Running     0          45h
      pod/resource-operator-controller-manager-86b8f7bb54-cvz6d             2/2     Running     0          45h

   3. Scale down the Ansible Automation Platform Gateway Operator and Ansible Automation Platform Controller Operator:

      oc scale --replicas=0 deployment aap-gateway-operator-controller-manager automation-controller-operator-controller-manager

      Example output:

      deployment.apps/aap-gateway-operator-controller-manager scaled
      deployment.apps/automation-controller-operator-controller-manager scaled

2. Scale up the idled Postgres StatefulSet.

   oc scale --replicas=1 statefulset.apps/aap-postgres-15

3. Prepare a temporary environment for the database restore.

   1. Create a temporary Persistent Volume Claim (PVC) with appropriate settings and sizing.

      aap-temp-pvc.yaml

      ---
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        name: aap-temp-pvc
        namespace: aap
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 200Gi

      oc create -f aap-temp-pvc.yaml

   2. Obtain the existing PostgreSQL image to use for temporary deployment:

      echo $(oc get pod/aap-postgres-15-0 -o jsonpath="{.spec.containers[].image}")

   3. Create a temporary PostgreSQL deployment with the mounted temporary PVC:

      aap-temp-postgres.yaml

      ---
      kind: Deployment
      apiVersion: apps/v1
      metadata:
        name: aap-temp-postgres
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: aap-temp-postgres
        template:
          metadata:
            labels:
              app: aap-temp-postgres
          spec:
            containers:
              - name: aap-temp-postgres
                image: <postgres image from previous step>
                command:
                  - /bin/sh
                  - '-c'
                  - sleep infinity
                imagePullPolicy: Always
                securityContext:
                  runAsNonRoot: true
                  allowPrivilegeEscalation: false
                volumeMounts:
                  - name: aap-temp-pvc
                    mountPath: /tmp/aap-temp-pvc
            volumes:
              - name: aap-temp-pvc
                persistentVolumeClaim:
                  claimName: aap-temp-pvc

      oc create -f aap-temp-postgres.yaml

4. Copy the export artifact to the temporary PostgreSQL pod.

   1. First, obtain the pod name and set it as an environment variable:

      export AAP_TEMP_POSTGRES=$(oc get pods --no-headers -o custom-columns="metadata.name" | grep aap-temp-postgres)

   2. Test the environment variable:

      echo $AAP_TEMP_POSTGRES

      Example output:

      aap-temp-postgres-7b6c57f87f-s2ldp

   3. Copy the artifact and checksum to the PVC:

      oc cp artifact.tar $AAP_TEMP_POSTGRES:/tmp/aap-temp-pvc/
      oc cp artifact.tar.sha256 $AAP_TEMP_POSTGRES:/tmp/aap-temp-pvc/

5. Restore databases to Ansible Automation Platform PostgreSQL by using the temporary PostgreSQL pod.

   1. First, obtain the PostgreSQL passwords for all three databases and the PostgreSQL admin password:

      echo "" && for secret in aap-controller-postgres-configuration aap-hub-postgres-configuration aap-gateway-postgres-configuration
      do
      echo $secret
      echo "PASSWORD: `oc get secrets $secret -o jsonpath="{.data['password']}" | base64 -d`"
      echo "USER: `oc get secrets $secret -o jsonpath="{.data['username']}" | base64 -d`"
      echo "DATABASE: `oc get secrets $secret -o jsonpath="{.data['database']}" | base64 -d`"
      echo ""
      done && echo "POSTGRES ADMIN PASSWORD: `oc get secrets aap-gateway-postgres-configuration -o jsonpath="{.data['postgres_admin_password']}" | base64 -d`"

   2. Enter into the temporary PostgreSQL deployment and change directory to the mounted PVC containing the copied artifact:

      oc exec -it deployment.apps/aap-temp-postgres -- /bin/bash

   3. Inside the pod, change directory to /tmp/aap-temp-pvc and list its contents:

      cd /tmp/aap-temp-pvc && ls -l

      Example output:

      total 2240
      -rw-r--r--. 1 1000900000 1000900000 2273280 Jun 13 17:41 artifact.tar
      -rw-r--r--. 1 1000900000 1000900000      79 Jun 13 17:42 artifact.tar.sha256
      drwxrws---. 2 root       1000900000   16384 Jun 13 17:40 lost+found

   4. Verify the archive:

      sha256sum --check artifact.tar.sha256

      Example output:

      artifact.tar: OK

   5. Extract the artifact and verify its contents:

      tar xf artifact.tar && cd artifact && sha256sum --check sha256sum.txt

      Example output:

       ./controller/controller.pgc: OK
       ./gateway/gateway.pgc: OK
       ./hub/hub.pgc: OK

   6. Drop the automation controller database:

      dropdb -h aap-postgres-15 automationcontroller

   7. Alter the user temporarily with the CREATEDB role:

      postgres=# ALTER USER automationcontroller WITH CREATEDB;

   8. Create the database:

      createdb -h aap-postgres-15 -U automationcontroller automationcontroller

   9. Revert temporary user permission:

      postgres=# ALTER USER automationcontroller NOCREATEDB;

   10. Restore the automation controller database:

       pg_restore --clean --create --no-owner -h aap-postgres-15 -U automationcontroller -d automationcontroller controller/controller.pgc

   11. Restore the automation hub database:

       pg_restore --clean --create --no-owner -h aap-postgres-15 -U automationhub -d automationhub hub/hub.pgc

   12. Restore the platform gateway database:

       pg_restore --clean --create --no-owner -h aap-postgres-15 -U gateway -d gateway gateway/gateway.pgc

   13. Exit the pod:

       exit

6. Replace database field encryption secrets and clean up temporary resources.

   1. Replace database field encryption secrets:

      oc set data secret/aap-controller-secret-key secret_key="<unencoded controller_secret_key value from secrets.yml>"

      oc set data secret/aap-db-fields-encryption-secret secret_key="<unencoded gateway_secret_key value from secrets.yml>"

      oc set data secret/aap-hub-db-fields-encryption database_fields.symmetric.key="<unencoded hub_db_fields_encryption_key value from secrets.yml>"

   2. Clean up the temporary PostgreSQL and PVC:

      oc delete -f aap-temp-postgres.yaml

      oc delete -f aap-temp-pvc.yaml

7. Scale Ansible Automation Platform components back up.

   1. Scale the platform gateway and automation controller Operators back up and wait for the platform gateway Operator reconciliation loop to complete:

      The PostgreSQL StatefulSet returns to idle.

      oc scale --replicas=1 deployment aap-gateway-operator-controller-manager automation-controller-operator-controller-manager

      Example output:

      deployment.apps/aap-gateway-operator-controller-manager scaled
      deployment.apps/automation-controller-operator-controller-manager scaled

      oc logs -f $(oc get pods  --no-headers -o custom-columns=":metadata.name" | grep aap-gateway-operator)

   2. Wait for reconciliation to stop.

      Example output:

      META: ending play
      {"level":"info","ts":"2025-06-12T15:41:29Z","logger":"runner","msg":"Ansible-runner exited successfully","job":"5672263053238024330","name":"aap","namespace":"aap"}
      
      ----- Ansible Task Status Event StdOut (aap.ansible.com/v1alpha1, Kind=AnsibleAutomationPlatform, aap/aap) -----
      
      
      PLAY RECAP *********************************************************************
      localhost                  : ok=45   changed=0    unreachable=0    failed=0    skipped=63   rescued=0    ignored=0

   3. Scale Ansible Automation Platform back up using idle_aap:

      oc patch ansibleautomationplatform aap --type=merge -p '{"spec":{"idle_aap":false}}'

      Example output:

      ansibleautomationplatform.aap.ansible.com/aap patched

8. Wait for the aap-gateway pod to be running and clean up old service endpoints.

   Example output:

   pod/aap-gateway-6c989b846c-47b91 2/2 Running 0 45s

   for i in HTTPPort Route ServiceNode; do; oc exec -it deployment.apps/aap-gateway -- aap-gateway-manage shell -c 'from aap_gateway_api.models import '$i';print('$i'.objects.all().delete())'; done

   Example output:

   (23, {'aap_gateway_api.ServiceAPIRoute': 4, 'aap_gateway_api.AdditionalRoute': 7, 'aap_gateway_api.Route': 11, 'aap_gateway_api.HTTPPort': 1})
   (0, {})
   (4, {'aap_gateway_api.ServiceNode': 4})

9. Run awx-manage to deprovision instances.

   1. Obtain the automation controller pod:

      export AAP_CONTROLLER_POD=$(oc get pods --no-headers -o custom-columns=":metadata.name" | grep aap-controller-task)

   2. Test the environment variable:

      echo $AAP_CONTROLLER_POD

      Example output:

      aap-controller-task-759b6d9759-r59q9

   3. Enter into the automation controller pod:

      oc exec -it $AAP_CONTROLLER_POD -- /bin/bash
      awx-manage list_instances

      Example output:

      bash-4.4$
      [controlplane capacity=642 policy=100%]
      	aap-controller-task-759b6d9759-r59q9 capacity=642 node_type=control version=4.6.15 heartbeat="2025-06-12 21:39:48"
      	node1.example.org capacity=0 node_type=hybrid version=4.6.13 heartbeat="2025-05-30 17:22:11"
      
      [default capacity=0 policy=100%]
      	node1.example.org capacity=0 node_type=hybrid version=4.6.13 heartbeat="2025-05-30 17:22:11"
      	node2.example.org capacity=0 node_type=execution version=ansible-runner-2.4.1 heartbeat="2025-05-30 17:22:08"

   4. Remove old nodes with awx-manage, leaving only aap-controller-task:

      awx-manage deprovision_instance --host=node1.example.org
      awx-manage deprovision_instance --host=node2.example.org

10. Remove the aap-resource-server secret and allow the deployments to reconcile. This will recreate the resource service keys and secret for the components:

    $ oc delete secret/aap-resource-server

11. Run the curl command to repair automation hub filesystem data.

    curl -d '{\"verify_checksums\": true }' -X POST -k https://<aap url>/api/galaxy/pulp/api/v3/repair/ -u <admin_user>:<restored_admin_password>

Procedure

1. Modify the Django SECRET_KEY secrets to match the source platform.
2. Deprovision and reconfigure platform gateway service nodes.
3. Re-run platform gateway nodes and services register logic.
4. Convert container-specific settings to OpenShift Container Platform-appropriate formats.
5. Reconcile container resource allocations to OpenShift Container Platform resources.

Procedure

1. Verify all migrated components are functional.
2. Test workflows and automation processes.
3. Validate user access and permissions.
4. Confirm content synchronization and availability.
5. Test integration with OpenShift Container Platform-specific features.

Procedure

1. Submit a support ticket on the Red Hat Customer Portal requesting a migration to Managed Ansible Automation Platform.

   The support ticket should include:

   • Source installation type (RPM, Containerized, OpenShift)
   • Managed Ansible Automation Platform URL or deployment name
   • Source version (installer or Operator version)
2. The Ansible Site Reliability Engineering (SRE) team provides instructions in the support ticket on how to upload the resulting migration artifact to secure storage for processing.
3. The Ansible SRE team imports the migration artifact into the identified target instance and notifies the customer through the support ticket.
4. The Ansible SRE team notifies customers of successful migration.

Procedure

1. Log in to the Managed Ansible Automation Platform instance by using the local administrator account to confirm that data was imported.

2. Perform the following actions based on the configuration of the source deployment:

   1. Reconfigure Single Sign-On (SSO) authenticators and mappings to reflect the new URLs.

   2. Update private automation hub content to reflect the new URLs.

      1. Run the following command to update the automation hub repositories:

         curl -d '{\"verify_checksums\": true }' -X POST -k https://<platform url>/api/galaxy/pulp/api/v3/repair/ -u <admin_user>:<admin_password>

      2. Perform a sync on any repositories configured in automation hub.
      3. Push any custom execution environments from the source automation hub to the target automation hub.
   3. Reconfigure automation mesh.
3. After migration, you can request standard Site Reliability Engineering (SRE) tasks through support tickets, such as configuration of custom certificates, a custom domain, or connectivity through private endpoints.