Troubleshooting Ansible Automation Platform

Procedure

1. Log in to your cluster:

   oc login <openshift_url>

   Copy to Clipboard Toggle word wrap

2. Run one of the following commands based on your level of access in the cluster:

   • Run must-gather across the entire cluster:

     oc adm must-gather --image=registry.redhat.io/ansible-automation-platform-25/aap-must-gather-rhel8 --dest-dir <dest_dir>

     Copy to Clipboard Toggle word wrap
     • --image specifies the image that gathers data
     • --dest-dir specifies the directory for the output

   • Run must-gather for a specific namespace in the cluster:

     oc adm must-gather --image=registry.redhat.io/ansible-automation-platform-25/aap-must-gather-rhel8 --dest-dir <dest_dir> – /usr/bin/ns-gather <namespace>

     Copy to Clipboard Toggle word wrap
     • – /usr/bin/ns-gather limits the must-gather data collection to a specified namespace

3. To attach the must-gather archive to your support case, create a compressed file from the must-gather directory created before and attach it to your support case.

   • For example, on a computer that uses a Linux operating system, run the following command, replacing <must-gather-local.5421342344627712289/> with the must-gather directory name:

     $ tar cvaf must-gather.tar.gz <must-gather.local.5421342344627712289/>

     Copy to Clipboard Toggle word wrap

Procedure

1. Change the /etc/pulp/settings.py file on private automation hub and add one of the following parameters depending on your configuration:

   • Single controller

     CONNECTED_ANSIBLE_CONTROLLERS = ['<https://my.controller.node>']

     Copy to Clipboard Toggle word wrap

   • Many controllers behind a load balancer

     CONNECTED_ANSIBLE_CONTROLLERS = ['<https://my.controller.loadbalancer>']

     Copy to Clipboard Toggle word wrap

   • Many controllers without a load balancer

     CONNECTED_ANSIBLE_CONTROLLERS = ['<https://my.controller.node1>', '<https://my.controller2.node2>']

     Copy to Clipboard Toggle word wrap

2. Stop all of the private automation hub services:

   # systemctl stop pulpcore.service pulpcore-api.service pulpcore-content.service pulpcore-worker@1.service pulpcore-worker@2.service nginx.service redis.service

   Copy to Clipboard Toggle word wrap

3. Restart all of the private automation hub services:

   # systemctl start pulpcore.service pulpcore-api.service pulpcore-content.service pulpcore-worker@1.service pulpcore-worker@2.service nginx.service redis.service

   Copy to Clipboard Toggle word wrap

   Verification

   • Verify that you can now use the Use in Controller option in private automation hub.

Procedure

1. Create a collections folder inside of the project repository.

2. Add a requirements.yml file inside of the collections folder and add the collection:

   collections:
   - <collection_name>

   Copy to Clipboard Toggle word wrap

Procedure

1. Run the following commands to list all of the pending jobs:

   # awx-manage shell_plus

   Copy to Clipboard Toggle word wrap

   >>> UnifiedJob.objects.filter(status='pending')

   Copy to Clipboard Toggle word wrap

2. Cancel the pending jobs by using one of the following methods:

   • To cancel all pending jobs, run the following command:

     >>> UnifiedJob.objects.filter(status='pending').update(status='canceled')

     Copy to Clipboard Toggle word wrap

   • To cancel a single job, run the following command, replacing <job_id> with the job ID to cancel:

     >>> UnifiedJob.objects.filter(id=<job_id>).update(status='canceled')

     Copy to Clipboard Toggle word wrap

Procedure

1. Go to automation controller.
2. From the navigation panel, select Administration → Execution Environments.
3. Click the execution environment assigned to the job template that is failing.
4. Click Edit.
5. Assign the appropriate Registry credential from your private automation hub to the execution environment.

Procedure

1. In all controller and hybrid nodes, run the following commands to create a file called custom.py:

   # touch /etc/tower/conf.d/custom.py

   Copy to Clipboard Toggle word wrap

   # chmod 640 /etc/tower/conf.d/custom.py

   Copy to Clipboard Toggle word wrap

   # chown root:awx /etc/tower/conf.d/custom.py

   Copy to Clipboard Toggle word wrap

2. Add the following to the /etc/tower/conf.d/custom.py file:

   DEFAULT_CONTAINER_RUN_OPTIONS = ['--network', 'slirp4netns:enable_ipv6=true,cidr=192.168.1.0/24']

   Copy to Clipboard Toggle word wrap
   • 192.168.1.0/24 is the value for the new CIDR in this example.

3. Stop and start the automation controller service in all controller and hybrid nodes:

   # automation-controller-service stop

   Copy to Clipboard Toggle word wrap

   # automation-controller-service start

   Copy to Clipboard Toggle word wrap

   All containers will start on the new CIDR.

Procedure

1. Check if the server is reachable over SSL/TLS.

   1. Run the following command to confirm whether the server is reachable over SSL/TLS and to see the full certificate chain:

      # true | openssl s_client -showcerts -connect <fqdn_or_ip>:<port>

      Copy to Clipboard Toggle word wrap
   2. Replace <fqdn_or_ip> and <port> with suitable values.

2. Verify the certificate details.

   1. Run the following command to view the details of a certificate:

      # openssl x509 -in <path_to_certificate> -noout -text

      Copy to Clipboard Toggle word wrap

3. Replace <path_to_certificate> with the path to the certificate file you want to inspect.

   The result of the command shows information such as:

   • Subject - The entity the certificate has been issued to.
   • Issuer - The CA that issued the certificate.
   • Validity "Not Before" - The date the certificate was issued.
   • Validity "Not After" - The date the certificate expires.

4. Verify a trusted CA signed the certificate.

   1. Run the following command to verify that a specific certificate is valid and was signed by a trusted CA:

      openssl verify -CAfile <path_to_ca_public_certificate> <path_to_server_certificate_file_to_verify>

      Copy to Clipboard Toggle word wrap
   2. If the command returns OK, it means the certificate file is valid and signed by a trusted CA.

Procedure

1. To resolve this issue, perform the following tasks for all controller hosts:

   1. For each controller host, add the platform gateway URL as a trusted source in the CSRF_TRUSTED_ORIGIN setting in the settings.py file.

      For example, if you configured the platform gateway URL as https://www.example.com, you must add that URL in the settings.py file too as shown:

      CSRF_TRUSTED_ORIGINS = ['https://appX.example.com:8443','https://www.example.com']

      Copy to Clipboard Toggle word wrap
   2. Restart each controller host by using the automation-controller-service restart command so that the URL changes are implemented. For the procedure, see Start, stop, and restart automation controller in Configuring automation execution.