Chapter 9. Rulebook activations troubleshooting
Occasionally, rulebook activations might fail due to a variety of reasons that can be resolved. This section contains a list of possible issues and how you can resolve them.
9.1. Activation stuck in Pending state
Perform the following steps if your rulebook activation is stuck in Pending state.
Procedure
Confirm whether there are other running activations and if you have reached the limits (for example, memory or CPU limits).
- If there are other activations running, terminate one or more of them, if possible.
If not, check that the default worker, Redis, and activation worker are all running. If all systems are working as expected, check your eda-server internal logs in the worker, scheduler, API, and nginx containers and services to see if the problem can be determined.
NoteThese logs reveal the source of the issue, such as an exception thrown by the code, a runtime error with network issues, or an error with the rulebook code. If your internal logs do not provide information that leads to resolution, report the issue to Red Hat support.
If you need to make adjustments, see the Modifying the number of simultaneous rulebook activations.
NoteTo adjust the maximum number of simultaneous activations for Ansible Automation Platform Operator on OpenShift Container Platform deployments, see Modifying the number of simultaneous rulebook activations during or after Event-Driven Ansible controller installation in Installing on OpenShift Container Platform.
9.2. Activation keeps restarting
Perform the following steps if your rulebook activation keeps restarting.
Procedure
- Log in to Ansible Automation Platform.
-
From the navigation panel, select
. - From the Rulebook Activations page, select the activation in your list that keeps restarting. The Details page is displayed.
- Click the History tab for more information and select the rulebook activation that keeps restarting. The Details tab is displayed and shows the output information.
Check the Restart policy field for your activation.
There are three selections available: On failure (restarts a rulebook activation when the container process fails), Always (always restarts regardless of success or failure with no more than 5 restarts), or Never (never restarts when the container process ends).
- Confirm that your rulebook activation Restart policy is set to On failure. This is an indication that an issue is causing it to fail.
- To possibly diagnose the problem, check the YAML code and the instance logs of the rulebook activation for errors.
- If you cannot find a solution with the restart policy values, proceed to the next steps related to the Log level.
Check your log level for your activation.
- If your default log level is Error, go back to the Rulebook Activation page and recreate your activation following procedures in Setting up rulebook a activation.
- Change the Log level to Debug.
- Run the activation again and navigate to the History tab from the activation details page.
- On the History page, click one of your recent activations and view the Output.
9.3. Event streams not sending events to activation
If you are using event streams to send events to your rulebook activations, occasionally those events might not be successfully routed to your rulebook activation.
Procedure
Try the following options to resolve this.
- Ensure that each of your event streams in Event-Driven Ansible controller is not in Test mode . This means activations would not receive the events.
- Verify that the origin service is sending the request properly.
- Check that the network connection to your platform gateway instance is stable. If you have set up event streams, this is the entry of the event stream request from the sender.
- Verify that the proxy in the platform gateway is running.
- Confirm that the event stream worker is up and running, and able to process the request.
- Verify that your credential is correctly set up in the event stream.
Confirm that the request complies with the authentication mechanism determined by the set credential (for example, basic must contain a header with the credentials or HMAC must contain the signature of the content in a header, and similar).
NoteThe credentials might have been changed in Event-Driven Ansible controller, but not updated in the origin service.
- Verify that the rulebook that is running in the activation reacts to these events. This would indicate that you wrote down the event source and added actions that consume the events coming in. Otherwise, the event does reach the activation but there is nothing to activate it.
- If you are using self-signed certificates, you might want to disable certificate validation when sending webhooks from vendors. Most of the vendors have an option to disable certificate validation for testing or non-production environments.
9.4. Cannot connect to the 2.5 automation controller when running activations
You might experience a failed connection to automation controller when you run your activations.
Procedure
To help resolve the issue, confirm that you have set up a Red Hat Ansible Automation Platform credential and have obtained the correct automation controller URL.
- If you have not set up a Red Hat Ansible Automation Platform credential, follow the procedures in Setting up a Red Hat Ansible Automation Platform credential. Ensure that this credential has the host set to the following URL format: https://<your_gateway>/api/controller
- When you have completed this process, try setting up your rulebook activation again.