Chapter 11. Performance tuning for Event-Driven Ansible controller
Event-Driven Ansible is a highly scalable, flexible automation capability. Event-Driven Ansible controller provides the interface in which Event-Driven Ansible automation performs. Tune your Event-Driven Ansible controller to optimize performance and scalability through:
- Characterizing your workload
- System level monitoring
- Performance troubleshooting
11.1. Characterizing your workload Copy linkLink copied to clipboard!
In Event-Driven Ansible controller, your workload includes the number of rulebook activations and events being received. Consider the following factors to characterize your Event-Driven Ansible controller workload:
- Number of simultaneous rulebook activations
- Number of events received by Event-Driven Ansible controller
11.2. Modifying the default memory limit for each rulebook activation Copy linkLink copied to clipboard!
Memory usage is based on the number of events that Event-Driven Ansible controller has to process. By default, each rulebook activation container has a 200 MB memory limit. For example, with 4 CPU and 16 GB of RAM, one rulebook activation container with an assigned 200 MB memory limit cannot handle more than 150,000 events per minute. If the number of parallel running rulebook activations is higher, then the maximum number of events each rulebook activation can process is reduced. If there are too many incoming events at a very high rate, the container can run out of memory trying to process the events. This will kill the container, and your rulebook activation will fail with a status code of 137.
To mitigate this status, you can modify the default memory limit for each rulebook activation during or after installation.
Procedure
Perform the following steps to modify your default memory limit for your rulebook activations during installation:
- Navigate to the setup inventory file.
-
Add
automationedacontroller_podman_mem_limitin the [all:vars] section. For example,automationedacontroller_podman_mem_limit='400m'. - Run the setup.
Perform the following steps to modify your default memory limit for your rulebook activations after installation:
-
Navigate to the environment file at
/etc/ansible-automation-platform/eda/settings.yaml. -
Modify the default container memory limit. For example,
PODMAN_MEM_LIMIT = '300m'. -
Restart the Event-Driven Ansible controller services using
automation-eda-controller-service restart.
-
Navigate to the environment file at
11.3. System level monitoring for Event-Driven Ansible controller Copy linkLink copied to clipboard!
After characterizing your workload to determine how many rulebook activations you are running in parallel and how many events you are receiving at any given point, you must consider monitoring your Event-Driven Ansible controller host at the system level. Using system level monitoring to review information about Event-Driven Ansible’s performance over time helps when diagnosing problems or when considering capacity for future growth.
System level monitoring includes the following information:
- Disk I/O
- RAM utilization
- CPU utilization
- Network traffic
Higher CPU, RAM, or Disk utilization can affect the overall performance of Event-Driven Ansible controller. For example, a high utilization of any of these system level resources indicates that either the Event-Driven Ansible controller is running too many rulebook activations, or some of the individual rulebook activations are using a high volume of resources. In this case, you must increase your system level resources to support your workload.