Chapter 5. Monitoring broker runtime metrics
When you install AMQ Broker, a Prometheus metrics plugin is included in your installation. Prometheus is software built for monitoring large, scalable systems and storing historical runtime data over an extended time period. You must modify the broker configuration to enable the plugin. When enabled, the plugin collects runtime metrics for the broker and exports these to Prometheus format. You can then use Prometheus to review the metrics. You might also use a graphical tool such as Grafana to configure more advanced visualizations of the data.
The Prometheus metrics plugin enables you to collect and export broker metrics in Prometheus format. However, Red Hat does not provide support for installation or configuration of Prometheus itself, nor of visualization tools such as Grafana. If you require support with installing, configuring, or running Prometheus or Grafana, visit the product websites for resources such as community support and documentation.
In addition to the broker metrics collected by the Prometheus plugin, you can modify the broker configuration to capture standard sets of metrics relating to the host Java Virtual Machine (JVM) for the broker. Specifically, you can capture JVM metrics for Garbage Collection (GC), memory, and threads.
The sections that follow describe:
5.1. Metrics overview
To monitor the health and performance of your broker instances, you can use the Prometheus plugin for AMQ Broker to monitor and store broker runtime metrics. The AMQ Broker Prometheus plugin exports the broker runtime metrics to Prometheus format, enabling you to use Prometheus itself to visualize and run queries on the data.
You can also use a graphical tool, such as Grafana, to configure more advanced visualizations and dashboards for the metrics that the Prometheus plugin collects.
The metrics that the plugin exports to Prometheus format are described below.
Broker metrics
artemis_address_memory_usage
- Number of bytes used by all addresses on this broker for in-memory messages.
artemis_address_memory_usage_percentage
-
Memory used by all the addresses on this broker as a percentage of the
global-max-size
parameter. artemis_connection_count
- Number of clients connected to this broker.
artemis_total_connection_count
- Number of clients that have connected to this broker since it was started.
Address metrics
artemis_routed_message_count
- Number of messages routed to one or more queue bindings.
artemis_unrouted_message_count
- Number of messages not routed to any queue bindings.
Queue metrics
artemis_consumer_count
- Number of clients consuming messages from a given queue.
artemis_delivering_durable_message_count
- Number of durable messages that a given queue is currently delivering to consumers.
artemis_delivering_durable_persistent_size
- Persistent size of durable messages that a given queue is currently delivering to consumers.
artemis_delivering_message_count
- Number of messages that a given queue is currently delivering to consumers.
artemis_delivering_persistent_size
- Persistent size of messages that a given queue is currently delivering to consumers.
artemis_durable_message_count
- Number of durable messages currently in a given queue. This includes scheduled, paged, and in-delivery messages.
artemis_durable_persistent_size
- Persistent size of durable messages currently in a given queue. This includes scheduled, paged, and in-delivery messages.
artemis_messages_acknowledged
- Number of messages acknowledged from a given queue since the queue was created.
artemis_messages_added
- Number of messages added to a given queue since the queue was created.
artemis_message_count
- Number of messages currently in a given queue. This includes scheduled, paged, and in-delivery messages.
artemis_messages_killed
- Number of messages removed from a given queue since the queue was created. The broker kills a message when the message exceeds the configured maximum number of delivery attempts.
artemis_messages_expired
- Number of messages expired from a given queue since the queue was created.
artemis_persistent_size
- Persistent size of all messages (both durable and non-durable) currently in a given queue. This includes scheduled, paged, and in-delivery messages.
artemis_scheduled_durable_message_count
- Number of durable, scheduled messages in a given queue.
artemis_scheduled_durable_persistent_size
- Persistent size of durable, scheduled messages in a given queue.
artemis_scheduled_message_count
- Number of scheduled messages in a given queue.
artemis_scheduled_persistent_size
- Persistent size of scheduled messages in a given queue.
For higher-level broker metrics that are not listed above, you can calculate these by aggregating lower-level metrics. For example, to calculate total message count, you can aggregate the artemis_message_count
metrics from all queues in your broker deployment.
For an on-premise deployment of AMQ Broker, metrics for the Java Virtual Machine (JVM) hosting the broker are also exported to Prometheus format. This does not apply to a deployment of AMQ Broker on OpenShift Container Platform.
5.2. Enabling the Prometheus metrics plugin for AMQ Broker
When you install AMQ Broker, a Prometheus metrics plugin is included in your installation. Although the plugin is already configured for use, you need to enable the plugin in your broker configuration. When enabled, the plugin collects runtime metrics for the broker and exports these to Prometheus format.
The following procedure shows how to enable the Prometheus plugin for AMQ Broker.
Procedure
Copy the Prometheus metrics plugin
.jar
file from your AMQ Broker 7.9 extracted archive to thelib
directory of your broker instance.$ cp amq-broker-7.9.0/lib/artemis-prometheus-metrics-plugin-1.0.0.CR1-redhat-00010.jar <broker_instance_dir>/lib
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. Enable the Prometheus plugin in the broker configuration. Add a
<metrics>
element with a<plugin>
sub-element, configured as shown below.<metrics> <plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/> </metrics>
-
Save the
broker.xml
configuration file. The metrics plugin starts to gather broker runtime metrics in Prometheus format.
5.3. Configuring the broker to collect JVM metrics
The following procedure shows how to configure the broker to collect Java Virtual Machine (JVM) metrics for Garbage Collection (GC), memory, and threads.
Prerequisites
- You have previously enabled the Prometheus metrics plugin in your broker configuration. For more information, see Section 5.2, “Enabling the Prometheus metrics plugin for AMQ Broker”.
Procedure
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. In the
<metrics>
element that you added to the configuration when enabling the Prometheus metrics plugin, specify whether the broker collects JVM metrics for Garbage Collection (GC), memory, and threads. For example:<metrics> <jvm-gc>true</jvm-gc> <jvm-memory>true</jvm-memory <jvm-threads>true</jvm-threads> <plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/> </metrics>
NoteIf you do not explicitly add the
jvm-memory
parameter to your configuration and specify a value, the broker uses a default value oftrue
. This means that the broker exports JVM memory metrics by default. The default values of thejvm-gc
andjvm-threads
parameters arefalse
.-
Save the
broker.xml
configuration file. The broker starts to gather the JVM metrics that you have enabled. These metrics are also exported to Prometheus format.
5.4. Disabling metrics collection for specific addresses
When you configure a metrics plugin for AMQ Broker (for example, the Prometheus metrics plugin), metrics collection is enabled by default. However, within the address-setting
configuration element of a specific address or set of addresses, you can explicitly disable metrics collection.
The following procedure shows how disable metrics collection for a specific address or set of addresses.
Procedure
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. In the
address-setting
element of a matching address or set of addresses, add theenable-metrics
parameter and set the value of the parameter tofalse
. For example, the following configuration disables metrics collection for an address calledorders
.<configuration> <core> ... <address-settings> <address-setting match="orders"> ... <enable-metrics>false</enable-metrics> ... </address-setting> </address-settings> ... </core> </configuration>
5.5. Accessing broker runtime data using Prometheus
Prerequisites
- To query and visualize the broker runtime data collected by the Prometheus plugin, you need to install Prometheus. For more information, see Installing Prometheus in the Prometheus documentation.
Procedure
-
From your Prometheus installation directory, open the
prometheus.yml
configuration file. -
In the
static_configs
section of the configuration file, change thetargets
element tolocalhost:8161
. This location is where the broker runs its web server. By default,/metrics
is appended to this host name, forming the full path to the metrics stored on the broker web server. To view the broker runtime metrics collected by the Prometheus plugin, open
localhost:8161/metrics
in a web browser.On the resulting web page, you see the current values of the metrics collected by the plugin, based on the queues and addresses that you have configured on the broker. If you have more than one running broker instance in your JVM, you see metrics for each broker.
From your Prometheus installation directory, run Prometheus.
$ ./prometheus
When Prometheus starts, the shell output includes the following line:
component=web, msg=”Start listening for connections” address=0.0.0.0:9090
The preceding line indicates that Prometheus is listening for HTTP traffic on port 9090.
-
To access the Prometheus web console, open
127.0.0.1:9090
in a web browser. In the Prometheus web console, you can use the
Expression
field to create a query on your broker data. The queries you create are based on the Prometheus query language, PromQL. Broker metrics that are available to insert in your query are in theInsert metric
drop-down list.As a simple example, suppose you want to query the message count on the DLQ queue, over time. In this case, select
artemis_message_count
from the metrics drop-down list. Complete your query by specifying the DLQ queue name and address. This example query is shown below.artemis_message_count{address=“DLQ”, queue=“DLQ”}
For more advanced visualizations, you can use regular expressions to create complex queries that overlay several metrics, for example. Or, you can perform mathematical operations on a number of metrics, such as aggregating them. For more information about creating Prometheus queries, see Querying Prometheus in the Prometheus documentation.