Chapter 9. Monitoring Debezium
You can use the JMX metrics provided by Apache Zookeeper, Apache Kafka, and Kafka Connect to monitor Debezium. To use these metrics, you must enable them when you start the Zookeeper, Kafka, and Kafka Connect services. Enabling JMX involves setting the correct environment variables.
If you are running multiple services on the same machine, be sure to use distinct JMX ports for each service.
9.1. Metrics for monitoring Debezium connectors
In addition to the built-in support for JMX metrics in Kafka, Zookeeper, and Kafka Connect, each connector provides additional metrics that you can use to monitor their activities.
9.2. Enabling JMX in local installations
With Zookeeper, Kafka, and Kafka Connect, you enable JMX by setting the appropriate environment variables when you start each service.
9.2.1. Zookeeper JMX environment variables
Zookeeper has built-in support for JMX. When running Zookeeper using a local installation, the zkServer.sh script recognizes the following environment variables:
JMXPORT-
Enables JMX and specifies the port number that will be used for JMX. The value is used to specify the JVM parameter
-Dcom.sun.management.jmxremote.port=$JMXPORT. JMXAUTH-
Whether JMX clients must use password authentication when connecting. Must be either
trueorfalse. The default isfalse. The value is used to specify the JVM parameter-Dcom.sun.management.jmxremote.authenticate=$JMXAUTH. JMXSSL-
Whether JMX clients connect using SSL/TLS. Must be either
trueorfalse. The default isfalse. The value is used to specify the JVM parameter-Dcom.sun.management.jmxremote.ssl=$JMXSSL. JMXLOG4J-
Whether the Log4J JMX MBeans should be disabled. Must be either
true(default) orfalse. The default istrue. The value is used to specify the JVM parameter-Dzookeeper.jmx.log4j.disable=$JMXLOG4J.
9.2.2. Kafka JMX environment variables
When running Kafka using a local installation, the kafka-server-start.sh script recognizes the following environment variables:
JMX_PORT-
Enables JMX and specifies the port number that will be used for JMX. The value is used to specify the JVM parameter
-Dcom.sun.management.jmxremote.port=$JMX_PORT. KAFKA_JMX_OPTSThe JMX options, which are passed directly to the JVM during startup. The default options are:
-
-Dcom.sun.management.jmxremote -
-Dcom.sun.management.jmxremote.authenticate=false -
-Dcom.sun.management.jmxremote.ssl=false
-
9.2.3. Kafka Connect JMX environment variables
When running Kafka using a local installation, the connect-distributed.sh script recognizes the following environment variables:
JMX_PORT-
Enables JMX and specifies the port number that will be used for JMX. The value is used to specify the JVM parameter
-Dcom.sun.management.jmxremote.port=$JMX_PORT. KAFKA_JMX_OPTSThe JMX options, which are passed directly to the JVM during startup. The default options are:
-
-Dcom.sun.management.jmxremote -
-Dcom.sun.management.jmxremote.authenticate=false -
-Dcom.sun.management.jmxremote.ssl=false
-
9.3. Monitoring Debezium on OpenShift
If you are using Debezium on OpenShift, you can obtain JMX metrics by opening a JMX port on 9999. For information about configuring JMX connection options, see the KafkaJmxOptions schema reference in the AMQ Streams API Reference.
In addition, you can use Prometheus and Grafana to monitor the JMX metrics. For more information, see Monitoring in AMQ Streams on OpenShift Overview and Setting up metrics and dashboards in Deploying and Managing AMQ Streams on OpenShift
