Chapter 12. Configuring logging for Kafka components

Procedure

1. List all loggers for a Kafka node using the kafka-configs.sh tool:

   ./bin/kafka-configs.sh --bootstrap-server localhost:9092 --describe --entity-type broker-loggers --entity-name 0

   Here, --entity-name 0 specifies Kafka node 0. The node ID corresponds to the broker or controller ID in the Kafka cluster.

   This returns the logging level for each logger: TRACE, DEBUG, INFO, WARN, ERROR, or FATAL.

   Example output:

   #...
   kafka.controller.ControllerChannelManager=INFO sensitive=false synonyms={}
   kafka.log.TimeIndex=INFO sensitive=false synonyms={}

2. Change the logging level for one or more loggers. Use the --alter and --add-config options and specify each logger and its level as a comma-separated list in double quotes:

   ./bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config "kafka.log.LogCleaner=TRACE" --entity-type broker-loggers --entity-name 0

   Here, we update the logging level for kafka.log.LogCleaner from to TRACE.

   If successful, the command returns:

   Completed updating config for broker: 0.

Procedure

1. Check the current logging levels in the connect-log4j2.yaml file:

   cat ./config/connect-log4j2.yaml

   Example output:

   Root:
     level: INFO
   # ...

   Use a curl command to check the logging levels from the admin/loggers endpoint of the Kafka Connect API:

   curl -s http://localhost:8083/admin/loggers/ | jq

   Example response:

   {
     "Root": {
       "level": "INFO"
     }
   }

   jq prints the output in JSON format. The list shows the standard root level logger, plus any specific loggers with modified logging levels.

   If TLS is enabled, use https:// instead of http://, and specify the port configured in admin.listeners.

   You can also get the log level of a specific logger:

   curl -s http://localhost:8083/admin/loggers/org.apache.kafka.connect.runtime.Worker | jq

   Here, we retrieve the log level for org.apache.kafka.connect.runtime.Worker.

   Example response:

   {
     "level": "INFO"
   }

2. Change a logger’s level dynamically using a PUT request:

   curl -X PUT -H 'Content-Type: application/json' -d '{"level": "TRACE"}' http://localhost:8083/admin/loggers/root

   Example response:

   {
     "Root": {
       "level": "TRACE"
     }
   }
   # ...

   Changing the Root logger affects all loggers that inherit from it.

   You can also adjust the logging level for a specific component:

   curl -X PUT -H 'Content-Type: application/json' -d '{"level": "DEBUG"}' http://localhost:8083/admin/loggers/org.apache.kafka.connect.runtime.Worker

   Example response:

   {
     "org.apache.kafka.connect.runtime.Worker": {
       "level": "DEBUG"
     }
   }