Chapter 12. Configuring logging for Kafka components

Procedure

1. List all loggers for a Kafka node using the kafka-configs.sh tool:

   ./bin/kafka-configs.sh --bootstrap-server localhost:9092 --describe --entity-type broker-loggers --entity-name 0

   Copy to Clipboard Toggle word wrap

   Here, --entity-name 0 specifies Kafka node 0. The node ID corresponds to the broker or controller ID in the Kafka cluster.

   This returns the logging level for each logger: TRACE, DEBUG, INFO, WARN, ERROR, or FATAL.

   Example output:

   #...
   kafka.controller.ControllerChannelManager=INFO sensitive=false synonyms={}
   kafka.log.TimeIndex=INFO sensitive=false synonyms={}

   Copy to Clipboard Toggle word wrap

2. Change the logging level for one or more loggers. Use the --alter and --add-config options and specify each logger and its level as a comma-separated list in double quotes:

   ./bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config "kafka.log.LogCleaner=TRACE" --entity-type broker-loggers --entity-name 0

   Copy to Clipboard Toggle word wrap

   Here, we update the logging level for kafka.log.LogCleaner from to TRACE.

   If successful, the command returns:

   Completed updating config for broker: 0.

   Copy to Clipboard Toggle word wrap

Procedure

1. Check the current logging levels in the connect-log4j2.yaml file:

   cat ./config/connect-log4j2.yaml

   Copy to Clipboard Toggle word wrap

   Example output:

   Root:
     level: INFO
   # ...

   Copy to Clipboard Toggle word wrap

   Use a curl command to check the logging levels from the admin/loggers endpoint of the Kafka Connect API:

   curl -s http://localhost:8083/admin/loggers/ | jq

   Copy to Clipboard Toggle word wrap

   Example response:

   {
     "Root": {
       "level": "INFO"
     }
   }

   Copy to Clipboard Toggle word wrap

   jq prints the output in JSON format. The list shows the standard root level logger, plus any specific loggers with modified logging levels.

   If TLS is enabled, use https:// instead of http://, and specify the port configured in admin.listeners.

   You can also get the log level of a specific logger:

   curl -s http://localhost:8083/admin/loggers/org.apache.kafka.connect.runtime.Worker | jq

   Copy to Clipboard Toggle word wrap

   Here, we retrieve the log level for org.apache.kafka.connect.runtime.Worker.

   Example response:

   {
     "level": "INFO"
   }

   Copy to Clipboard Toggle word wrap

2. Change a logger’s level dynamically using a PUT request:

   curl -X PUT -H 'Content-Type: application/json' -d '{"level": "TRACE"}' http://localhost:8083/admin/loggers/root

   Copy to Clipboard Toggle word wrap

   Example response:

   {
     "Root": {
       "level": "TRACE"
     }
   }
   # ...

   Copy to Clipboard Toggle word wrap

   Changing the Root logger affects all loggers that inherit from it.

   You can also adjust the logging level for a specific component:

   curl -X PUT -H 'Content-Type: application/json' -d '{"level": "DEBUG"}' http://localhost:8083/admin/loggers/org.apache.kafka.connect.runtime.Worker

   Copy to Clipboard Toggle word wrap

   Example response:

   {
     "org.apache.kafka.connect.runtime.Worker": {
       "level": "DEBUG"
     }
   }

   Copy to Clipboard Toggle word wrap