Chapter 10. Configuring logging for Kafka components

Procedure

1. Switch to the kafka user:

   su - kafka

   Copy to Clipboard Toggle word wrap

2. List all the broker loggers for a broker by using the kafka-configs.sh tool:

   /opt/kafka/bin/kafka-configs.sh --bootstrap-server <broker_address> --describe --entity-type broker-loggers --entity-name BROKER-ID

   Copy to Clipboard Toggle word wrap

   For example, for broker 0:

   /opt/kafka/bin/kafka-configs.sh --bootstrap-server localhost:9092 --describe --entity-type broker-loggers --entity-name 0

   Copy to Clipboard Toggle word wrap

   This returns the logging level for each logger: TRACE, DEBUG, INFO, WARN, ERROR, or FATAL.

   For example:

   #...
   kafka.controller.ControllerChannelManager=INFO sensitive=false synonyms={}
   kafka.log.TimeIndex=INFO sensitive=false synonyms={}

   Copy to Clipboard Toggle word wrap

3. Change the logging level for one or more broker loggers. Use the --alter and --add-config options and specify each logger and its level as a comma-separated list in double quotes.

   /opt/kafka/bin/kafka-configs.sh --bootstrap-server <broker_address> --alter --add-config "LOGGER-ONE=NEW-LEVEL,LOGGER-TWO=NEW-LEVEL" --entity-type broker-loggers --entity-name BROKER-ID

   Copy to Clipboard Toggle word wrap

   For example, for broker 0:

   /opt/kafka/bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config "kafka.controller.ControllerChannelManager=WARN,kafka.log.TimeIndex=WARN" --entity-type broker-loggers --entity-name 0

   Copy to Clipboard Toggle word wrap

   If successful this returns:

   Completed updating config for broker: 0.

   Copy to Clipboard Toggle word wrap

Procedure

1. Switch to the kafka user:

   su - kafka

   Copy to Clipboard Toggle word wrap

2. Check the current logging level for the loggers configured in the connect-log4j.properties file:

   $ cat /opt/kafka/config/connect-log4j.properties
   
   # ...
   log4j.rootLogger=INFO, stdout, connectAppender
   # ...
   log4j.logger.org.apache.zookeeper=ERROR
   log4j.logger.org.reflections=ERROR

   Copy to Clipboard Toggle word wrap

   Use a curl command to check the logging levels from the admin/loggers endpoint of the Kafka Connect API:

   curl -s http://localhost:8083/admin/loggers/ | jq
   
   {
     "org.apache.zookeeper": {
       "level": "ERROR"
     },
     "org.reflections": {
       "level": "ERROR"
     },
     "root": {
       "level": "INFO"
     }
   }

   Copy to Clipboard Toggle word wrap

   jq prints the output in JSON format. The list shows standard org and root level loggers, plus any specific loggers with modified logging levels.

   If you configure TLS (Transport Layer Security) authentication for the admin.listeners configuration in Kafka Connect, then the address of the loggers endpoint is the value specified for admin.listeners with the protocol as https, such as https://localhost:8083.

   You can also get the log level of a specific logger:

   curl -s http://localhost:8083/admin/loggers/org.apache.kafka.connect.mirror.MirrorCheckpointConnector | jq
   
   {
     "level": "INFO"
   }

   Copy to Clipboard Toggle word wrap

3. Use a PUT method to change the log level for a logger:

   curl -Ss -X PUT -H 'Content-Type: application/json' -d '{"level": "TRACE"}' http://localhost:8083/admin/loggers/root
   
   {
     # ...
   
     "org.reflections": {
       "level": "TRACE"
     },
     "org.reflections.Reflections": {
       "level": "TRACE"
     },
     "root": {
       "level": "TRACE"
     }
   }

   Copy to Clipboard Toggle word wrap

   If you change the root logger, the logging level for loggers that used the root logging level by default are also changed.