Chapter 6. Topics page

Procedure

1. Log in to the Kafka cluster in the Streams for Apache Kafka Console, then click Topics.
2. On the Topics page, click the name of the topic you want to inspect.

3. Check the information on the Messages tab.

   For each message, view the timestamp (in UTC), offset, key, and value.

   Click on a message to view the full message details.

   Click the Manage columns icon (represented as two-column icon) to choose the information to display.

4. Click the search dropdown and select the advanced search options to refine your search.

   Choose to display the latest messages or messages from a specified time or offset. You can display messages for all partitions or a specified partition.

   When you are done, you can click the CSV icon (represented as a CSV file icon) to download the information on the returned messages.

   Refining your search

   In this example, search terms, and message, retrieval, and partition options are combined:

   • messages=timestamp:2024-03-01T00:00:00Z retrieve=50 partition=1 Error on page load where=value

     The filter searches for the text "Error on page load" in partition 1 as a message value, starting from March 1, 2024, and retrieves up to 50 messages.

   Search terms

   Enter search terms as text (has the words) to find specific matches and define where in a message to look for the term. You can search anywhere in the message or narrow the search to the key, header, or value.

   For example:

   • messages=latest retrieve=100 642-26-1594 where=key

   This example searches the latest 100 messages on message key 642-26-1594.

   Message options

   Set the starting point for returning messages.

   • Latest messages to start from the latest message.

     • messages=latest

   • From offset to start from an offset in a partition. In some cases, you may want to specify an offset without a partition. However, the most common scenario is to search by offset within a specific partition.

     • messages=offset:5600253 partition=0

   • From timestamp to start from an exact time and date in ISO 8601 format.

     • messages=timestamp:2024-03-14T00:00:00Z

   • From Unix timestamp to start from a time and date in Unix format.

     • messages=epoch:1

   Retrieval options

   Set a retrieval option.

   • Number of messages to return a specified number of messages.

     • messages=latest retrieve=50

   • Continuously to return the latest messages in real-time. Click the pause button (represented by two vertical lines) to pause the refresh. Unpause to continue the refresh.

     • retrieve=continuously

   Partition options

   Choose to run a search against all partitions or a specific partition.

Procedure

1. Log in to the Kafka cluster in the Streams for Apache Kafka Console, then click Topics.
2. On the Topics page, click the name of the topic you want to inspect.

3. Check the information on the Partitions tab.

   For each partition, view the replication status, as well as information on designated partition leaders, replica brokers, and the amount of data stored by the partition.

   Replication status:

   In-sync

   All partitions in the topic are fully replicated. A partition is fully-replicated when its replicas (followers) are 'in-sync' with the designated partition leader. Replicas are 'in-sync' if they have fetched records up to the log end offset of the leader partition within an allowable lag time, as determined by replica.lag.time.max.ms.

   Under-replicated

   A partition is under-replicated if some of its replicas (followers) are not in-sync. An under-replicated status signals potential issues in data replication.

   Offline

   Some or all partitions in the topic are currently unavailable. This may be due to issues such as broker failures or network problems, which need investigating and addressing.

   In addition, check information on the broker designated as partition leader and the brokers that contain the replicas:

   Leader

   The leader handles all produce requests. Followers on other brokers replicate the leader’s data. A follower is considered in-sync if it catches up with the leader’s latest committed message.

   Preferred leader

   When creating a new topic, Kafka’s leader election algorithm assigns a leader from the list of replicas for each partition. The algorithm aims for a balanced spread of leadership assignments. A "Yes" value indicates the current leader is the preferred leader, suggesting a balanced leadership distribution. A "No" value may suggest imbalances in the leadership assignments, requiring further investigation. If the leadership assignments of partitions are not well-balanced, it can contribute to size discrepancies. A well-balanced Kafka cluster should distribute leadership roles across brokers evenly.

   Replicas

   Followers that replicate the leader’s data. Replicas provide fault tolerance and data availability.

   Note

   Discrepancies in the distribution of data across brokers may indicate balancing issues in the Kafka cluster. If certain brokers are consistently handling larger amounts of data, it may indicate that partitions are not evenly distributed across the brokers. This could lead to uneven resource utilization and potentially impact the performance of those brokers.

Procedure

1. Log in to the Kafka cluster in the Streams for Apache Kafka Console, then click Topics.
2. On the Topics page, click the name of the topic you want to inspect.
3. Check the information on the Consumer groups tab.

4. Click a consumer group name to view consumer group members.

   Consumer groups

   For each consumer group, view the status, the overall consumer lag across all partitions, and the number of members. For more information on checking consumer groups, see Chapter 9, Consumer Groups page.

   Group members

   For each group member, you see the unique (consumer) client ID assigned to the consumer within the consumer group, overall consumer lag, and the number of assigned partitions. For more information on checking consumer group members, see Section 9.1, “Checking consumer group members”.

Procedure

1. Log in to the Kafka cluster in the Streams for Apache Kafka Console, then click Topics.
2. On the Topics page, click the name of the topic you want to inspect.

3. Check the information on the Configuration tab.

   Filter the configuration properties to narrow results by data source:

   • DEFAULT_CONFIG: The fallback value used when no other configuration is specified.
   • STATIC_BROKER_CONFIG: Predefined, broker-wide values that apply to all topics by default.
   • DYNAMIC_TOPIC_CONFIG: Topic-specific values that override all default or broker-level settings.