Chapter 9. Known issues

When using multiple log directories per broker (JBOD) and performing intra-broker log directory reassignment (moving replicas between log directories on the same broker), Apache Kafka can incorrectly mark a log directory as failed if a transient filesystem or I/O error occurs during the operation.

This issue is caused by a race condition between background log flush operations and file deletion during replica movement. Under these conditions, Kafka may encounter a NoSuchFileException or a related I/O error and treat it as a fatal storage failure. As a result, the broker takes the entire log directory offline to protect data integrity, and any partitions stored on that directory become unavailable. The log directory can remain marked as failed even after the reassignment completes.

This behavior affects intra-broker log directory reassignment only. Inter-broker partition reassignment is not affected.

Workaround

Restart the affected Kafka broker. On restart, the broker re-scans the log directories and marks the disk as healthy if no underlying filesystem issue is present.

This is a known issue in Apache Kafka. Work to address this issue is being tracked in the Apache Kafka issue tracker (KAFKA-19571). A fix will be included in a future release of Red Hat Streams for Apache Kafka once it is available in the underlying Apache Kafka distribution.

When running Streams for Apache Kafka in FIPS mode with JMX authentication enabled, clients may fail authentication. To work around this issue, do not enable JMX authentication while running in FIPS mode. We are investigating the issue and working to resolve it in a future release.