Chapter 9. Known issues

Deploying a Kafka instance on OpenShift Container Platform (OCP) version 4.16 triggers a continuous creation of dockercfg secrets within the Kafka namespace.

This issue is caused by the openshift.io/internal-registry-pull-secret-ref annotation being added to service accounts, which leads to a reconciliation loop as Streams for Apache Kafka and OpenShift repeatedly rewrite the annotation. Over time, this can result in the accumulation of thousands of unnecessary secrets.

If upgrading is not immediately possible, a temporary workaround is to manually configure the openshift.io/internal-registry-pull-secret-ref annotation for each service account to prevent the reconciliation loop.

template:
  serviceAccount:
    metadata:
      annotations:
        openshift.io/internal-registry-pull-secret-ref: my-cluster-entity-operator-dockercfg-qxwxd

There are known incompatibility issues when using RHEL 7 with Kafka 3.7. As a result, RHEL 7 is no longer supported. The issues arise due to an outdated GCC (GNU Compiler Collection) version on RHEL 7, which is incompatible with the version of the RocksDB JNI library (org.rocksdb:rocksdbjni:7.9.2) required by Kafka 3.7.

RocksDB JNI version 7.9.2 requires a newer version of the GCC and the associated libstdc++ library than what is available on RHEL 7. Snappy compression and Kafka Streams, which depend on RocksDB, will not function correctly on RHEL 7 due to these outdated libraries.

Auto-restart of MirrorMaker 2 connectors does not work because the state is not passed to the connector operator. This will be fixed in a future release of Streams for Apache Kafka.

The Streams for Apache Kafka Cluster Operator does not start on Internet Protocol version 6 (IPv6) clusters.

Cruise Control for Streams for Apache Kafka has a known issue that relates to the calculation of CPU utilization estimation. CPU utilization is calculated as a percentage of the defined capacity of a broker pod. The issue occurs when running Kafka brokers across nodes with varying CPU cores. For example, node1 might have 2 CPU cores and node2 might have 4 CPU cores. In this situation, Cruise Control can underestimate and overestimate CPU load of brokers The issue can prevent cluster rebalances when the pod is under heavy load.

There are two workarounds for this issue.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    # ...
  zookeeper:
    # ...
  entityOperator:
    topicOperator: {}
    userOperator: {}
  cruiseControl:
    brokerCapacity:
      inboundNetwork: 10000KB/s
      outboundNetwork: 10000KB/s
    config:
      hard.goals: >
        com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.MinTopicLeadersPerBrokerGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal
      default.goals: >
        com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.MinTopicLeadersPerBrokerGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.PotentialNwOutGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskUsageDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundUsageDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundUsageDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderReplicaDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderBytesInDistributionGoal

For more information, see Insufficient CPU capacity.

When running Streams for Apache Kafka in FIPS mode with JMX authentication enabled, clients may fail authentication. To work around this issue, do not enable JMX authentication while running in FIPS mode. We are investigating the issue and working to resolve it in a future release.