Chapter 6. Securing Kafka

AMQ Streams supports Transport Layer Security (TLS), a protocol for encrypted communication.

Communication is always encrypted for communication between:

You can also configure TLS between Kafka brokers and clients by applying TLS encryption to the listeners of the Kafka broker. TLS is specified for external clients when configuring an external listener.

AMQ Streams components and Kafka clients use digital certificates for encryption. The Cluster Operator sets up certificates to enable encryption within the Kafka cluster. You can provide your own server certificates, referred to as Kafka listener certificates, for communication between Kafka clients and Kafka brokers, and inter-cluster communication.

AMQ Streams uses Secrets to store the certificates and private keys required for TLS in PEM and PKCS #12 format.

A TLS Certificate Authority (CA) issues certificates to authenticate the identity of a component. AMQ Streams verifies the certificates for the components against the CA certificate.

As ZooKeeper does not have native support for TLS, a TLS sidecar deployment with ZooKeeper allows the Cluster Operator to provide data encryption and authentication between ZooKeeper nodes in a cluster.

Kafka listeners use authentication to ensure a secure client connection to the Kafka cluster.

Supported authentication mechanisms:

The User Operator manages user credentials for TLS and SCRAM authentication, but not OAuth 2.0. For example, through the User Operator you can create a user representing a client that requires access to the Kafka cluster, and specify TLS as the authentication type.

Using OAuth 2.0 token based authentication, application clients can access Kafka brokers without exposing account credentials. An authorization server handles the granting of access and inquiries about access.

You can apply authorization to a Kafka cluster using SimpleACLAuthorizer. If applied to a Kafka cluster, authorization is enabled for all listeners used for client connection.

SimpleACLAuthorizer uses Access Control Lists (ACLs) to define which users have access to which resources.

Super users provide user access to all resources in a Kafka cluster or to a specific broker.