Red Hat Summit 红帽全球峰会支持控制台（Console）开发人员开始试用联系人

第 15 章 Securing access to a Kafka cluster

Secure connections by configuring Kafka and Kafka users. Through configuration, you can implement encryption, authentication, and authorization mechanisms.

Kafka configuration

To establish secure access to Kafka, configure the Kafka resource to set up the following configurations based on your specific requirements:

  • Listeners with specified authentication types to define how clients authenticate

    • TLS encryption for communication between Kafka and clients
    • Supported TLS versions and cipher suites for additional security
  • Authorization for the entire Kafka cluster
  • Network policies for restricting access
  • Super users for unconstrained access to brokers

Authentication is configured independently for each listener, while authorization is set up for the whole Kafka cluster.

For more information on access configuration for Kafka, see the Kafka schema reference and GenericKafkaListener schema reference.

User (client-side) configuration

To enable secure client access to Kafka, configure KafkaUser resources. These resources represent clients and determine how they authenticate and authorize with the Kafka cluster.

Configure the KafkaUser resource to set up the following configurations based on your specific requirements:

  • Authentication that must match the enabled listener authentication

    • Supported TLS versions and cipher suites that must match the Kafka configuration

  • Simple authorization to apply Access Control List (ACL) rules

    • ACLs for fine-grained control over user access to topics and actions
  • Quotas to limit client access based on byte rates or CPU utilization

The User Operator creates the user representing the client and the security credentials used for client authentication, based on the chosen authentication type.

For more information on access configuration for users, see the KafkaUser schema reference.

15.1. Configuring client authentication on listeners
复制链接

Configure client authentication for Kafka brokers when creating listeners. Specify the listener authentication type using the Kafka.spec.kafka.listeners.authentication property in the Kafka resource.

For clients inside the OpenShift cluster, you can create plain (without encryption) or tls internal listeners. The internal listener type use a headless service and the DNS names given to the broker pods. As an alternative to the headless service, you can also create a cluster-ip type of internal listener to expose Kafka using per-broker ClusterIP services. For clients outside the OpenShift cluster, you create external listeners and specify a connection mechanism, which can be nodeport, loadbalancer, ingress (Kubernetes only), or route (OpenShift only).

For more information on the configuration options for connecting an external client, see 第 14 章 Setting up client access to a Kafka cluster.

Supported authentication options:

  1. mTLS authentication (only on the listeners with TLS enabled encryption)
  2. SCRAM-SHA-512 authentication
  3. OAuth 2.0 token-based authentication
  4. Custom authentication
  5. TLS versions and cipher suites

If you’re using OAuth 2.0 for client access management, user authentication and authorization credentials are handled through the authorization server.

The authentication option you choose depends on how you wish to authenticate client access to Kafka brokers.

注意

Try exploring the standard authentication options before using custom authentication. Custom authentication allows for any type of Kafka-supported authentication. It can provide more flexibility, but also adds complexity.

图 15.1. Kafka listener authentication options

options for listener authentication configuration

The listener authentication property is used to specify an authentication mechanism specific to that listener.

If no authentication property is specified then the listener does not authenticate clients which connect through that listener. The listener will accept all connections without authentication.

Authentication must be configured when using the User Operator to manage KafkaUsers.

The following example shows:

  • A plain listener configured for SCRAM-SHA-512 authentication
  • A tls listener with mTLS authentication
  • An external listener with mTLS authentication

Each listener is configured with a unique name and port within a Kafka cluster.

重要

When configuring listeners for client access to brokers, you can use port 9092 or higher (9093, 9094, and so on), but with a few exceptions. The listeners cannot be configured to use the ports reserved for interbroker communication (9090 and 9091), Prometheus metrics (9404), and JMX (Java Management Extensions) monitoring (9999).

Example listener authentication configuration

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
  namespace: myproject
spec:
  kafka:
    # ...
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: true
        authentication:
          type: scram-sha-512
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: tls
      - name: external3
        port: 9094
        type: loadbalancer
        tls: true
        authentication:
          type: tls
# ...

15.1.1. mTLS authentication
复制链接

mTLS authentication is always used for the communication between Kafka brokers and ZooKeeper pods.

Streams for Apache Kafka can configure Kafka to use TLS (Transport Layer Security) to provide encrypted communication between Kafka brokers and clients either with or without mutual authentication. For mutual, or two-way, authentication, both the server and the client present certificates. When you configure mTLS authentication, the broker authenticates the client (client authentication) and the client authenticates the broker (server authentication).

mTLS listener configuration in the Kafka resource requires the following:

  • tls: true to specify TLS encryption and server authentication
  • authentication.type: tls to specify the client authentication

When a Kafka cluster is created by the Cluster Operator, it creates a new secret with the name <cluster_name>-cluster-ca-cert. The secret contains a CA certificate. The CA certificate is in PEM and PKCS #12 format. To verify a Kafka cluster, add the CA certificate to the truststore in your client configuration. To verify a client, add a user certificate and key to the keystore in your client configuration. For more information on configuring a client for mTLS, see 第 15.3.2 节 “Configuring user authentication”.

注意

TLS authentication is more commonly one-way, with one party authenticating the identity of another. For example, when HTTPS is used between a web browser and a web server, the browser obtains proof of the identity of the web server.

15.1.2. SCRAM-SHA-512 authentication
复制链接

SCRAM (Salted Challenge Response Authentication Mechanism) is an authentication protocol that can establish mutual authentication using passwords. Streams for Apache Kafka can configure Kafka to use SASL (Simple Authentication and Security Layer) SCRAM-SHA-512 to provide authentication on both unencrypted and encrypted client connections.

When SCRAM-SHA-512 authentication is used with a TLS connection, the TLS protocol provides the encryption, but is not used for authentication.

The following properties of SCRAM make it safe to use SCRAM-SHA-512 even on unencrypted connections:

  • The passwords are not sent in the clear over the communication channel. Instead the client and the server are each challenged by the other to offer proof that they know the password of the authenticating user.
  • The server and client each generate a new challenge for each authentication exchange. This means that the exchange is resilient against replay attacks.

When KafkaUser.spec.authentication.type is configured with scram-sha-512 the User Operator will generate a random 32-character password consisting of upper and lowercase ASCII letters and numbers.

Control listener access by configuring the networkPolicyPeers property in the Kafka resource.

By default, Streams for Apache Kafka automatically creates a NetworkPolicy resource for every enabled Kafka listener, allowing connections from all namespaces.

To restrict listener access to specific applications or namespaces at the network level, configure the networkPolicyPeers property. Each listener can have its own networkPolicyPeers configuration. For more information on network policy peers, refer to the NetworkPolicyPeer API reference.

If you want to use custom network policies, you can set the STRIMZI_NETWORK_POLICY_GENERATION environment variable to false in the Cluster Operator configuration. For more information, see 第 9.6 节 “Configuring the Cluster Operator”.

注意

Your configuration of OpenShift must support ingress NetworkPolicies in order to use network policies.

Prerequisites

  • An OpenShift cluster with support for Ingress NetworkPolicies.
  • The Cluster Operator is running.

Procedure

  1. Configure the networkPolicyPeers property to define the application pods or namespaces allowed to access the Kafka cluster.

    This example shows configuration for a tls listener to allow connections only from application pods with the label app set to kafka-client: 

    apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
spec:
  kafka:
    # ...
    listeners:
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: tls
        networkPolicyPeers:
          - podSelector:
              matchLabels:
                app: kafka-client
    # ...
  zookeeper:
    # ...
  2. Apply the changes to the Kafka resource configuration.

This procedure shows how to configure custom server certificates for TLS listeners or external listeners which have TLS encryption enabled.

By default, Kafka listeners use certificates signed by Streams for Apache Kafka’s internal CA (certificate authority). The Cluster Operator automatically generates a CA certificate when creating a Kafka cluster. To configure a client for TLS, the CA certificate is included in its truststore configuration to authenticate the Kafka cluster. Alternatively, you have the option to install and use your own CA certificates.

However, if you prefer more granular control by using your own custom certificates at the listener-level, you can configure listeners using brokerCertChainAndKey properties. You create a secret with your own private key and server certificate, then specify them in the brokerCertChainAndKey configuration.

User-provided certificates allow you to leverage existing security infrastructure. You can use a certificate signed by a public (external) CA or a private CA. Kafka clients need to trust the CA which was used to sign the listener certificate. If signed by a public CA, you usually won’t need to add it to a client’s truststore configuration.

Custom certificates are not managed by Streams for Apache Kafka, so you need to renew them manually.

注意

Listener certificates are used for TLS encryption and server authentication only. They are not used for TLS client authentication. If you want to use your own certificate for TLS client authentication as well, you must install and use your own clients CA.

Prerequisites

  • The Cluster Operator is running.

  • Each listener requires the following:

If you are not using a self-signed certificate, you can provide a certificate that includes the whole CA chain in the certificate.

You can only use the brokerCertChainAndKey properties if TLS encryption (tls: true) is configured for the listener.

注意

Streams for Apache Kafka does not support the use of encrypted private keys for TLS. The private key stored in the secret must be unencrypted for this to work.

Procedure

  1. Create a Secret containing your private key and server certificate: 

    oc create secret generic <my_secret> --from-file=<my_listener_key.key> --from-file=<my_listener_certificate.crt>

  2. Edit the Kafka resource for your cluster.

    Configure the listener to use your Secret, certificate file, and private key file in the configuration.brokerCertChainAndKey property.

    Example configuration for a loadbalancer external listener with TLS encryption enabled

    # ...
listeners:
  - name: plain
    port: 9092
    type: internal
    tls: false
  - name: external3
    port: 9094
    type: loadbalancer
    tls: true
    configuration:
      brokerCertChainAndKey:
        secretName: my-secret
        certificate: my-listener-certificate.crt
        key: my-listener-key.key
# ...

    Example configuration for a TLS listener

    # ...
listeners:
  - name: plain
    port: 9092
    type: internal
    tls: false
  - name: tls
    port: 9093
    type: internal
    tls: true
    configuration:
      brokerCertChainAndKey:
        secretName: my-secret
        certificate: my-listener-certificate.crt
        key: my-listener-key.key
# ...

  3. Apply the changes to the Kafka resource configuration.

    The Cluster Operator starts a rolling update of the Kafka cluster, which updates the configuration of the listeners.

    注意

    A rolling update is also started if you update a Kafka listener certificate in a Secret that is already used by a listener.

15.1.5. Specifying SANs for custom listener certificates
复制链接

In order to use TLS hostname verification with custom Kafka listener certificates, you must specify the correct Subject Alternative Names (SANs) for each listener.

The certificate SANs must specify hostnames for the following:

  • All of the Kafka brokers in your cluster
  • The Kafka cluster bootstrap service

You can use wildcard certificates if they are supported by your CA.

15.1.5.1. Examples of SANs for internal listeners
复制链接

Use the following examples to help you specify hostnames of the SANs in your certificates for your internal listeners.

Replace <cluster-name> with the name of the Kafka cluster and <namespace> with the OpenShift namespace where the cluster is running.

Wildcards example for a type: internal listener

//Kafka brokers
*.<cluster_name>-kafka-brokers
*.<cluster_name>-kafka-brokers.<namespace>.svc

// Bootstrap service
<cluster_name>-kafka-bootstrap
<cluster_name>-kafka-bootstrap.<namespace>.svc

Non-wildcards example for a type: internal listener

// Kafka brokers
<cluster_name>-kafka-0.<cluster_name>-kafka-brokers
<cluster_name>-kafka-0.<cluster_name>-kafka-brokers.<namespace>.svc
<cluster_name>-kafka-1.<cluster_name>-kafka-brokers
<cluster_name>-kafka-1.<cluster_name>-kafka-brokers.<namespace>.svc
# ...

// Bootstrap service
<cluster_name>-kafka-bootstrap
<cluster_name>-kafka-bootstrap.<namespace>.svc

Non-wildcards example for a type: cluster-ip listener

// Kafka brokers
<cluster_name>-kafka-<listener-name>-0
<cluster_name>-kafka-<listener-name>-0.<namespace>.svc
<cluster_name>-kafka-_listener-name>-1
<cluster_name>-kafka-<listener-name>-1.<namespace>.svc
# ...

// Bootstrap service
<cluster_name>-kafka-<listener-name>-bootstrap
<cluster_name>-kafka-<listener-name>-bootstrap.<namespace>.svc

15.1.5.2. Examples of SANs for external listeners
复制链接

For external listeners which have TLS encryption enabled, the hostnames you need to specify in certificates depends on the external listener type.

Expand
表 15.1. SANs for each type of external listener
External listener typeIn the SANs, specify…​

ingress

Addresses of all Kafka broker Ingress resources and the address of the bootstrap Ingress.

You can use a matching wildcard name.

route

Addresses of all Kafka broker Routes and the address of the bootstrap Route.

You can use a matching wildcard name.

loadbalancer

Addresses of all Kafka broker loadbalancers and the bootstrap loadbalancer address.

You can use a matching wildcard name.

nodeport

Addresses of all OpenShift worker nodes that the Kafka broker pods might be scheduled to.

You can use a matching wildcard name.

Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

Legal Notice

Theme

© 2026 Red Hat返回顶部