支持控制台（Console）开发人员开始试用联系人

6.3. Authentication

To authenticate client connections to your Kafka cluster, the following options are available:

TLS client authentication
TLS (Transport Layer Security) using X.509 certificates on encrypted connections
Kafka SASL
Kafka SASL (Simple Authentication and Security Layer) using supported authentication mechanisms
OAuth 2.0
OAuth 2.0 token-based authentication

SASL authentication supports various mechanisms for both plain unencrypted connections and TLS connections:

  • PLAIN ― Authentication based on usernames and passwords.
  • SCRAM-SHA-256 and SCRAM-SHA-512 ― Authentication using Salted Challenge Response Authentication Mechanism (SCRAM).
  • GSSAPI ― Authentication against a Kerberos server.
警告

The PLAIN mechanism sends usernames and passwords over the network in an unencrypted format. It should only be used in combination with TLS encryption.

6.3.1. Enabling TLS client authentication
复制链接

Enable TLS client authentication in Kafka brokers to enhance security for connections to Kafka nodes already using TLS encryption.

Use the ssl.client.auth property to set TLS authentication with one of these values:

  • none ― TLS client authentication is off (default)
  • requested ― Optional TLS client authentication
  • required ― Clients must authenticate using a TLS client certificate

When a client authenticates using TLS client authentication, the authenticated principal name is derived from the distinguished name in the client certificate. For instance, a user with a certificate having a distinguished name CN=someuser will be authenticated with the principal CN=someuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown. This principal name provides a unique identifier for the authenticated user or entity. When TLS client authentication is not used, and SASL is disabled, the principal name defaults to ANONYMOUS.

Prerequisites

Procedure

  1. Prepare a JKS (Java Keystore ) truststore containing the public key of the CA (Certification Authority) used to sign the user certificates.

  2. Edit the Kafka configuration properties file on all cluster nodes as follows:

    • Specify the path to the JKS truststore using the ssl.truststore.location property.
    • If the truststore is password-protected, set the password using ssl.truststore.password property.

    • Set the ssl.client.auth property to required.

      TLS client authentication configuration

      ssl.truststore.location=/path/to/truststore.jks
ssl.truststore.password=123456
ssl.client.auth=required

  3. (Re)start the Kafka brokers.

6.3.2. Enabling SASL PLAIN client authentication
复制链接

Enable SASL PLAIN authentication in Kafka to enhance security for connections to Kafka nodes.

SASL authentication is enabled through the Java Authentication and Authorization Service (JAAS) using the KafkaServer JAAS context. You can define the JAAS configuration in a dedicated file or directly in the Kafka configuration.

The recommended location for the dedicated file is ./config/jaas.conf. Ensure that the file is readable by the Kafka user. Keep the JAAS configuration file in sync on all Kafka nodes.

Prerequisites

Procedure

  1. Edit or create the ./config/jaas.conf JAAS configuration file to enable the PlainLoginModule and specify the allowed usernames and passwords.

    Make sure this file is the same on all Kafka brokers.

    JAAS configuration

    KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    user_admin="123456"
    user_user1="123456"
    user_user2="123456";
};

  2. Edit the Kafka configuration properties file on all cluster nodes as follows:

    • Enable SASL PLAIN authentication on specific listeners using the listener.security.protocol.map property. Specify SASL_PLAINTEXT or SASL_SSL.

    • Set the sasl.enabled.mechanisms property to PLAIN.

      SASL plain configuration

      listeners=INSECURE://:9092,AUTHENTICATED://:9093,REPLICATION://:9094
listener.security.protocol.map=INSECURE:PLAINTEXT,AUTHENTICATED:SASL_PLAINTEXT,REPLICATION:PLAINTEXT
sasl.enabled.mechanisms=PLAIN

  3. (Re)start the Kafka brokers using the KAFKA_OPTS environment variable to pass the JAAS configuration to Kafka brokers: 

    export KAFKA_OPTS="-Djava.security.auth.login.config=./config/jaas.conf"; ./bin/kafka-server-start.sh -daemon ./config/server.properties

6.3.3. Enabling SASL SCRAM client authentication
复制链接

Enable SASL SCRAM authentication in Kafka to enhance security for connections to Kafka nodes.

SASL authentication is enabled through the Java Authentication and Authorization Service (JAAS) using the KafkaServer JAAS context. You can define the JAAS configuration in a dedicated file or directly in the Kafka configuration.

The recommended location for the dedicated file is ./config/jaas.conf. Ensure that the file is readable by the Kafka user. Keep the JAAS configuration file in sync on all Kafka nodes.

Prerequisites

Procedure

  1. Edit or create the ./config/jaas.conf JAAS configuration file to enable the ScramLoginModule.

    Make sure this file is the same on all Kafka brokers.

    JAAS configuration

    KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required;
};

  2. Edit the Kafka configuration properties file on all cluster nodes as follows:

    • Enable SASL SCRAM authentication on specific listeners using the listener.security.protocol.map property. Specify SASL_PLAINTEXT or SASL_SSL.

    • Set the sasl.enabled.mechanisms option to SCRAM-SHA-256 or SCRAM-SHA-512.

      For example: 

      listeners=INSECURE://:9092,AUTHENTICATED://:9093,REPLICATION://:9094
listener.security.protocol.map=INSECURE:PLAINTEXT,AUTHENTICATED:SASL_PLAINTEXT,REPLICATION:PLAINTEXT
sasl.enabled.mechanisms=SCRAM-SHA-512

  3. (Re)start the Kafka brokers using the KAFKA_OPTS environment variable to pass the JAAS configuration to Kafka brokers. 

    export KAFKA_OPTS="-Djava.security.auth.login.config=./config/jaas.conf"; ./bin/kafka-server-start.sh -daemon ./config/server.properties

6.3.4. Enabling multiple SASL mechanisms
复制链接

When using SASL authentication, you can enable more than one mechanism. Kafka can use more than one SASL mechanism simultaneously. When multiple mechanisms are enabled, you can choose the mechanism specific clients use.

To use more than one mechanism, you set up the configuration required for each mechanism. You can add different KafkaServer JAAS configurations to the same context and enable more than one mechanism in the Kafka configuration as a comma-separated list using the sasl.mechanism.inter.broker.protocol property.

JAAS configuration for more than one SASL mechanism

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    user_admin="123456"
    user_user1="123456"
    user_user2="123456";

    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    keyTab="/etc/security/keytabs/kafka_server.keytab"
    principal="kafka/kafka1.hostname.com@EXAMPLE.COM";

    org.apache.kafka.common.security.scram.ScramLoginModule required;
};

SASL mechanisms enabled

sasl.enabled.mechanisms=PLAIN,SCRAM-SHA-256,SCRAM-SHA-512

6.3.5. Enabling SASL for inter-broker authentication
复制链接

Enable SASL SCRAM authentication between Kafka nodes to enhance security for inter-broker connections. As well as using SASL authentication for client connections to a Kafka cluster, you can also use SASL for inter-broker authentication. Unlike SASL for client connections, you can only choose one mechanism for inter-broker communication.

Prerequisites

  • ZooKeeper is installed on each host, and the configuration files are available.

  • If you are using a SCRAM mechanism, register SCRAM credentials on the Kafka cluster.

    For all nodes in the Kafka cluster, add the inter-broker SASL SCRAM user to ZooKeeper. This ensures that the credentials for authentication are updated for bootstrapping before the Kafka cluster is running.

    Registering an inter-broker SASL SCRAM user

    bin/kafka-configs.sh \
--zookeeper localhost:2181 \
--alter \
--add-config 'SCRAM-SHA-512=[password=changeit]' \
--entity-type users \
--entity-name kafka

Procedure

  1. Specify an inter-broker SASL mechanism in the Kafka configuration using the sasl.mechanism.inter.broker.protocol property.

    Inter-broker SASL mechanism

    sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512

  2. (Optional) If you are using a SCRAM mechanism, register SCRAM credentials on the Kafka cluster by adding SCRAM users.

    This ensures that the credentials for authentication are updated for bootstrapping before the Kafka cluster is running.

  3. Specify the username and password for inter-broker communication in the KafkaServer JAAS context using the username and password fields.

    Inter-broker JAAS context

    KafkaServer {
    org.apache.kafka.common.security.plain.ScramLoginModule required
    username="admin"
    password="123456"
    # ...
};

6.3.6. Adding SASL SCRAM users
复制链接

This procedure outlines the steps to register new users for authentication using SASL SCRAM in Kafka. SASL SCRAM authentication enhances the security of client connections.

Prerequisites

Procedure

  • Use the kafka-configs.sh tool to add new SASL SCRAM users. 

    ./bin/kafka-configs.sh \
--bootstrap-server <broker_host>:<port> \
--alter \
--add-config 'SCRAM-SHA-512=[password=<password>]' \
--entity-type users --entity-name <username>

    For example: 

    ./bin/kafka-configs.sh \
--bootstrap-server localhost:9092 \
--alter \
--add-config 'SCRAM-SHA-512=[password=123456]' \
--entity-type users \
--entity-name user1

6.3.7. Deleting SASL SCRAM users
复制链接

This procedure outlines the steps to remove users registered for authentication using SASL SCRAM in Kafka.

Prerequisites

Procedure

  • Use the kafka-configs.sh tool to delete SASL SCRAM users.

    /bin/kafka-configs.sh \

    --bootstrap-server <broker_host>:<port> \
--alter \
--delete-config 'SCRAM-SHA-512' \
--entity-type users \
--entity-name <username>

    For example:

    /bin/kafka-configs.sh \

    --bootstrap-server localhost:9092 \
--alter \
--delete-config 'SCRAM-SHA-512' \
--entity-type users \
--entity-name user1

6.3.8. Enabling Kerberos (GSSAPI) authentication
复制链接

Streams for Apache Kafka supports the use of the Kerberos (GSSAPI) authentication protocol for secure single sign-on access to your Kafka cluster. GSSAPI is an API wrapper for Kerberos functionality, insulating applications from underlying implementation changes.

Kerberos is a network authentication system that allows clients and servers to authenticate to each other by using symmetric encryption and a trusted third party, the Kerberos Key Distribution Centre (KDC).

This procedure shows how to configure Streams for Apache Kafka so that Kafka clients can access Kafka and ZooKeeper using Kerberos (GSSAPI) authentication. For this setup, Kafka is installed in the /opt/kafka/ directory.

The procedure assumes that a Kerberos krb5 resource server has been set up on a Red Hat Enterprise Linux host.

The procedure shows, with examples, how to configure:

  1. Service principals
  2. Kafka brokers to use the Kerberos login
  3. ZooKeeper to use Kerberos login
  4. Producer and consumer clients to access Kafka using Kerberos authentication

The instructions describe Kerberos set up for a single ZooKeeper and Kafka installation on a single host, with additional configuration for a producer and consumer client.

Prerequisites

  • You are logged in to Red Hat Enterprise Linux as the Kafka user.

To be able to configure Kafka and ZooKeeper to authenticate and authorize Kerberos credentials, you will need:

  • Access to a Kerberos server
  • A Kerberos client on each Kafka broker host

For more information on the steps to set up a Kerberos server, and clients on broker hosts, see the example Kerberos on RHEL set up configuration.

Add service principals for authentication

From your Kerberos server, create service principals (users) for ZooKeeper, Kafka brokers, and Kafka producer and consumer clients.

Service principals must take the form SERVICE-NAME/FULLY-QUALIFIED-HOST-NAME@DOMAIN-REALM.

  1. Create the service principals, and keytabs that store the principal keys, through the Kerberos KDC.

    Make sure the domain name in the Kerberos principal is in uppercase.

    For example:

    • zookeeper/node1.example.redhat.com@EXAMPLE.REDHAT.COM
    • kafka/node1.example.redhat.com@EXAMPLE.REDHAT.COM
    • producer1/node1.example.redhat.com@EXAMPLE.REDHAT.COM

    • consumer1/node1.example.redhat.com@EXAMPLE.REDHAT.COM

      The ZooKeeper service principal must have the same hostname as the zookeeper.connect configuration in the Kafka config/server.properties file: 

      zookeeper.connect=node1.example.redhat.com:2181

      If the hostname is not the same, localhost is used and authentication will fail.

  2. Create a directory on the host and add the keytab files:

    For example: 

    /opt/kafka/krb5/zookeeper-node1.keytab
/opt/kafka/krb5/kafka-node1.keytab
/opt/kafka/krb5/kafka-producer1.keytab
/opt/kafka/krb5/kafka-consumer1.keytab

Configure ZooKeeper to use a Kerberos Login

Configure ZooKeeper to use the Kerberos Key Distribution Center (KDC) for authentication using the user principals and keytabs previously created for zookeeper.

  1. Create or modify the opt/kafka/config/jaas.conf file to support ZooKeeper client and server operations: 

    Client {
    com.sun.security.auth.module.Krb5LoginModule required debug=true
    useKeyTab=true
    1 
    
    storeKey=true
    2 
    
    useTicketCache=false
    3 
    
    keyTab="/opt/kafka/krb5/zookeeper-node1.keytab"
    4 
    
    principal="zookeeper/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
    5 
    
};

Server {
    com.sun.security.auth.module.Krb5LoginModule required debug=true
    useKeyTab=true
    storeKey=true
    useTicketCache=false
    keyTab="/opt/kafka/krb5/zookeeper-node1.keytab"
    principal="zookeeper/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
};

QuorumServer {
    com.sun.security.auth.module.Krb5LoginModule required debug=true
    useKeyTab=true
    storeKey=true
    keyTab="/opt/kafka/krb5/zookeeper-node1.keytab"
    principal="zookeeper/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
};

QuorumLearner {
    com.sun.security.auth.module.Krb5LoginModule required debug=true
    useKeyTab=true
    storeKey=true
    keyTab="/opt/kafka/krb5/zookeeper-node1.keytab"
    principal="zookeeper/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
};
    1
    Set to true to get the principal key from the keytab.
    2
    Set to true to store the principal key.
    3
    Set to true to obtain the Ticket Granting Ticket (TGT) from the ticket cache.
    4
    The keyTab property points to the location of the keytab file copied from the Kerberos KDC. The location and file must be readable by the Kafka user.
    5
    The principal property is configured to match the fully-qualified principal name created on the KDC host, which follows the format SERVICE-NAME/FULLY-QUALIFIED-HOST-NAME@DOMAIN-NAME.

  2. Edit opt/kafka/config/zookeeper.properties to use the updated JAAS configuration: 

    # ...

requireClientAuthScheme=sasl
jaasLoginRenew=3600000
    1 
    
kerberos.removeHostFromPrincipal=false
    2 
    
kerberos.removeRealmFromPrincipal=false
    3 
    
quorum.auth.enableSasl=true
    4 
    
quorum.auth.learnerRequireSasl=true
    5 
    
quorum.auth.serverRequireSasl=true
quorum.auth.learner.loginContext=QuorumLearner
    6 
    
quorum.auth.server.loginContext=QuorumServer
quorum.auth.kerberos.servicePrincipal=zookeeper/_HOST
    7 
    
quorum.cnxn.threads.size=20
    1
    Controls the frequency for login renewal in milliseconds, which can be adjusted to suit ticket renewal intervals. Default is one hour.
    2
    Dictates whether the hostname is used as part of the login principal name. If using a single keytab for all nodes in the cluster, this is set to true. However, it is recommended to generate a separate keytab and fully-qualified principal for each broker host for troubleshooting.
    3
    Controls whether the realm name is stripped from the principal name for Kerberos negotiations. It is recommended that this setting is set as false.
    4
    Enables SASL authentication mechanisms for the ZooKeeper server and client.
    5
    The RequireSasl properties controls whether SASL authentication is required for quorum events, such as master elections.
    6
    The loginContext properties identify the name of the login context in the JAAS configuration used for authentication configuration of the specified component. The loginContext names correspond to the names of the relevant sections in the opt/kafka/config/jaas.conf file.
    7
    Controls the naming convention to be used to form the principal name used for identification. The placeholder _HOST is automatically resolved to the hostnames defined by the server.1 properties at runtime.

  3. Start ZooKeeper with JVM parameters to specify the Kerberos login configuration: 

    export EXTRA_ARGS="-Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/opt/kafka/config/jaas.conf"; /opt/kafka/bin/zookeeper-server-start.sh -daemon /opt/kafka/config/zookeeper.properties

    If you are not using the default service name (zookeeper), add the name using the -Dzookeeper.sasl.client.username=NAME parameter.

    注意

    If you are using the /etc/krb5.conf location, you do not need to specify -Djava.security.krb5.conf=/etc/krb5.conf when starting ZooKeeper, Kafka, or the Kafka producer and consumer.

Configure the Kafka broker server to use a Kerberos login

Configure Kafka to use the Kerberos Key Distribution Center (KDC) for authentication using the user principals and keytabs previously created for kafka.

  1. Modify the opt/kafka/config/jaas.conf file with the following elements: 

    KafkaServer {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    keyTab="/opt/kafka/krb5/kafka-node1.keytab"
    principal="kafka/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
};
KafkaClient {
    com.sun.security.auth.module.Krb5LoginModule required debug=true
    useKeyTab=true
    storeKey=true
    useTicketCache=false
    keyTab="/opt/kafka/krb5/kafka-node1.keytab"
    principal="kafka/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
};

  2. Configure each broker in the Kafka cluster by modifying the listener configuration in the config/server.properties file so the listeners use the SASL/GSSAPI login.

    Add the SASL protocol to the map of security protocols for the listener, and remove any unwanted protocols.

    For example: 

    # ...
broker.id=0
# ...
listeners=SECURE://:9092,REPLICATION://:9094
    1 
    
inter.broker.listener.name=REPLICATION
# ...
listener.security.protocol.map=SECURE:SASL_PLAINTEXT,REPLICATION:SASL_PLAINTEXT
    2 
    
# ..
sasl.enabled.mechanisms=GSSAPI
    3 
    
sasl.mechanism.inter.broker.protocol=GSSAPI
    4 
    
sasl.kerberos.service.name=kafka
    5 
    
...
    1
    Two listeners are configured: a secure listener for general-purpose communications with clients (supporting TLS for communications), and a replication listener for inter-broker communications.
    2
    For TLS-enabled listeners, the protocol name is SASL_PLAINTEXT. For non-TLS-enabled connectors, the protocol name is SASL_PLAINTEXT. If SSL is not required, you can remove the ssl.* properties.
    3
    SASL mechanism for Kerberos authentication is GSSAPI.
    4
    Kerberos authentication for inter-broker communication.
    5
    The name of the service used for authentication requests is specified to distinguish it from other services that may also be using the same Kerberos configuration.

  3. Start the Kafka broker, with JVM parameters to specify the Kerberos login configuration: 

    export KAFKA_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/opt/kafka/config/jaas.conf"; /opt/kafka/bin/kafka-server-start.sh -daemon /opt/kafka/config/server.properties

    If the broker and ZooKeeper cluster were previously configured and working with a non-Kerberos-based authentication system, it is possible to start the ZooKeeper and broker cluster and check for configuration errors in the logs.

    After starting the broker and Zookeeper instances, the cluster is now configured for Kerberos authentication.

Configure Kafka producer and consumer clients to use Kerberos authentication

Configure Kafka producer and consumer clients to use the Kerberos Key Distribution Center (KDC) for authentication using the user principals and keytabs previously created for producer1 and consumer1.

  1. Add the Kerberos configuration to the producer or consumer configuration file.

    For example:

    Configuration in producer.properties

    # ...
sasl.mechanism=GSSAPI
    1 
    
security.protocol=SASL_PLAINTEXT
    2 
    
sasl.kerberos.service.name=kafka
    3 
    
sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    4 
    
    useKeyTab=true  \
    useTicketCache=false \
    storeKey=true  \
    keyTab="/opt/kafka/krb5/producer1.keytab" \
    principal="producer1/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
# ...

    1
    Configuration for Kerberos (GSSAPI) authentication.
    2
    Kerberos uses the SASL plaintext (username/password) security protocol.
    3
    The service principal (user) for Kafka that was configured in the Kerberos KDC.
    4
    Configuration for the JAAS using the same properties defined in jaas.conf.

    Configuration in consumer.properties

    # ...
sasl.mechanism=GSSAPI
security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka
sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    useKeyTab=true  \
    useTicketCache=false \
    storeKey=true  \
    keyTab="/opt/kafka/krb5/consumer1.keytab" \
    principal="consumer1/node1.example.redhat.com@EXAMPLE.REDHAT.COM";
# ...

  2. Run the clients to verify that you can send and receive messages from the Kafka brokers.

    Producer client: 

    export KAFKA_HEAP_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true"; /opt/kafka/bin/kafka-console-producer.sh --producer.config /opt/kafka/config/producer.properties  --topic topic1 --bootstrap-server node1.example.redhat.com:9094

    Consumer client: 

    export KAFKA_HEAP_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true"; /opt/kafka/bin/kafka-console-consumer.sh --consumer.config /opt/kafka/config/consumer.properties  --topic topic1 --bootstrap-server node1.example.redhat.com:9094
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

Legal Notice

Theme

© 2026 Red Hat返回顶部