Chapter 6. GenericKafkaListener schema reference
Used in: KafkaClusterSpec
Full list of GenericKafkaListener
schema properties
Configures listeners to connect to Kafka brokers within and outside OpenShift.
You configure the listeners in the Kafka
resource.
Example Kafka
resource showing listener configuration
apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata: name: my-cluster spec: kafka: #... listeners: - name: plain port: 9092 type: internal tls: false - name: tls port: 9093 type: internal tls: true authentication: type: tls - name: external1 port: 9094 type: route tls: true - name: external2 port: 9095 type: ingress tls: true authentication: type: tls configuration: bootstrap: host: bootstrap.myingress.com brokers: - broker: 0 host: broker-0.myingress.com - broker: 1 host: broker-1.myingress.com - broker: 2 host: broker-2.myingress.com #...
6.1. listeners
You configure Kafka broker listeners using the listeners
property in the Kafka
resource. Listeners are defined as an array.
Example listener configuration
listeners: - name: plain port: 9092 type: internal tls: false
The name and port must be unique within the Kafka cluster. By specifying a unique name and port for each listener, you can configure multiple listeners. The name can be up to 25 characters long, comprising lower-case letters and numbers.
6.2. port
The port number is the port used in the Kafka cluster, which might not be the same port used for access by a client.
-
loadbalancer
listeners use the specified port number, as dointernal
andcluster-ip
listeners -
ingress
androute
listeners use port 443 for access -
nodeport
listeners use the port number assigned by OpenShift
For client connection, use the address and port for the bootstrap service of the listener. You can retrieve this from the status of the Kafka
resource.
Example command to retrieve the address and port for client connection
oc get kafka <kafka_cluster_name> -o=jsonpath='{.status.listeners[?(@.name=="<listener_name>")].bootstrapServers}{"\n"}'
When configuring listeners for client access to brokers, you can use port 9092 or higher (9093, 9094, and so on), but with a few exceptions. The listeners cannot be configured to use the ports reserved for interbroker communication (9090 and 9091), Prometheus metrics (9404), and JMX (Java Management Extensions) monitoring (9999).
6.3. type
The type is set as internal
, or for external listeners, as route
, loadbalancer
, nodeport
, ingress
or cluster-ip
. You can also configure a cluster-ip
listener, a type of internal listener you can use to build custom access mechanisms.
- internal
You can configure internal listeners with or without encryption using the
tls
property.Example
internal
listener configuration#... spec: kafka: #... listeners: #... - name: plain port: 9092 type: internal tls: false - name: tls port: 9093 type: internal tls: true authentication: type: tls #...
- route
Configures an external listener to expose Kafka using OpenShift
Routes
and the HAProxy router.A dedicated
Route
is created for every Kafka broker pod. An additionalRoute
is created to serve as a Kafka bootstrap address. Kafka clients can use theseRoutes
to connect to Kafka on port 443. The client connects on port 443, the default router port, but traffic is then routed to the port you configure, which is9094
in this example.Example
route
listener configuration#... spec: kafka: #... listeners: #... - name: external1 port: 9094 type: route tls: true #...
- ingress
Configures an external listener to expose Kafka using Kubernetes
Ingress
and the Ingress NGINX Controller for Kubernetes.A dedicated
Ingress
resource is created for every Kafka broker pod. An additionalIngress
resource is created to serve as a Kafka bootstrap address. Kafka clients can use theseIngress
resources to connect to Kafka on port 443. The client connects on port 443, the default controller port, but traffic is then routed to the port you configure, which is9095
in the following example.You must specify the hostnames used by the bootstrap and per-broker services using
GenericKafkaListenerConfigurationBootstrap
andGenericKafkaListenerConfigurationBroker
properties.Example
ingress
listener configuration#... spec: kafka: #... listeners: #... - name: external2 port: 9095 type: ingress tls: true authentication: type: tls configuration: bootstrap: host: bootstrap.myingress.com brokers: - broker: 0 host: broker-0.myingress.com - broker: 1 host: broker-1.myingress.com - broker: 2 host: broker-2.myingress.com #...
NoteExternal listeners using
Ingress
are currently only tested with the Ingress NGINX Controller for Kubernetes.- loadbalancer
Configures an external listener to expose Kafka using a
Loadbalancer
typeService
.A new loadbalancer service is created for every Kafka broker pod. An additional loadbalancer is created to serve as a Kafka bootstrap address. Loadbalancers listen to the specified port number, which is port
9094
in the following example.You can use the
loadBalancerSourceRanges
property to configure source ranges to restrict access to the specified IP addresses.Example
loadbalancer
listener configuration#... spec: kafka: #... listeners: - name: external3 port: 9094 type: loadbalancer tls: true configuration: loadBalancerSourceRanges: - 10.0.0.0/8 - 88.208.76.87/32 #...
- nodeport
Configures an external listener to expose Kafka using a
NodePort
typeService
.Kafka clients connect directly to the nodes of OpenShift. An additional
NodePort
type of service is created to serve as a Kafka bootstrap address.When configuring the advertised addresses for the Kafka broker pods, Streams for Apache Kafka uses the address of the node on which the given pod is running. You can use
preferredNodePortAddressType
property to configure the first address type checked as the node address.Example
nodeport
listener configuration#... spec: kafka: #... listeners: #... - name: external4 port: 9095 type: nodeport tls: false configuration: preferredNodePortAddressType: InternalDNS #...
NoteTLS hostname verification is not currently supported when exposing Kafka clusters using node ports.
- cluster-ip
Configures an internal listener to expose Kafka using a per-broker
ClusterIP
typeService
.The listener does not use a headless service and its DNS names to route traffic to Kafka brokers. You can use this type of listener to expose a Kafka cluster when using the headless service is unsuitable. You might use it with a custom access mechanism, such as one that uses a specific Ingress controller or the OpenShift Gateway API.
A new
ClusterIP
service is created for each Kafka broker pod. The service is assigned aClusterIP
address to serve as a Kafka bootstrap address with a per-broker port number. For example, you can configure the listener to expose a Kafka cluster over an Nginx Ingress Controller with TCP port configuration.Example
cluster-ip
listener configuration#... spec: kafka: #... listeners: - name: clusterip type: cluster-ip tls: false port: 9096 #...
6.4. tls
The TLS property is required.
To enable TLS encryption, set the tls
property to true
. For route
and ingress
type listeners, TLS encryption must be always enabled.
6.5. authentication
Authentication for the listener can be specified as:
-
mTLS (
tls
) -
SCRAM-SHA-512 (
scram-sha-512
) -
Token-based OAuth 2.0 (
oauth
) -
Custom (
custom
)
6.6. networkPolicyPeers
Use networkPolicyPeers
to configure network policies that restrict access to a listener at the network level. The following example shows a networkPolicyPeers
configuration for a plain
and a tls
listener.
In the following example:
-
Only application pods matching the labels
app: kafka-sasl-consumer
andapp: kafka-sasl-producer
can connect to theplain
listener. The application pods must be running in the same namespace as the Kafka broker. -
Only application pods running in namespaces matching the labels
project: myproject
andproject: myproject2
can connect to thetls
listener.
The syntax of the networkPolicyPeers
property is the same as the from
property in NetworkPolicy
resources.
Example network policy configuration
listeners: #... - name: plain port: 9092 type: internal tls: true authentication: type: scram-sha-512 networkPolicyPeers: - podSelector: matchLabels: app: kafka-sasl-consumer - podSelector: matchLabels: app: kafka-sasl-producer - name: tls port: 9093 type: internal tls: true authentication: type: tls networkPolicyPeers: - namespaceSelector: matchLabels: project: myproject - namespaceSelector: matchLabels: project: myproject2 # ...
6.7. GenericKafkaListener
schema properties
Property | Property type | Description |
---|---|---|
name | string | Name of the listener. The name will be used to identify the listener and the related OpenShift objects. The name has to be unique within given a Kafka cluster. The name can consist of lowercase characters and numbers and be up to 11 characters long. |
port | integer | Port number used by the listener inside Kafka. The port number has to be unique within a given Kafka cluster. Allowed port numbers are 9092 and higher with the exception of ports 9404 and 9999, which are already used for Prometheus and JMX. Depending on the listener type, the port number might not be the same as the port number that connects Kafka clients. |
type | string (one of [ingress, internal, route, loadbalancer, cluster-ip, nodeport]) | Type of the listener. The supported types are as follows:
|
tls | boolean | Enables TLS encryption on the listener. This is a required property. |
authentication |
| Authentication configuration for this listener. |
configuration | Additional listener configuration. | |
networkPolicyPeers | NetworkPolicyPeer array | List of peers which should be able to connect to this listener. Peers in this list are combined using a logical OR operation. If this field is empty or missing, all connections will be allowed for this listener. If this field is present and contains at least one item, the listener only allows the traffic which matches at least one item in this list. |