支持控制台（Console）开发人员开始试用联系人

9.8. Configuring Kafka Connect connectors

The KafkaConnector resource provides a Kubernetes-native approach to management of connectors by the Cluster Operator. To create, delete, or reconfigure connectors with KafkaConnector resources, you must set the use-connector-resources annotation to true in your KafkaConnect custom resource.

Annotation to enable KafkaConnectors

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect-cluster
  annotations:
    strimzi.io/use-connector-resources: "true"
  # ...

When the use-connector-resources annotation is enabled in your KafkaConnect configuration, you must define and manage connectors using KafkaConnector resources.

注意

Alternatively, you can manage connectors using the Kafka Connect REST API instead of KafkaConnector resources. To use the API, you must remove the strimzi.io/use-connector-resources annotation to use KafkaConnector resources in the KafkaConnect the resource.

KafkaConnector resources provide the configuration needed to create connectors within a Kafka Connect cluster, which interacts with a Kafka cluster as specified in the KafkaConnect configuration. The Kafka cluster does not need to be managed by Streams for Apache Kafka or deployed to a Kubernetes cluster.

Kafka components contained in the same Kubernetes cluster

Kafka and Kafka Connect clusters

The configuration also specifies how the connector instances interact with external data systems, including any required authentication methods. Additionally, you must define the data to watch. For example, in a source connector that reads data from a database, the configuration might include the database name. You can also define where this data should be placed in Kafka by specifying the target topic name.

Use the tasksMax property to specify the maximum number of tasks. For instance, a source connector with tasksMax: 2 might split the import of source data into two tasks.

Example source connector configuration

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: my-source-connector
1 

  labels:
    strimzi.io/cluster: my-connect-cluster
2 

spec:
  class: org.apache.kafka.connect.file.FileStreamSourceConnector
3 

  tasksMax: 2
4 

  autoRestart:
5 

    enabled: true
  config:
6 

    file: "/opt/kafka/LICENSE"
7 

    topic: my-topic
8 

    # ...

1
Name of the KafkaConnector resource, which is used as the name of the connector. Use any name that is valid for an OpenShift resource.
2
Name of the Kafka Connect cluster to create the connector instance in. Connectors must be deployed to the same namespace as the Kafka Connect cluster they link to.
3
Full name of the connector class. This should be present in the image being used by the Kafka Connect cluster.
4
Maximum number of Kafka Connect tasks that the connector can create.
5
Enables automatic restarts of failed connectors and tasks. By default, the number of restarts is indefinite, but you can set a maximum on the number of automatic restarts using the maxRestarts property.
6
Connector configuration as key-value pairs.
7
Location of the external data file. In this example, we’re configuring the FileStreamSourceConnector to read from the /opt/kafka/LICENSE file.
8
Kafka topic to publish the source data to.

To include external connector configurations, such as user access credentials stored in a secret, use the template property of the KafkaConnect resource. You can also load values using configuration providers.

If you are using KafkaConnector resources to configure connectors, use the state configuration to either stop or pause a connector. In contrast to the paused state, where the connector and tasks remain instantiated, stopping a connector retains only the configuration, with no active processes. Stopping a connector from running may be more suitable for longer durations than just pausing. While a paused connector is quicker to resume, a stopped connector has the advantages of freeing up memory and resources.

注意

The state configuration replaces the (deprecated) pause configuration in the KafkaConnectorSpec schema, which allows pauses on connectors. If you were previously using the pause configuration to pause connectors, we encourage you to transition to using the state configuration only to avoid conflicts.

Prerequisites

  • The Cluster Operator is running.

Procedure

  1. Find the name of the KafkaConnector custom resource that controls the connector you want to pause or stop: 

    oc get KafkaConnector

  2. Edit the KafkaConnector resource to stop or pause the connector.

    Example configuration for stopping a Kafka Connect connector

    apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: my-source-connector
  labels:
    strimzi.io/cluster: my-connect-cluster
spec:
  class: org.apache.kafka.connect.file.FileStreamSourceConnector
  tasksMax: 2
  config:
    file: "/opt/kafka/LICENSE"
    topic: my-topic
  state: stopped
  # ...

    Change the state configuration to stopped or paused. The default state for the connector when this property is not set is running.

  3. Apply the changes to the KafkaConnector configuration.

    You can resume the connector by changing state to running or removing the configuration.

注意

Alternatively, you can expose the Kafka Connect API and use the stop and pause endpoints to stop a connector from running. For example, PUT /connectors/<connector_name>/stop. You can then use the resume endpoint to restart it.

9.8.2. Manually restarting Kafka Connect connectors
复制链接

If you are using KafkaConnector resources to manage connectors, use the strimzi.io/restart annotation to manually trigger a restart of a connector.

Prerequisites

  • The Cluster Operator is running.

Procedure

  1. Find the name of the KafkaConnector custom resource that controls the Kafka connector you want to restart: 

    oc get KafkaConnector

  2. Restart the connector by annotating the KafkaConnector resource in OpenShift: 

    oc annotate KafkaConnector <kafka_connector_name> strimzi.io/restart="true"

    The restart annotation is set to true.

  3. Wait for the next reconciliation to occur (every two minutes by default).

    The Kafka connector is restarted, as long as the annotation was detected by the reconciliation process. When Kafka Connect accepts the restart request, the annotation is removed from the KafkaConnector custom resource.

9.8.3. Manually restarting Kafka Connect connector tasks
复制链接

If you are using KafkaConnector resources to manage connectors, use the strimzi.io/restart-task annotation to manually trigger a restart of a connector task.

Prerequisites

  • The Cluster Operator is running.

Procedure

  1. Find the name of the KafkaConnector custom resource that controls the Kafka connector task you want to restart: 

    oc get KafkaConnector

  2. Find the ID of the task to be restarted from the KafkaConnector custom resource: 

    oc describe KafkaConnector <kafka_connector_name>

    Task IDs are non-negative integers, starting from 0.

  3. Use the ID to restart the connector task by annotating the KafkaConnector resource in OpenShift: 

    oc annotate KafkaConnector <kafka_connector_name> strimzi.io/restart-task="0"

    In this example, task 0 is restarted.

  4. Wait for the next reconciliation to occur (every two minutes by default).

    The Kafka connector task is restarted, as long as the annotation was detected by the reconciliation process. When Kafka Connect accepts the restart request, the annotation is removed from the KafkaConnector custom resource.

Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

Legal Notice

Theme

© 2026 Red Hat返回顶部