Red Hat Summit 红帽全球峰会支持控制台（Console）开发人员开始试用联系人

此内容没有您所选择的语言版本。

Chapter 23. Using Cruise Control to reassign partitions on JBOD disks

If you are using JBOD storage and have Cruise Control installed with Streams for Apache Kafka, you can reassign partitions between the JBOD disks used for storage on the same broker. This capability also allows you to remove JBOD disks without data loss.

To reassign partitions, configure a KafkaRebalance resource in remove-disks mode and specify a list of broker IDs with corresponding volume IDs for partition reassignment. Cruise Control generates an optimization proposal based on the configuration and reassigns the partitions when approved manually or automatically.

Use the Kafka kafka-log-dirs.sh tool to check information about Kafka topic partitions and their location on brokers before and after moving them. Use an interactive pod to avoid running the tool within the broker container and causing any disruptions.

Prerequisites

In this procedure, we use a Kafka cluster named my-cluster, which is deployed to the my-project namespace with node pools and Cruise Control enabled.

Example Kafka cluster configuration

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
  namespace: my-project
  annotations:
    strimzi.io/node-pools: enabled
spec:
  kafka:
    # ...
  cruiseControl: {}
    # ...
Copy to Clipboard Toggle word wrap

A node pool named pool-a is configured with three broker replicas that use three JBOD storage volumes. In the procedure, we show how partitions are reassigned from volume 1 and 2 to volume 0.

Example node pool configuration with JBOD storage

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: pool-a
  labels:
    strimzi.io/cluster: my-cluster
spec:
  replicas: 3
  roles:
    - broker
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 2000Gi
        deleteClaim: false
      - id: 1
        type: persistent-claim
        size: 2000Gi
        deleteClaim: false
      - id: 2
        type: persistent-claim
        size: 2000Gi
        deleteClaim: false
  # ...
Copy to Clipboard Toggle word wrap

Procedure

  1. Run a new interactive pod container using the Kafka image to connect to a running Kafka broker. 

    oc run --restart=Never --image=registry.redhat.io/amq-streams/kafka-39-rhel9:2.9.3 helper-pod -- /bin/sh -c "sleep 3600"
    Copy to Clipboard Toggle word wrap

    In this procedure, we use a pod named helper-pod.

  2. (Optional) Check the partition replica data on broker 0 by opening a terminal inside the interactive pod and running the Kafka kafka-log-dirs.sh tool: 

    oc exec -n myproject -ti my-cluster-pool-a-0 bin/kafka-log-dirs.sh --describe --bootstrap-server my-cluster-kafka-bootstrap:9092 --broker-list 0,1,2 --topic-list my-topic
    Copy to Clipboard Toggle word wrap

    my-cluster-pool-a-0 is the pod name for broker 0. The tool returns topic information for each log directory. In this example, we are restricting the information to my-topic to show the steps against a single topic. The JBOD volumes used for log directories are mounted at /var/lib/kafka/data-<volume_id>/kafka-log<pod_id>.

    Example output data for each log directory

    {
  "brokers": [
    {
      "broker": 0,
    1 
    
      "logDirs": [
        {
          "partitions": [
    2 
    
            {
              "partition": "my-topic-5",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-2",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            }
          ],
          "error": null,
    3 
    
          "logDir": "/var/lib/kafka/data-2/kafka-log0"
    4 
    
        },
        {
          "partitions": [
            {
              "partition": "my-topic-0",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-3",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            }
          ],
          "error": null,
          "logDir": "/var/lib/kafka/data-0/kafka-log0"
        },
        {
          "partitions": [
            {
              "partition": "my-topic-4",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-1",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            }
          ],
          "error": null,
          "logDir": "/var/lib/kafka/data-1/kafka-log0"
        }
      ]
    }
    Copy to Clipboard Toggle word wrap

    1
    The broker ID.
    2
    Partition details: name, size, offset lag. The (isFuture) property indicates that the partition is moving between log directories when showing as true.
    3
    If error is not null, there is an issue with the disk hosting the log directory.
    4
    The path and name of the log directory.

  3. Create a KafkaRebalance resource in remove-disks mode, listing the brokers and volume IDs to reassign partitions from. Without specific configuration, the default rebalance goals are used.

    Example Cruise Control configuration

    apiVersion: {KafkaRebalanceApiVersion}
kind: KafkaRebalance
metadata:
  name: my-rebalance
  labels:
    strimzi.io/cluster: my-cluster
spec:
  mode: remove-disks
  moveReplicasOffVolumes:
    - brokerId: 0
    1 
    
      volumeIds: [1, 2]
    2
    Copy to Clipboard Toggle word wrap

    1
    The broker from which to reassign partitions.
    2
    The volume IDs to reassign partitions from.

    In this example, my-rebalance reassigns partitions from volumes with IDs 1 and 2 on broker 0.

  4. (Optional) To approve the optimization proposal automatically, set the strimzi.io/rebalance-auto-approval annotation to true: 

    apiVersion: {KafkaRebalanceApiVersion}
kind: KafkaRebalance
metadata:
  name: my-rebalance
  labels:
    strimzi.io/cluster: my-cluster
  annotations:
    strimzi.io/rebalance-auto-approval: "true"
spec:
  mode: remove-disks
  moveReplicasOffVolumes:
    - brokerId: 0
      volumeIds: [1, 2]
    Copy to Clipboard Toggle word wrap
  5. Apply the KafkaRebalance configuration.

  6. If manually approving, wait for the status of the proposal to move to ProposalReady before approving the changes.

    1. Check the summary of the changes in the KafkaRebalance status: 

      oc get kafkarebalance my-rebalance -n my-project -o yaml
      Copy to Clipboard Toggle word wrap

      Example summary of changes

      apiVersion: {KafkaRebalanceApiVersion}
kind: KafkaRebalance
metadata:
  name: my-rebalance
  labels:
    strimzi.io/cluster: my-cluster
spec:
  mode: remove-disks
  moveReplicasOffVolumes:
    - brokerId: 0
      volumeIds: [1, 2]
status:
  - lastTransitionTime: "2024-11-13T06:55:42.217794891Z"
    status: "True"
    type: ProposalReady
  observedGeneration: 1
  optimizationResult:
    afterBeforeLoadConfigMap: my-rebalance
    dataToMoveMB: 0
    excludedBrokersForLeadership: []
    excludedBrokersForReplicaMove: []
    excludedTopics: []
    intraBrokerDataToMoveMB: 0
    monitoredPartitionsPercentage: 100
    numIntraBrokerReplicaMovements: 26
    numLeaderMovements: 0
    numReplicaMovements: 0
    onDemandBalancednessScoreAfter: 100
    onDemandBalancednessScoreBefore: 0
    provisionRecommendation: ""
    provisionStatus: UNDECIDED
    recentWindows: 1
  sessionId: 24537b9c-a315-4715-8e86-01481e914771
      Copy to Clipboard Toggle word wrap

      Note

      The summary only shows the changes after optimization, not the load before optimization.

    2. Annotate the KafkaRebalance resource to approve the changes: 

      oc annotate kafkarebalance my-rebalance strimzi.io/rebalance="approve"
      Copy to Clipboard Toggle word wrap
  7. Wait for the status of the proposal to change to Ready.

  8. Use the Kafka kafka-log-dirs.sh tool again to verify data movement.

    In this example, the log directories for volumes 1 and 2 no longer have partitions assigned to them and volume 0 holds 6 partitions for my-topic, indicating that the partitions have been successfully reassigned.

    Example output data following reassignment of partitions

    {
  "brokers": [
    {
      "broker": 0,
      "logDirs": [
        {
          "partitions": [],
          "error": null,
          "logDir": "/var/lib/kafka/data-2/kafka-log0"
        },
        {
          "partitions": [
            {
              "partition": "my-topic-4",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-5",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-0",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-1",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-2",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "my-topic-3",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            }
          ],
          "error": null,
          "logDir": "/var/lib/kafka/data-0/kafka-log0"
        },
        {
          "partitions": [],
          "error": null,
          "logDir": "/var/lib/kafka/data-1/kafka-log0"
        }
      ]
    }
    Copy to Clipboard Toggle word wrap

返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

Theme

© 2025 Red Hat