Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 4. Configure performance improvements

Some deployments benefit from additional configuration to achieve optimal performance. This section covers recommended additional configuration for certain deployments.

The default value of the shard-block-size parameter changed from 4MB to 64MB between Red Hat Hyperconverged Infrastructure for Virtualization version 1.0 and 1.1. This means that all new volumes are created with a shard-block-size value of 64MB. However, existing volumes retain the original shard-block-size value of 4MB.

There is no safe way to modify the shard-block-size value on volumes that contain data. Because shard block size applies only to writes that occur after the value is set, attempting to change the value on a volume that contains data results in a mixed shard block size, which results in poor performance.

This section shows you how to safely modify the shard block size on an existing volume after upgrading to Red Hat Hyperconverged Infrastructure for Virtualization 1.1 or higher, in order to take advantage of the performance benefits of a larger shard size.

4.1.1. Changing shard size on replicated volumes
Copy link

  1. Create an inventory file

    Create an inventory file called normal_replicated_inventory.yml based on the following example.

    Replace host1, host2, and host3 with the FQDNs of your hosts, and edit device details to match your environment.

    Example normal_replicated_inventory.yml inventory file

    hc_nodes:
  hosts:
    # Host1
    host1:
      # Dedupe & Compression config
      # If logicalsize >= 1000G then slabsize=32G else slabsize=2G
      #gluster_infra_vdo:
      #   - { name: 'vdo_sdb', device: '/dev/sdb', logicalsize: '3000G', emulate512: 'on', slabsize: '32G',
      #      blockmapcachesize:  '128M', readcachesize: '20M', readcache: 'enabled', writepolicy: 'auto' }

      # With Dedupe & Compression
      #gluster_infra_volume_groups:
      #  - vgname: <volgroup_name>
      #    pvname: /dev/mapper/vdo_sdb

      # Without Dedupe & Compression
      gluster_infra_volume_groups:
        - vgname: <volgroup_name>
          pvname: /dev/sdb

      gluster_infra_mount_devices:
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>

      gluster_infra_thinpools:
        - {vgname: '<volgroup_name>', thinpoolname: 'thinpool_<volgroup_name>', thinpoolsize: '500G', poolmetadatasize: '4G'}

      gluster_infra_lv_logicalvols:
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G

      # Mount the devices
      gluster_infra_mount_devices:
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }

    #Host2
    host2:
      # Dedupe & Compression config
      # If logicalsize >= 1000G then slabsize=32G else slabsize=2G
      #gluster_infra_vdo:
      #   - { name: 'vdo_sdb', device: '/dev/sdb', logicalsize: '3000G', emulate512: 'on', slabsize: '32G',
      #      blockmapcachesize:  '128M', readcachesize: '20M', readcache: 'enabled', writepolicy: 'auto' }

      # With Dedupe & Compression
      #gluster_infra_volume_groups:
      #  - vgname: <volgroup_name>
      #    pvname: /dev/mapper/vdo_sdb

      # Without Dedupe & Compression
      gluster_infra_volume_groups:
        - vgname: <volgroup_name>
          pvname: /dev/sdb

      gluster_infra_mount_devices:
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>

      gluster_infra_thinpools:
        - {vgname: '<volgroup_name>', thinpoolname: 'thinpool_<volgroup_name>', thinpoolsize: '500G', poolmetadatasize: '4G'}

      gluster_infra_lv_logicalvols:
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G

      # Mount the devices
      gluster_infra_mount_devices:
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }

    #Host3
    host3:
      # Dedupe & Compression config
      # If logicalsize >= 1000G then slabsize=32G else slabsize=2G
      #gluster_infra_vdo:
      #   - { name: 'vdo_sdb', device: '/dev/sdb', logicalsize: '3000G', emulate512: 'on', slabsize: '32G',
      #      blockmapcachesize:  '128M', readcachesize: '20M', readcache: 'enabled', writepolicy: 'auto' }

      # With Dedupe & Compression
      #gluster_infra_volume_groups:
      #  - vgname: <volgroup_name>
      #    pvname: /dev/mapper/vdo_sdb

      # Without Dedupe & Compression
      gluster_infra_volume_groups:
        - vgname: <volgroup_name>
          pvname: /dev/sdb

      gluster_infra_mount_devices:
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>

      gluster_infra_thinpools:
        - {vgname: '<volgroup_name>', thinpoolname: 'thinpool_<volgroup_name>', thinpoolsize: '500G', poolmetadatasize: '4G'}

      gluster_infra_lv_logicalvols:
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G

      # Mount the devices
      gluster_infra_mount_devices:
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }

  # Common configurations
  vars:
    cluster_nodes:
       - host1
       - host2
       - host3
    gluster_features_hci_cluster: "{{ cluster_nodes }}"
    gluster_features_hci_volumes:
       - { volname: 'data', brick: '<brick_mountpoint>' }
    gluster_features_hci_volume_options:
      {
        group: 'virt',
        storage.owner-uid: '36',
        storage.owner-gid: '36',
        network.ping-timeout: '30',
        performance.strict-o-direct: 'on',
        network.remote-dio: 'off',
        cluster.granular-entry-heal: 'enable',
        features.shard-block-size: '64MB'
      }
    Copy to Clipboard Toggle word wrap

  2. Create the normal_replicated.yml playbook

    Create a normal_replicated.yml playbook file using the following example:

    Example normal_replicated.yml playbook

    ---

# Safely changing the shard block size parameter value for normal replicated volume
- name: Changing the shard block size
  hosts: hc_nodes
  remote_user: root
  gather_facts: no
  any_errors_fatal: true

  roles:
     - gluster.infra
     - gluster.features
    Copy to Clipboard Toggle word wrap

  3. Run the playbook 

    ansible-playbook -i normal_replicated_inventory.yml normal_replicated.yml
    Copy to Clipboard Toggle word wrap

4.1.2. Changing shard size on arbitrated volumes
Copy link

  1. Create an inventory file

    Create an inventory file called arbitrated_replicated_inventory.yml based on the following example.

    Replace host1, host2, and host3 with the FQDNs of your hosts, and edit device details to match your environment.

    Example arbitrated_replicated_inventory.yml inventory file

    hc_nodes:
  hosts:
    # Host1
    host1:
      # Dedupe & Compression config
      # If logicalsize >= 1000G then slabsize=32G else slabsize=2G
      #gluster_infra_vdo:
      #   - { name: 'vdo_sdb', device: '/dev/sdb', logicalsize: '3000G', emulate512: 'on', slabsize: '32G',
      #      blockmapcachesize:  '128M', readcachesize: '20M', readcache: 'enabled', writepolicy: 'auto' }

      # With Dedupe & Compression
      #gluster_infra_volume_groups:
      #  - vgname: <volgroup_name>
      #    pvname: /dev/mapper/vdo_sdb

      # Without Dedupe & Compression
      gluster_infra_volume_groups:
        - vgname: <volgroup_name>
          pvname: /dev/sdb

      gluster_infra_mount_devices:
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>

      gluster_infra_thinpools:
        - {vgname: '<volgroup_name>', thinpoolname: 'thinpool_<volgroup_name>', thinpoolsize: '500G', poolmetadatasize: '4G'}

      gluster_infra_lv_logicalvols:
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G

      # Mount the devices
      gluster_infra_mount_devices:
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }

    #Host2
    host2:
      # Dedupe & Compression config
      # If logicalsize >= 1000G then slabsize=32G else slabsize=2G
      #gluster_infra_vdo:
      #   - { name: 'vdo_sdb', device: '/dev/sdb', logicalsize: '3000G', emulate512: 'on', slabsize: '32G',
      #      blockmapcachesize:  '128M', readcachesize: '20M', readcache: 'enabled', writepolicy: 'auto' }

      # With Dedupe & Compression
      #gluster_infra_volume_groups:
      #  - vgname: <volgroup_name>
      #    pvname: /dev/mapper/vdo_sdb

      # Without Dedupe & Compression
      gluster_infra_volume_groups:
        - vgname: <volgroup_name>
          pvname: /dev/sdb

      gluster_infra_mount_devices:
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>

      gluster_infra_thinpools:
        - {vgname: '<volgroup_name>', thinpoolname: 'thinpool_<volgroup_name>', thinpoolsize: '500G', poolmetadatasize: '4G'}

      gluster_infra_lv_logicalvols:
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G

      # Mount the devices
      gluster_infra_mount_devices:
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }

    #Host3
    host3:
      # Dedupe & Compression config
      # If logicalsize >= 1000G then slabsize=32G else slabsize=2G
      #gluster_infra_vdo:
      #   - { name: 'vdo_sdb', device: '/dev/sdb', logicalsize: '3000G', emulate512: 'on', slabsize: '32G',
      #      blockmapcachesize:  '128M', readcachesize: '20M', readcache: 'enabled', writepolicy: 'auto' }

      # With Dedupe & Compression
      #gluster_infra_volume_groups:
      #  - vgname: <volgroup_name>
      #    pvname: /dev/mapper/vdo_sdb

      # Without Dedupe & Compression
      gluster_infra_volume_groups:
        - vgname: <volgroup_name>
          pvname: /dev/sdb

      gluster_infra_mount_devices:
        - path: <brick_mountpoint>
          lvname: <lv_name>
          vgname: <volgroup_name>

      gluster_infra_thinpools:
        - {vgname: '<volgroup_name>', thinpoolname: 'thinpool_<volgroup_name>', thinpoolsize: '500G', poolmetadatasize: '4G'}

      gluster_infra_lv_logicalvols:
        - vgname: <volgroup_name>
          thinpool: thinpool_<volgroup_name>
          lvname: <lv_name>
          lvsize: <size>G

      # Mount the devices
      gluster_infra_mount_devices:
         - { path: '<brick_mountpoint>', vgname: <volgroup_name>, lvname: <lv_name> }

  # Common configurations
  vars:
    cluster_nodes:
       - host1
       - host2
       - host3
    gluster_features_hci_cluster: "{{ cluster_nodes }}"
    gluster_features_hci_volumes:
       - { volname: 'data_one', brick: '<brick_mountpoint>', arbiter: 1 }
    gluster_features_hci_volume_options:
      {
        group: 'virt',
        storage.owner-uid: '36',
        storage.owner-gid: '36',
        network.ping-timeout: '30',
        performance.strict-o-direct: 'on',
        network.remote-dio: 'off',
        cluster.granular-entry-heal: 'enable',
        features.shard-block-size: '64MB',
      }
    Copy to Clipboard Toggle word wrap

  2. Create the arbitrated_replicated.yml playbook

    Create a arbitrated_replicated.yml playbook file using the following example:

    Example arbitrated_replicated.yml playbook

    ---

# Safely changing the shard block size parameter value for arbitrated replicated volume
- name: Changing the shard block size
  hosts: hc_nodes
  remote_user: root
  gather_facts: no
  any_errors_fatal: true

  roles:
     - gluster.infra
     - gluster.features
    Copy to Clipboard Toggle word wrap

  3. Run the playbook 

    ansible-playbook -i arbitrated_replicated_inventory.yml arbitrated_replicated.yml
    Copy to Clipboard Toggle word wrap

If your main storage devices are not Solid State Disks (SSDs), Red Hat recommends configuring a logical volume cache (lvmcache) to achieve the required performance for Red Hat Hyperconverged Infrastructure for Virtualization deployments.

  1. Create inventory file

    Create an inventory file called cache_inventory.yml based on the example below.

    Replace <host1>, <host2>, and <host3> with the FQDNs of the hosts on which to configure the cache.

    Replace the following values throughout the file.

    <slow_device>,<fast_device>
    Specify the device to which the cache should attach, followed by the cache device, as a comma-delimited list, for example, cachedisk: '/dev/sdb,/dev/sde'.
    <fast_device_name>
    Specify the name of the cache logical volume to create, for example, cachelv_thinpool_gluster_vg_sde
    <fast_device_thinpool>
    Specify the name of the cache thin pool to create, for example, gluster_thinpool_gluster_vg_sde.

    Example cache_inventory.yml file

    hc_nodes:
  hosts:
    # Host1
    <host1>:
      gluster_infra_cache_vars:
        - vgname: gluster_vg_sdb
          cachedisk: '<slow_device>,<fast_device>'
          cachelvname: <fast_device_name>
          cachethinpoolname: <fast_device_thinpool>
          cachelvsize: '10G'
          cachemode: writethrough

    #Host2
    <host2>:
      gluster_infra_cache_vars:
        - vgname: gluster_vg_sdb
          cachedisk: '<slow_device>,<fast_device>'
          cachelvname: <fast_device_name>
          cachethinpoolname: <fast_device_thinpool>
          cachelvsize: '10G'
          cachemode: writethrough

    #Host3
    <host3>:
      gluster_infra_cache_vars:
        - vgname: gluster_vg_sdb
          cachedisk: '<slow_device>,<fast_device>'
          cachelvname: <fast_device_name>
          cachethinpoolname: <fast_device_thinpool>
          cachelvsize: '10G'
          cachemode: writethrough
    Copy to Clipboard Toggle word wrap

  2. Create a playbook file

    Create an ansible playbook file named lvm_cache.yml.

    Example lvm_cache.yml file

    ---
# Create LVM Cache
- name: Setup LVM Cache
  hosts: hc_nodes
  remote_user: root
  gather_facts: no
  any_errors_fatal: true

  roles:
     - gluster.infra
    Copy to Clipboard Toggle word wrap

  3. Run the playbook with the cachesetup tag

    Run the following command to apply the configuration specified in lvm_cache.yml to the hosts and devices specified in cache_inventory.yml. 

    ansible-playbook -i cache_inventory.yml lvm_cache.yml --tags=cachesetup
    Copy to Clipboard Toggle word wrap
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat