Chapter 4. Hosting Virtual Machine Images on Red Hat Gluster Storage volumes

Procedure 4.1. To Configure Volumes Using the Command Line Interface

1. Configure the rhgs-random-io tuned profile

   Install the tuned tuning daemon and configure Red Hat Gluster Storage servers to use the rhgs-random-io profile:

   # yum install tuned
   # tuned-adm profile rhgs-random-io

   For more information on available tuning profiles, refer to the tuned-adm man page, or see the Red Hat Gluster Storage 3.3 Administration Guide: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/.

2. Review virt volume group configuration details

   The settings stored in the /var/lib/glusterd/groups/virt file are used to configure volumes in the virt volume group.

   Important

   When you upgrade, a new virt file may be created in /var/lib/glusterd/groups/virt.rpmnew. Ensure to apply the new virt file on the existing volumes by renaming the virt.rpmnew file to virt, along with the customized settings.
   By default, the /var/lib/glusterd/groups/virt file contains the following recommended settings.

   performance.quick-read=off
   performance.read-ahead=off
   performance.io-cache=off
   performance.stat-prefetch=off
   performance.low-prio-threads=32
   network.remote-dio=enable
   cluster.eager-lock=enable
   cluster.quorum-type=auto
   cluster.server-quorum-type=server
   cluster.data-self-heal-algorithm=full
   cluster.locking-scheme=granular
   cluster.shd-max-threads=8
   cluster.shd-wait-qlength=10000
   features.shard=on
   user.cifs=off

   With the exception of cluster.eager-lock, features.shard, and cluster.data-self-heal-algorithm, these settings prevent caching within GlusterFS client stack, as it is the preferred mode for attaching disks to a virtual machine. The cluster.eager-lock option optimizes write performance with synchronous replication when there is a single writer to a file. The features.shard option enables sharding behavior. The cluster.data-self-heal-algorithm option specifies how self-heal operations are performed. For more information about any of these settings, see the Red Hat Gluster Storage 3.3 Administration Guide: https://access.redhat.com/documentation/en-US/Red_Hat_storage/3.3/html/Administration_Guide/chap-Managing_Red_Hat_Storage_Volumes.html#Configuring_Volume_Options
   Red Hat also recommends adding the following line to the file to improve self-heal performance.

   cluster.granular-entry-heal=enable

   Note

   Server-Side and Client-Side Quorum are enabled by default in the /var/lib/glusterd/groups/virt file to minimize split-brain scenarios. If Server-Side Quorum is not met, then the Red Hat Gluster Storage volumes become unavailable causing the Virtual Machines (VMs) to move to a paused state. If Client-Side Quorum is not met, although a replica pair in a Red Hat Gluster Storage volume is available in the read-only mode, the VMs move to a paused state.

   Manual intervention is required to make the VMs resume the operations after the quorum is restored. Consistency is achieved at the cost of fault tolerance. If fault tolerance is preferred over consistency, disable server-side and client-side quorum with the commands:

   # gluster volume reset <vol-name> server-quorum-type
   # gluster volume reset <vol-name> quorum-type

   For more information on these configuration settings, see the following sections in the Red Hat Gluster Storage Administration Guide:
   • https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-managing_red_hat_storage_volumes
   • https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-managing_red_hat_storage_volumes#Configuring_Volume_Options

3. Assign volumes to virt group

   Red Hat recommends assigning volumes that store virtual machine images to the virt volume group so that these volumes can use common configuration details for their common use case. This has the same effect as the Optimize for Virt Store option in the management console.

   # gluster volume set VOLNAME group virt

   Important

   After tagging the volume as group virt, use the volume for storing virtual machine images only and always access the volume through the glusterFS native client.

4. Allow KVM and VDSM brick access

   Set the brick permissions for vdsm and kvm. If you do not set the required brick permissions, creation of virtual machines fails.
   1. Set the user and group permissions using the following commands:

      # gluster volume set VOLNAME storage.owner-uid 36
      # gluster volume set VOLNAME storage.owner-gid 36

   2. If you are using QEMU/KVM as a hypervisor, set the user and group permissions using the following commands:

      # gluster volume set VOLNAME storage.owner-uid 107
      # gluster volume set VOLNAME storage.owner-gid 107

5. Configure granular healing

   Red Hat recommends setting cluster.granular-entry-heal to enable for this use case. To configure granular healing, run the following commands.

   # gluster volume set VOLNAME cluster.granular-entry-heal enable