Configuring Red Hat Virtualization with Red Hat Gluster Storage

Procedure 3.1. To Enable Red Hat Gluster Storage in Red Hat Virtualization Manager

1. Install Red Hat Virtualization Manager

   The Red Hat Virtualization Manager is the control center of the Red Hat Virtualization environment. Ensure that Red Hat Virtualization Manger is installed and configured.
   Install and configure Red Hat Virtualization Manager. See Installing the Red Hat Virtualization Manager in the Red Hat Virtualization Installation Guide for more information: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Installation_Guide/part-Installing_Red_Hat_Enterprise_Virtualization.html.

2. Install Hosts

   Install the virtual machine hosts using either Red Hat Virtualization Hypervisor or Red Hat Enterprise Linux as hosts. Before beginning the installation process, ensure that all virtualization hosts meet the hardware requirements and that the installation of Red Hat Virtualization Manager is complete.
   You can use the following hypervisors as hosts:
   • Red Hat Virtualization Hypervisor 3.4
   • Red Hat Enterprise Linux 6.3
   • Red Hat Enterprise Linux 6.4
   • Red Hat Enterprise Linux 6.5
   During installation, set the fully qualified domain name (FQDN) and IP address for the server in order to avoid network conflicts. For information on installing the hypervisor, see Installing Hypervisor Hosts in the Red Hat Virtualization Installation Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Installation_Guide/part-Installing_Hypervisor_Hosts.html.

   Note

   You must install Red Hat Enterprise Linux 6.3, Red Hat Enterprise Linux 6.4 Server or Red Hat Enterprise Linux 6.5 on a system to use it as a virtualization host in a Red Hat Virtualization environment. See Installing Red Hat Enterprise Linux Hosts in the Red Hat Virtualization Installation Guide for details: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Installation_Guide/chap-Red_Hat_Enterprise_Linux_Hosts.html.

3. Subscribe hosts to channels or repositories

   Red Hat Enterprise Linux hosts need to be registered and subscribed to a number of entitlements using either Subscription Manager or Red Hat Network Classic. For more information, see Subscribing to Required Channels using RHN Classic in the Red Hat Virtualization Installation Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Installation_Guide/Subscribing_to_Required_Channels_using_Subscription_Manager1.html.
   Subscribe Red Hat Enterprise Linux 6.3, 6.4, and 6.5 hosts to the channels listed below:
   • rhel-x86_64-server-6
   • rhel-x86_64-rhev-mgmt-agent-6
   Apart from the above channels, you must also subscribe to the Red Hat Gluster Storage 3.3 Native Client channels to get the latest version:
   Run the rhn-channel --add --channel command to subscribe the system to the correct Red Hat Gluster Storage Native Client channel:
   • For Red Hat Enterprise Linux 7.x clients using Red Hat Satellite Server:

     # rhn-channel --add --channel= rhel-x86_64-server-rh-common-7

   • For Red Hat Enterprise Linux 6.x clients:

     # rhn-channel --add --channel=rhel-x86_64-server-rhsclient-6

   • For Red Hat Enterprise Linux 5.x clients:

     # rhn-channel --add --channel=rhel-x86_64-server-rhsclient-5

   For information on how to install native clients, see Installing Native Client in the Red Hat Gluster Storage Administration Guide: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-accessing_data_-_setting_up_clients#Installing_Native_Client

4. Configure the Hypervisor

   Red Hat Virtualization Hypervisor host

   Install Red Hat Virtualization Hypervisor 3.4 on bare metal. See Installing the Red Hat Virtualization Hypervisor chapter of the Red Hat Virtualization Installation Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Installation_Guide/part-Installing_Hypervisor_Hosts.html

   Red Hat Enterprise Linux 6.3, 6.4, or 6.5 host

   A Red Hat Enterprise Linux virtualization host must be registered and subscribed to a number of entitlements using either Subscription Manager or Red Hat Network Classic. See Subscribing to Required Channels using RHN Classic in the Installing Red Hat Enterprise Linux Hosts chapter of the Red Hat Virtualization Installation Guide for more information.

   Subscribe to the Red Hat Gluster Storage Native Client from the Additional Services Channels to ensure the server has the latest clients installed:

   # yum install glusterfs-fuse glusterfs

   For more information, see Installing Native Client in the Red Hat Gluster Storage Administration Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Installation_Guide/chap-Red_Hat_Enterprise_Linux_Hosts.html.

5. Configure the data centers

   Define data centers and clusters to organize the virtualization hosts. The installation process creates a data center and associated cluster named Default.

   Note

   A server can be a Red Hat Virtualization Hypervisor or a Red Hat Enterprise Linux host. A trusted storage pool requires a minimum of one storage host, and at least one active virtualization host is required to connect the system to a storage pool.

6. Install Red Hat Gluster Storage

   Install the latest version of Red Hat Gluster Storage on new servers, not the virtualization hosts.
   For more information on obtaining and installing the software, see the Red Hat Gluster Storage Installation Guide: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/installation_guide/chap-installing_red_hat_storage.
   If you want to configure combined virtualization hosts and storage servers, see the Red Hat Hyperconverged Infrastructure documentation: https://access.redhat.com/documentation/en-us/red_hat_hyperconverged_infrastructure/1.0/html/deploying_red_hat_hyperconverged_infrastructure/

7. Configure Storage

   Configure the Red Hat Gluster Storage installation for your requirements.
   For instructions on creating a Red Hat Gluster Storage Volume, see chapter Red Hat Gluster Storage Volume in the Red Hat Gluster Storage Administration Guide: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-red_hat_storage_volumes.
   Red Hat Virtualization uses a centralized storage system for virtual machine disk images. Before adding storage, ensure that you have a working Red Hat Virtualization Manager environment. You must be able to access the Administration Portal, and there must be at least one host connected with Up status.
   Optimize your Red Hat Gluster Storage volumes for storing virtual machine images by following the instructions in Procedure 4.1, “To Configure Volumes Using the Command Line Interface”.

   Note

   To reduce the risk of split-brain conditions developing in the cluster, Red Hat requires three-way replication when using Red Hat Gluster Storage with Red Hat Virtualization. Three-way replication is provided by the following volume types:

   • three-way replicated volumes (replica 3)
   • three-way distributed replicated volumes (replica count 3)
   • arbitrated replicated or distributed arbitrated replicated volumes (replica 3 arbiter 1)

8. Attach Storage to Hypervisor

   Check that you can access the Red Hat Virtualization Manager Administration Portal and that at least one virtualization host has Up status before you continue.
   Create a new storage domain with a Data domain function and the GlusterFS storage type. For further information, see the Adding a Red Hat Gluster Storage Volume as a Storage Domain section in the Red Hat Virtualization Administration Guide: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/html/administration_guide/sect-using_red_hat_gluster_storage_as_a_storage_domain

1. Install the Tune daemon

   Install the tuned daemon that tunes the system settings dynamically.

   # yum install tuned

2. On the virtual machine, edit the virtual-guest profile

   In the /etc/tune-profiles/virtual-guest/ktune.sh file, change the value of the multiply_disk_readahead option to 16.

3. Apply the modified virtual-guest profile on the virtual machines

   Activate the modified profile with the command:

   # tuned-adm profile virtual-guest

Procedure 4.1. To Configure Volumes Using the Command Line Interface

1. Configure the rhgs-random-io tuned profile

   Install the tuned tuning daemon and configure Red Hat Gluster Storage servers to use the rhgs-random-io profile:

   # yum install tuned
   # tuned-adm profile rhgs-random-io

   For more information on available tuning profiles, refer to the tuned-adm man page, or see the Red Hat Gluster Storage 3.3 Administration Guide: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/.

2. Review virt volume group configuration details

   The settings stored in the /var/lib/glusterd/groups/virt file are used to configure volumes in the virt volume group.

   Important

   When you upgrade, a new virt file may be created in /var/lib/glusterd/groups/virt.rpmnew. Ensure to apply the new virt file on the existing volumes by renaming the virt.rpmnew file to virt, along with the customized settings.
   By default, the /var/lib/glusterd/groups/virt file contains the following recommended settings.

   performance.quick-read=off
   performance.read-ahead=off
   performance.io-cache=off
   performance.stat-prefetch=off
   performance.low-prio-threads=32
   network.remote-dio=enable
   cluster.eager-lock=enable
   cluster.quorum-type=auto
   cluster.server-quorum-type=server
   cluster.data-self-heal-algorithm=full
   cluster.locking-scheme=granular
   cluster.shd-max-threads=8
   cluster.shd-wait-qlength=10000
   features.shard=on
   user.cifs=off

   With the exception of cluster.eager-lock, features.shard, and cluster.data-self-heal-algorithm, these settings prevent caching within GlusterFS client stack, as it is the preferred mode for attaching disks to a virtual machine. The cluster.eager-lock option optimizes write performance with synchronous replication when there is a single writer to a file. The features.shard option enables sharding behavior. The cluster.data-self-heal-algorithm option specifies how self-heal operations are performed. For more information about any of these settings, see the Red Hat Gluster Storage 3.3 Administration Guide: https://access.redhat.com/documentation/en-US/Red_Hat_storage/3.3/html/Administration_Guide/chap-Managing_Red_Hat_Storage_Volumes.html#Configuring_Volume_Options
   Red Hat also recommends adding the following line to the file to improve self-heal performance.

   cluster.granular-entry-heal=enable

   Note

   Server-Side and Client-Side Quorum are enabled by default in the /var/lib/glusterd/groups/virt file to minimize split-brain scenarios. If Server-Side Quorum is not met, then the Red Hat Gluster Storage volumes become unavailable causing the Virtual Machines (VMs) to move to a paused state. If Client-Side Quorum is not met, although a replica pair in a Red Hat Gluster Storage volume is available in the read-only mode, the VMs move to a paused state.

   Manual intervention is required to make the VMs resume the operations after the quorum is restored. Consistency is achieved at the cost of fault tolerance. If fault tolerance is preferred over consistency, disable server-side and client-side quorum with the commands:

   # gluster volume reset <vol-name> server-quorum-type
   # gluster volume reset <vol-name> quorum-type

   For more information on these configuration settings, see the following sections in the Red Hat Gluster Storage Administration Guide:
   • https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-managing_red_hat_storage_volumes
   • https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-managing_red_hat_storage_volumes#Configuring_Volume_Options

3. Assign volumes to virt group

   Red Hat recommends assigning volumes that store virtual machine images to the virt volume group so that these volumes can use common configuration details for their common use case. This has the same effect as the Optimize for Virt Store option in the management console.

   # gluster volume set VOLNAME group virt

   Important

   After tagging the volume as group virt, use the volume for storing virtual machine images only and always access the volume through the glusterFS native client.

4. Allow KVM and VDSM brick access

   Set the brick permissions for vdsm and kvm. If you do not set the required brick permissions, creation of virtual machines fails.
   1. Set the user and group permissions using the following commands:

      # gluster volume set VOLNAME storage.owner-uid 36
      # gluster volume set VOLNAME storage.owner-gid 36

   2. If you are using QEMU/KVM as a hypervisor, set the user and group permissions using the following commands:

      # gluster volume set VOLNAME storage.owner-uid 107
      # gluster volume set VOLNAME storage.owner-gid 107

5. Configure granular healing

   Red Hat recommends setting cluster.granular-entry-heal to enable for this use case. To configure granular healing, run the following commands.

   # gluster volume set VOLNAME cluster.granular-entry-heal enable

Procedure 4.2. To Add a Red Hat Gluster Storage Server for Virtualization Using Red Hat Virtualization Manager

1. Create a data center:
   1. Select the Data Centers resource tab to list all data centers.
   2. Click New to open the New Data Center window.
      

      Figure 4.1. New Data Center Window

   3. Enter the Name and Description of the data center.
   4. Select the storage Type as Shared from the drop-down menu.
   5. Select the Quota Mode as Disabled.
   6. Click OK.
   The new data center is Uninitialized until you configure the cluster, host, and storage settings.
2. Create a cluster:
   1. Select the Clusters resource tab to list all clusters.
   2. Click New to open the New Cluster window.
      

      Figure 4.2. New Cluster Window

   3. Select a Data Center for the cluster from the drop-down menu.
   4. Enter a Name and Description for the cluster.
   5. Select the CPU Name and Compatibility Version from the drop-down menus.
   6. Check Enable Virt Service.
   7. Click OK.
3. Add hosts:
   1. Select the Hosts resource tab to view a list of all hosts in the system.
   2. Click New to open the New Host window.
      

      Figure 4.3. New Host Window

      Important

      A Red Hat Enterprise Linux hypervisor and Red Hat Virtualization hypervisor on a single VDSM cluster accessing the same virtual machine image store is not supported.
   3. Select the Data Center and Host Cluster for the new host from the drop-down menus.
   4. Enter the Name, Address, and Root Password of the new hypervisor host.
   5. Check Automatically configure host firewall if required.
   6. Click OK.
   The new host appears in the list of hypervisor hosts with the status Installing. After the host is activated, the status changes to Up automatically.
4. Create and configure volumes on the Red Hat Gluster Storage cluster using the command line interface. For information on creating and configuring volumes, see Section 4.1, “Configuring Volumes Using the Command Line Interface” and Red Hat Gluster Storage Volumes in the Red Hat Gluster Storage Administration Guide: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-red_hat_storage_volumes.
5. Add a storage domain using Red Hat Virtualization Manager:
   1. Select the Storage resource tab to list existing storage domains.
   2. Click New Domain to open the New Domain window.
      

      Figure 4.4. New Domain Window

   3. Enter a Name for the storage domain.
   4. Select a shared Data Center to associate with the storage domain.
   5. Set the Domain Function to Data and the Storage Type to GlusterFS.
   6. Select a host from the Host to Use drop-down menu.
   7. Check the Use managed gluster volume checkbox and select the appropriate volume from the Gluster dropdown menu.

      Note

      This dropdown menu is only populated with volumes whose nodes are managed by Red Hat Virtualization Manager. See Chapter 5, Managing Red Hat Gluster Storage Servers and Volumes using Red Hat Virtualization Manager for instructions on how to set up management of your Red Hat Gluster Storage nodes by Red Hat Virtualization Manager.
   8. Enter the applicable Red Hat Gluster Storage native client Mount Options. Enter multiple mount options separated by commas. For more information on native client mount options, see Creating Access to Volumes in the Red Hat Gluster Storage Administration Guide: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html/administration_guide/chap-accessing_data_-_setting_up_clients. Note that only the native client is supported when integrating Red Hat Gluster Storage and Red Hat Virtualization.
   9. Click OK.
   10. Click Activate to activate the new Red Hat Gluster Storage domain.
       

       Figure 4.5. Activated Red Hat Gluster Storage Domain