Chapter 3. Preparing Storage for Red Hat Virtualization
Prepare storage to be used for storage domains in the new environment. A Red Hat Virtualization environment must have at least one data storage domain, but adding more is recommended.
A data domain holds the virtual hard disks and OVF files of all the virtual machines and templates in a data center, and cannot be shared across data centers while active (but can be migrated between data centers). Data domains of multiple storage types can be added to the same data center, provided they are all shared, rather than local, domains.
Self-hosted engines must have an additional data domain dedicated to the Manager virtual machine. This domain is created during the self-hosted engine deployment, and must be at least 74 GiB. You must prepare the storage for this domain before beginning the deployment.
You can use one of the following storage types:
If you are using iSCSI storage, the self-hosted engine storage domain must use its own iSCSI target. Any additional storage domains must use a different iSCSI target.
Creating additional data storage domains in the same data center as the self-hosted engine storage domain is highly recommended. If you deploy the self-hosted engine in a data center with only one active data storage domain, and that storage domain is corrupted, you will not be able to add new storage domains or remove the corrupted storage domain; you will have to redeploy the self-hosted engine.
3.1. Preparing NFS Storage
Set up NFS shares on your file storage or remote server to serve as storage domains on Red Hat Enterprise Virtualization Host systems. After exporting the shares on the remote storage and configuring them in the Red Hat Virtualization Manager, the shares will be automatically imported on the Red Hat Virtualization hosts.
For information on setting up and configuring NFS, see Network File System (NFS) in the Red Hat Enterprise Linux 7 Storage Administration Guide.
For information on how to export an 'NFS' share, see How to export 'NFS' share from NetApp Storage / EMC SAN in Red Hat Virtualization
Specific system user accounts and system user groups are required by Red Hat Virtualization so the Manager can store data in the storage domains represented by the exported directories. The following procedure sets the permissions for one directory. You must repeat the chown and chmod steps for all of the directories you intend to use as storage domains in Red Hat Virtualization.
Procedure
Create the group
kvm:# groupadd kvm -g 36
Create the user
vdsmin the groupkvm:# useradd vdsm -u 36 -g 36
Set the ownership of your exported directory to 36:36, which gives
vdsm:kvmownership:# chown -R 36:36 /exports/dataChange the mode of the directory so that read and write access is granted to the owner, and so that read and execute access is granted to the group and other users:
# chmod 0755 /exports/data
3.2. Preparing iSCSI Storage
Red Hat Virtualization supports iSCSI storage, which is a storage domain created from a volume group made up of LUNs. Volume groups and LUNs cannot be attached to more than one storage domain at a time.
For information on setting up and configuring iSCSI storage, see Online Storage Management in the Red Hat Enterprise Linux 7 Storage Administration Guide.
If you are using block storage and you intend to deploy virtual machines on raw devices or direct LUNs and to manage them with the Logical Volume Manager, you must create a filter to hide the guest logical volumes. This will prevent guest logical volumes from being activated when the host is booted, a situation that could lead to stale logical volumes and cause data corruption. See https://access.redhat.com/solutions/2662261 for details.
Red Hat Virtualization currently does not support block storage with a block size of 4K. You must configure block storage in legacy (512b block) mode.
If your host is booting from SAN storage and loses connectivity to the storage, the storage file systems become read-only and remain in this state after connectivity is restored.
To prevent this situation, Red Hat recommends adding a drop-in multipath configuration file on the root file system of the SAN for the boot LUN to ensure that it is queued when there is a connection:
# cat /etc/multipath/conf.d/host.conf
multipaths {
multipath {
wwid boot_LUN_wwid
no_path_retry queue
}3.3. Preparing FCP Storage
Red Hat Virtualization supports SAN storage by creating a storage domain from a volume group made of pre-existing LUNs. Neither volume groups nor LUNs can be attached to more than one storage domain at a time.
Red Hat Virtualization system administrators need a working knowledge of Storage Area Networks (SAN) concepts. SAN usually uses Fibre Channel Protocol (FCP) for traffic between hosts and shared external storage. For this reason, SAN may occasionally be referred to as FCP storage.
For information on setting up and configuring FCP or multipathing on Red Hat Enterprise Linux, see the Storage Administration Guide and DM Multipath Guide.
If you are using block storage and you intend to deploy virtual machines on raw devices or direct LUNs and to manage them with the Logical Volume Manager, you must create a filter to hide the guest logical volumes. This will prevent guest logical volumes from being activated when the host is booted, a situation that could lead to stale logical volumes and cause data corruption. See https://access.redhat.com/solutions/2662261 for details.
Red Hat Virtualization currently does not support block storage with a block size of 4K. You must configure block storage in legacy (512b block) mode.
If your host is booting from SAN storage and loses connectivity to the storage, the storage file systems become read-only and remain in this state after connectivity is restored.
To prevent this situation, Red Hat recommends adding a drop-in multipath configuration file on the root file system of the SAN for the boot LUN to ensure that it is queued when there is a connection:
# cat /etc/multipath/conf.d/host.conf
multipaths {
multipath {
wwid boot_LUN_wwid
no_path_retry queue
}3.4. Preparing Red Hat Gluster Storage
For information on setting up and configuring Red Hat Gluster Storage, see the Red Hat Gluster Storage Installation Guide.
For the Red Hat Gluster Storage versions that are supported with Red Hat Virtualization, see https://access.redhat.com/articles/2356261.
3.5. Customizing Multipath Configurations for SAN Vendors
To customize the multipath configuration settings, do not modify /etc/multipath.conf. Instead, create a new configuration file that overrides /etc/multipath.conf.
Upgrading Virtual Desktop and Server Manager (VDSM) overwrites the /etc/multipath.conf file. If multipath.conf contains customizations, overwriting it can trigger storage issues.
Prerequisites
-
This topic only applies to systems that have been configured to use multipath connections storage domains, and therefore have a
/etc/multipath.conffile. -
Do not override the
user_friendly_namesandfind_multipathssettings. For more information, see Section 3.6, “Recommended Settings for Multipath.conf” -
Avoid overriding
no_path_retryandpolling_intervalunless required by the storage vendor. For more information, see Section 3.6, “Recommended Settings for Multipath.conf”
Procedure
To override the values of settings in
/etc/multipath.conf, create a new configuration file in the/etc/multipath/conf.d/directory.NoteThe files in
/etc/multipath/conf.d/execute in alphabetical order. Follow the convention of naming the file with a number at the beginning of its name. For example,/etc/multipath/conf.d/90-myfile.conf.-
Copy the settings you want to override from
/etc/multipath.confto the new configuration file in/etc/multipath/conf.d/. Edit the setting values and save your changes. Apply the new configuration settings by entering the
systemctl reload multipathdcommand.NoteAvoid restarting the multipathd service. Doing so generates errors in the VDSM logs.
Verification steps
If you override the VDSM-generated settings in /etc/multipath.conf, verify that the new configuration performs as expected in a variety of failure scenarios.
For example, disable all of the storage connections. Then enable one connection at a time and verify that doing so makes the storage domain reachable.
Troubleshooting
If a Red Hat Virtualization Host has trouble accessing shared storage, check /etc/multpath.conf and files under /etc/multipath/conf.d/ for values that are incompatible with the SAN.
Additional resources
- Red Hat Enterprise Linux DM Multipath in the RHEL documentation.
- Configuring iSCSI Multipathing in the Administration Guide.
-
How do I customize /etc/multipath.conf on my RHVH hypervisors? What values must not change and why? on the Red Hat Customer Portal, which shows an example
multipath.conffile and was the basis for this topic.
3.6. Recommended Settings for Multipath.conf
When overriding /etc/multipath.conf, Do not override the following settings:
user_friendly_names no- This setting controls whether user-friendly names are assigned to devices in addition to the actual device names. Multiple hosts must use the same name to access devices. Disabling this setting prevents user-friendly names from interfering with this requirement.
find_multipaths no- This setting controls whether RHVH tries to access all devices through multipath, even if only one path is available. Disabling this setting prevents RHV from using the too-clever behavior when this setting is enabled.
Avoid overriding the following settings unless required by the storage system vendor:
no_path_retry 4-
This setting controls the number of polling attempts to retry when no paths are available. Before RHV version 4.2, the value of
no_path_retrywasfailbecause QEMU had trouble with the I/O queuing when no paths were available. Thefailvalue made it fail quickly and paused the virtual machine. RHV version 4.2 changed this value to4so when multipathd detects the last path has failed, it checks all of the paths four more times. Assuming the default 5-second polling interval, checking the paths takes 20 seconds. If no path is up, multipathd tells the kernel to stop queuing and fails all outstanding and future I/O until a path is restored. When a path is restored, the 20-second delay is reset for the next time all paths fail. For more details, see the commit that changed this setting. polling_interval 5- This setting determines the number of seconds between polling attempts to detect whether a path is open or has failed. Unless the vendor provides a clear reason for increasing the value, keep the VDSM-generated default so the system responds to path failures sooner.
