2.6. Storage

Prerequisites

1. Install the NFS utils package.

   # dnf install nfs-utils -y

2. To check the enabled versions:

   # cat /proc/fs/nfsd/versions

3. Enable the following services:

   # systemctl enable nfs-server
   # systemctl enable rpcbind

Procedure

1. Create the group kvm:

   # groupadd kvm -g 36

2. Create the user vdsm in the group kvm:

   # useradd vdsm -u 36 -g kvm

3. Create the storage directory and modify the access rights.

   # mkdir /storage
   # chmod 0755 /storage
   # chown 36:36 /storage/

4. Add the storage directory to /etc/exports with the relevant permissions.

   # vi /etc/exports
   # cat /etc/exports
    /storage *(rw)

5. Restart the following services:

   # systemctl restart rpcbind
   # systemctl restart nfs-server

6. To see which export are available for a specific IP address:

   # exportfs
    /nfs_server/srv
                  10.46.11.3/24
    /nfs_server       <world>

Procedure

1. In the Administration Portal, click Storage Domains.
2. Click New Domain.
3. Enter a Name for the storage domain.
4. Accept the default values for the Data Center, Domain Function, Storage Type, Format, and Host lists.
5. Enter the Export Path to be used for the storage domain. The export path should be in the format of 123.123.0.10:/data (for IPv4), [2001:0:0:0:0:0:0:5db1]:/data (for IPv6), or domain.example.com:/data.

6. Optionally, you can configure the advanced parameters:

   1. Click Advanced Parameters.
   2. Enter a percentage value into the Warning Low Space Indicator field. If the free space available on the storage domain is below this percentage, warning messages are displayed to the user and logged.
   3. Enter a GB value into the Critical Space Action Blocker field. If the free space available on the storage domain is below this value, error messages are displayed to the user and logged, and any new action that consumes space, even temporarily, will be blocked.
   4. Select the Wipe After Delete check box to enable the wipe after delete option. This option can be edited after the domain is created, but doing so will not change the wipe after delete property of disks that already exist.
7. Click OK.

Procedure

1. Click Storage Domains.
2. Click the NFS storage domain’s name. This opens the details view.
3. Click the Data Center tab and click Maintenance to place the storage domain into maintenance mode. This unmounts the existing share and makes it possible to resize the storage domain.
4. On the NFS server, resize the storage. For Red Hat Enterprise Linux 6 systems, see Red Hat Enterprise Linux 6 Storage Administration Guide. For Red Hat Enterprise Linux 7 systems, see Red Hat Enterprise Linux 7 Storage Administration Guide. For Red Hat Enterprise Linux 8 systems, see Resizing a partition.
5. In the details view, click the Data Center tab and click Activate to mount the storage domain.

Procedure for Red Hat Enterprise Linux hosts

1. On the host, create the directory to be used for the local storage:

   # mkdir -p /data/images

2. Ensure that the directory has permissions allowing read/write access to the vdsm user (UID 36) and kvm group (GID 36):

   # chown 36:36 /data /data/images
   # chmod 0755 /data /data/images

1. Create a local storage directory:

   # mkdir /data
   # lvcreate -L $SIZE rhvh -n data
   # mkfs.ext4 /dev/mapper/rhvh-data
   # echo "/dev/mapper/rhvh-data /data ext4 defaults,discard 1 2" >> /etc/fstab
   # mount /data

2. Mount the new local storage:

   # mount -a

3. Ensure that the directory has permissions allowing read/write access to the vdsm user (UID 36) and kvm group (GID 36):

   # chown 36:36 /data /rhvh-data
   # chmod 0755 /data /rhvh-data

Procedure

1. Click Compute Hosts and select the host.
2. Click Management Maintenance and OK. The host’s status changes to Maintenance.
3. Click Management Configure Local Storage.
4. Click the Edit buttons next to the Data Center, Cluster, and Storage fields to configure and name the local storage domain.
5. Set the path to your local storage in the text entry field.
6. If applicable, click the Optimization tab to configure the memory optimization policy for the new local storage cluster.
7. Click OK.

Verification

1. Click Storage Domains.
2. Locate the local storage domain you just added.

Procedure

1. Click Storage Domains.
2. Click New Domain.
3. Enter the Name for the storage domain.
4. Select the Data Center to be associated with the storage domain. The data center selected must be of type POSIX (POSIX compliant FS). Alternatively, select (none).

5. Select Data from the Domain Function drop-down list, and POSIX compliant FS from the Storage Type drop-down list.

   If applicable, select the Format from the drop-down menu.

6. Select a host from the Host drop-down list.
7. Enter the Path to the POSIX file system, as you would normally provide it to the mount command.
8. Enter the VFS Type, as you would normally provide it to the mount command using the -t argument. See man mount for a list of valid VFS types.
9. Enter additional Mount Options, as you would normally provide them to the mount command using the -o argument. The mount options should be provided in a comma-separated list. See man mount for a list of valid mount options.

10. Optionally, you can configure the advanced parameters.

    1. Click Advanced Parameters.
    2. Enter a percentage value in the Warning Low Space Indicator field. If the free space available on the storage domain is below this percentage, warning messages are displayed to the user and logged.
    3. Enter a GB value in the Critical Space Action Blocker field. If the free space available on the storage domain is below this value, error messages are displayed to the user and logged, and any new action that consumes space, even temporarily, will be blocked.
    4. Select the Wipe After Delete check box to enable the wipe after delete option. This option can be edited after the domain is created, but doing so will not change the wipe after delete property of disks that already exist.
11. Click OK.

Procedure

1. Click Storage Domains.
2. Click New Domain.
3. Enter the Name of the new storage domain.
4. Select a Data Center from the drop-down list.
5. Select Data as the Domain Function and iSCSI as the Storage Type.

6. Select an active host as the Host.

   Important

   Communication to the storage domain is from the selected host and not directly from the Manager. Therefore, all hosts must have access to the storage device before the storage domain can be configured.

7. The Manager can map iSCSI targets to LUNs or LUNs to iSCSI targets. The New Domain window automatically displays known targets with unused LUNs when the iSCSI storage type is selected. If the target that you are using to add storage does not appear, you can use target discovery to find it; otherwise proceed to the next step.

   1. Click Discover Targets to enable target discovery options. When targets have been discovered and logged in to, the New Domain window automatically displays targets with LUNs unused by the environment.

      Note

      LUNs used externally for the environment are also displayed.

      You can use the Discover Targets options to add LUNs on many targets or multiple paths to the same LUNs.

      Important

      If you use the REST API method discoveriscsi to discover the iscsi targets, you can use an FQDN or an IP address, but you must use the iscsi details from the discovered targets results to log in using the REST API method iscsilogin. See discoveriscsi in the REST API Guide for more information.

   2. Enter the FQDN or IP address of the iSCSI host in the Address field.
   3. Enter the port with which to connect to the host when browsing for targets in the Port field. The default is 3260.

   4. If CHAP is used to secure the storage, select the User Authentication check box. Enter the CHAP user name and CHAP password.

      Note

      You can define credentials for an iSCSI target for a specific host with the REST API. See StorageServerConnectionExtensions: add in the REST API Guide for more information.

   5. Click Discover.

   6. Select one or more targets from the discovery results and click Login for one target or Login All for multiple targets.

      Important

      If more than one path access is required, you must discover and log in to the target through all the required paths. Modifying a storage domain to add additional paths is currently not supported.

      Important

      When using the REST API iscsilogin method to log in, you must use the iscsi details from the discovered targets results in the discoveriscsi method. See iscsilogin in the REST API Guide for more information.

8. Click the + button next to the desired target. This expands the entry and displays all unused LUNs attached to the target.
9. Select the check box for each LUN that you are using to create the storage domain.

10. Optionally, you can configure the advanced parameters:

    1. Click Advanced Parameters.
    2. Enter a percentage value into the Warning Low Space Indicator field. If the free space available on the storage domain is below this percentage, warning messages are displayed to the user and logged.
    3. Enter a GB value into the Critical Space Action Blocker field. If the free space available on the storage domain is below this value, error messages are displayed to the user and logged, and any new action that consumes space, even temporarily, will be blocked.
    4. Select the Wipe After Delete check box to enable the wipe after delete option. This option can be edited after the domain is created, but doing so will not change the wipe after delete property of disks that already exist.
    5. Select the Discard After Delete check box to enable the discard after delete option. This option can be edited after the domain is created. This option is only available to block storage domains.
11. Click OK.

Procedure

1. Click Compute Data Centers.
2. Click the data center name. This opens the details view.
3. In the iSCSI Multipathing tab, click Add.
4. In the Add iSCSI Bond window, enter a Name and a Description.
5. Select a logical network from Logical Networks and a storage domain from Storage Targets. You must select all the paths to the same target.
6. Click OK.

Procedure

1. Modify the current logical network so that it is not Required:

   1. Click Compute Clusters.
   2. Click the cluster name. This opens the details view.
   3. In the Logical Networks tab, select the current logical network (net-1) and click Manage Networks.
   4. Clear the Require check box and click OK.

2. Create a new logical network that is not Required and not VM network:

   1. Click Add Network. This opens the New Logical Network window.
   2. In the General tab, enter the Name (net-2) and clear the VM network check box.
   3. In the Cluster tab, clear the Require check box and click OK.

3. Remove the current network bond and reassign the logical networks:

   1. Click Compute Hosts.
   2. Click the host name. This opens the details view.
   3. In the Network Interfaces tab, click Setup Host Networks.
   4. Drag net-1 to the right to unassign it.
   5. Drag the current bond to the right to remove it.
   6. Drag net-1 and net-2 to the left to assign them to physical interfaces.
   7. Click the pencil icon of net-2. This opens the Edit Network window.
   8. In the IPV4 tab, select Static.
   9. Enter the IP and Netmask/Routing Prefix of the subnet and click OK.

4. Create the iSCSI bond:

   1. Click Compute Data Centers.
   2. Click the data center name. This opens the details view.
   3. In the iSCSI Multipathing tab, click Add.
   4. In the Add iSCSI Bond window, enter a Name, select the networks, net-1 and net-2, and click OK.

Procedure

1. Click Storage Domains.
2. Click New Domain.
3. Enter the Name of the storage domain.

4. Select an FCP Data Center from the drop-down list.

   If you do not yet have an appropriate FCP data center, select (none).

5. Select the Domain Function and the Storage Type from the drop-down lists. The storage domain types that are not compatible with the chosen data center are not available.

6. Select an active host in the Host field. If this is not the first data domain in a data center, you must select the data center’s SPM host.

   Important

   All communication to the storage domain is through the selected host and not directly from the Red Hat Virtualization Manager. At least one active host must exist in the system and be attached to the chosen data center. All hosts must have access to the storage device before the storage domain can be configured.

7. The New Domain window automatically displays known targets with unused LUNs when Fibre Channel is selected as the storage type. Select the LUN ID check box to select all of the available LUNs.

8. Optionally, you can configure the advanced parameters.

   1. Click Advanced Parameters.
   2. Enter a percentage value into the Warning Low Space Indicator field. If the free space available on the storage domain is below this percentage, warning messages are displayed to the user and logged.
   3. Enter a GB value into the Critical Space Action Blocker field. If the free space available on the storage domain is below this value, error messages are displayed to the user and logged, and any new action that consumes space, even temporarily, will be blocked.
   4. Select the Wipe After Delete check box to enable the wipe after delete option. This option can be edited after the domain is created, but doing so will not change the wipe after delete property of disks that already exist.
   5. Select the Discard After Delete check box to enable the discard after delete option. This option can be edited after the domain is created. This option is only available to block storage domains.
9. Click OK.

1. Click Storage Domains and select an iSCSI or FCP domain.
2. Click Manage Domain.
3. Click Targets LUNs and click the Discover Targets expansion button.
4. Enter the connection information for the storage server and click Discover to initiate the connection.
5. Click LUNs Targets and select the check box of the newly available LUN.
6. Click OK to add the LUN to the selected storage domain.

1. Click Storage Domains and select an iSCSI or FCP domain.
2. Click Manage Domain.
3. Click LUNs Targets.
4. In the Additional Size column, click Add Additional_Storage_Size button of the LUN to refresh.
5. Click OK to refresh the LUN to indicate the new storage size.

1. Delete the partition mappings in <LUN_ID>:

   kpartx -dv /dev/mapper/<LUN_ID>

2. Erase the fileystem or raid signatures in <LUN_ID>:

   wipefs -a /dev/mapper/<LUN_ID>

3. Inform the operating system about the partition table changes on <LUN_ID>:

    partprobe

Procedure

1. Click Storage Domains.
2. Click the storage domain’s name. This opens the details view.
3. Click the Data Center tab.
4. Click Maintenance, then click OK.
5. Click Detatch, then click OK.
6. Click Remove.
7. Click OK to remove the storage domain from the source environment.
8. Remove the LUN from the storage server.

9. Remove the stale LUNs from the host using Ansible:

   # ansible-playbook --extra-vars "lun=<LUN>" /usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/remove_stale_lun/examples/remove_stale_lun.yml

   where LUN is the LUN removed from the storage server in the steps above.

   Note

   If you remove the stale LUN from the host using Ansible without first removing the LUN from the storage server, the stale LUN will reappear on the host the next time VDSM performs an iSCSI rescan.

Procedure

1. Click Storage Domains.
2. Click Import Domain.
3. Select the Data Center you want to import the storage domain to.
4. Enter a Name for the storage domain.
5. Select the Domain Function and Storage Type from the drop-down lists.

6. Select a host from the Host drop-down list.

   Important

   All communication to the storage domain is through the selected host and not directly from the Red Hat Virtualization Manager. At least one active host must exist in the system and be attached to the chosen data center. All hosts must have access to the storage device before the storage domain can be configured.

7. Enter the details of the storage domain.

   Note

   The fields for specifying the details of the storage domain change depending on the values you select in the Domain Function and Storage Type lists. These fields are the same as those available for adding a new storage domain.

8. Select the Activate Domain in Data Center check box to activate the storage domain after attaching it to the selected data center.
9. Click OK.

Procedure

1. Shut down all virtual machines running on the required storage domain.
2. Click Storage Domains.
3. Click the storage domain’s name. This opens the details view.
4. Click the Data Center tab.
5. Click Maintenance, then click OK.
6. Click Detach, then click OK.
7. Click Attach.
8. Select the destination data center and click OK.

Procedure

1. Log in to the Administration Portal of the source environment.
2. Shut down all virtual machines running on the required storage domain.
3. Click Storage Domains.
4. Click the storage domain’s name. This opens the details view.
5. Click the Data Center tab.
6. Click Maintenance, then click OK.
7. Click Detach, then click OK.
8. Click Remove.
9. In the Remove Storage(s) window, ensure the Format Domain, i.e. Storage Content will be lost! check box is not selected. This step preserves the data in the storage domain for later use.
10. Click OK to remove the storage domain from the source environment.
11. Log in to the Administration Portal of the destination environment.
12. Click Storage Domains.
13. Click Import Domain.
14. Select the destination data center from the Data Center drop-down list.
15. Enter a name for the storage domain.
16. Select the Domain Function and Storage Type from the appropriate drop-down lists.
17. Select a host from the Host drop-down list.

18. Enter the details of the storage domain.

    Note

    The fields for specifying the details of the storage domain change depending on the value you select in the Storage Type drop-down list. These fields are the same as those available for adding a new storage domain.

19. Select the Activate Domain in Data Center check box to automatically activate the storage domain when it is attached.
20. Click OK.

Procedure

1. Click Storage Domains.
2. Click the imported storage domain’s name. This opens the details view.
3. Click the Template Import tab.
4. Select one or more templates to import.
5. Click Import.
6. For each template in the Import Templates(s) window, ensure the correct target cluster is selected in the Cluster list.

7. Map external virtual machine vNIC profiles to profiles that are present on the target cluster(s):

   1. Click vNic Profiles Mapping.
   2. Select the vNIC profile to use from the Target vNic Profile drop-down list.
   3. If multiple target clusters are selected in the Import Templates window, select each target cluster in the Target Cluster drop-down list and ensure the mappings are correct.
   4. Click OK.
8. Click OK.

Procedure

1. Click Storage Disks.
2. Select Start from the Upload menu.
3. Click Choose File and select the image to upload.
4. Fill in the Disk Options fields. See Explanation of Settings in the New Virtual Disk Window for descriptions of the relevant fields.

5. Click OK.

   A progress bar indicates the status of the upload. You can pause, cancel, or resume uploads from the Upload menu.

1. Install the image files on the Manager machine:

   # dnf -y install virtio-win

   After you install it on the Manager machine, the image file is /usr/share/virtio-win/virtio-win_version.iso

2. Upload the image file to a data storage domain that was not created locally during installation. For more information, see Uploading Images to a Data Storage Domain in the Administration Guide.
3. Attach the image file to virtual machines.

Procedure

1. Login as root to the host that belongs to the Data Center where your ISO storage domain resides.

2. Get a directory tree of /rhv/data-center:

   # tree /rhev/data-center
   .
   |-- 80dfacc7-52dd-4d75-ab82-4f9b8423dc8b
   |   |-- 76d1ecba-b61d-45a4-8eb5-89ab710a6275  /rhev/data-center/mnt/10.10.10.10:_rhevnfssd/76d1ecba-b61d-45a4-8eb5-89ab710a6275
   |   |-- b835cd1c-111c-468d-ba70-fec5346af227  /rhev/data-center/mnt/10.10.10.10:_rhevisosd/b835cd1c-111c-468d-ba70-fec5346af227
   |   |-- mastersd  76d1ecba-b61d-45a4-8eb5-89ab710a6275
   |   |-- tasks  mastersd/master/tasks
   |   `-- vms  mastersd/master/vms
   |-- hsm-tasks
   `-- mnt
       |-- 10.10.10.10:_rhevisosd
       |   |-- b835cd1c-111c-468d-ba70-fec5346af227
       |   |   |-- dom_md
       |   |   |   |-- ids
       |   |   |   |-- inbox
       |   |   |   |-- leases
       |   |   |   |-- metadata
       |   |   |   `-- outbox
       |   |   `-- images
       |   |       `-- 11111111-1111-1111-1111-111111111111
       |   `-- lost+found [error opening dir]
   
   (output trimmed)

3. Securely copy the image from the source location into the full path of 11111111-1111-1111-1111-111111111111:

   # scp root@isosource:/isos/example.iso /rhev/data-center/mnt/10.96.4.50:_rhevisosd/b835cd1c-111c-468d-ba70-fec5346af227/images/11111111-1111-1111-1111-111111111111

4. File permissions for the newly copied ISO image should be 36:36 (vdsm:kvm). If they are not, change user and group ownership of the ISO file to 36:36 (vdsm’s user and group):

   # cd /rhev/data-center/mnt/10.96.4.50:_rhevisosd/b835cd1c-111c-468d-ba70-fec5346af227/images/11111111-1111-1111-1111-111111111111
   # chown 36.36 example.iso

Procedure

1. Shut down all the virtual machines running on the storage domain.
2. Click Storage Domains.
3. Click the storage domain’s name. This opens the details view.
4. Click the Data Center tab.

5. Click Maintenance.

   Note

   The Ignore OVF update failure check box allows the storage domain to go into maintenance mode even if the OVF update fails.

6. Click OK.

Editing an Active Storage Domain*

1. Click Storage Domains and select a storage domain.
2. Click Manage Domain.
3. Edit the available fields as required.
4. Click OK.

Editing an Inactive Storage Domain

1. Click Storage Domains.

2. If the storage domain is active, move it to maintenance mode:

   1. Click the storage domain’s name. This opens the details view.
   2. Click the Data Center tab.
   3. Click Maintenance.
   4. Click OK.
3. Click Manage Domain.
4. Edit the storage path and other details as required. The new connection details must be of the same storage type as the original connection.
5. Click OK.

6. Activate the storage domain:

   1. Click the storage domain’s name. This opens the details view.
   2. Click the Data Center tab.
   3. Click Activate.

Procedure

1. Click Storage Domains.

2. Select the storage domain and click More Actions ( ), then click Update OVFs.

   The OVFs are updated and a message appears in Events.

1. Click Storage Domains.
2. Click an inactive storage domain’s name. This opens the details view.
3. Click the Data Centers tab.
4. Click Activate.

Procedure

1. Click Storage Domains.
2. Click the storage domain’s name. This opens the details view.
3. Click the Data Center tab.
4. Click Maintenance.
5. Click OK to initiate maintenance mode.
6. Click Detach.
7. Click OK to detach the storage domain.

Procedure

1. Click Storage Domains.
2. Click the storage domain’s name. This opens the details view.
3. Click the Data Center tab.
4. Click Attach.
5. Select the appropriate data center.
6. Click OK.

Procedure

1. Click Storage Domains.

2. Move the storage domain to maintenance mode and detach it:

   1. Click the storage domain’s name. This opens the details view.
   2. Click the Data Center tab.
   3. Click Maintenance, then click OK.
   4. Click Detach, then click OK.
3. Click Remove.
4. Optionally select the Format Domain, i.e. Storage Content will be lost! check box to erase the content of the domain.
5. Click OK.

Procedure

1. Click Storage Domains.
2. Select the storage domain and click More Actions ( ), then click Destroy.
3. Select the Approve operation check box.
4. Click OK.

Procedure

1. Click Storage Domains.
2. Click the data storage domain’s name. This opens the details view.
3. Click the Disk Profiles tab.
4. Click New.
5. Enter a Name and a Description for the disk profile.
6. Select the quality of service to apply to the disk profile from the QoS list.
7. Click OK.

Procedure

1. Click Storage Domains.
2. Click the data storage domain’s name. This opens the details view.
3. Click the Disk Profiles tab.
4. Select the disk profile to remove.
5. Click Remove.
6. Click OK.

Procedure

1. On the Manager machine enable the required maximum number of hosts:

   # engine-config -s MaxNumberOfHostsInStoragePool=NUMBER_OF_HOSTS

2. Restart the JBoss Application Server:

   # service jboss-as restart

Procedure

1. Ensure that 4K block support is enabled.

   $ vdsm-client Host getCapabilities
   …​
   {
       "GLUSTERFS" : [
            0,
            512,
            4096,
       ]
       …​

2. Edit /etc/vdsm/vdsm.conf.d/gluster.conf and set enable_4k_storage to false. For example:

   $ vi  /etc/vdsm/vdsm.conf.d/gluster.conf
   
   [gluster]
   # Use to disable 4k support
   # if needed.
   enable_4k_storage = false

Procedure

1. In the Administration Portal, click Storage Storage Domain and click the name of a storage domain.
2. Click Manage Domain. The Manage Domains dialog box opens.
3. Expand Advanced Parameters.
4. For Warning Low Space Indicator (%) enter a percentage value. When the available space in the storage domain reaches this value, the Manager alerts you that the domain is nearing capacity.
5. For Critical Space Action Blocker (GB), enter a value in gigabytes. When the available space in the storage domain reaches this value, the Manager shuts down.
6. For Warning Low Confirmed Space Indicator (%) enter a percentage value. When the available space in the storage domain reaches this value, the Manager alerts you that the actual space available to write data is nearing capacity.