Search
SupportConsoleDevelopersStart a trialContact

Chapter 9. Configuring RAID logical volumes

download PDF

You can create and manage Redundant Array of Independent Disks (RAID) volumes by using logical volume manager (LVM).

9.1. RAID logical volumes

Logical volume manager (LVM) supports Redundant Array of Independent Disks (RAID) levels 0, 1, 4, 5, 6, and 10. An LVM RAID volume has the following characteristics:

  • LVM creates and manages RAID logical volumes that leverage the Multiple Devices (MD) kernel drivers.
  • You can temporarily split RAID1 images from the array and merge them back into the array later.
  • LVM RAID volumes support snapshots.

Other characteristics include:

Clusters

RAID logical volumes are not cluster-aware.

Although you can create and activate RAID logical volumes exclusively on one machine, you cannot activate them simultaneously on more than one machine.

Subvolumes

When you create a RAID logical volume (LV), LVM creates a metadata subvolume that is one extent in size for every data or parity subvolume in the array.

For example, creating a 2-way RAID1 array results in two metadata subvolumes (lv_rmeta_0 and lv_rmeta_1) and two data subvolumes (lv_rimage_0 and lv_rimage_1). Similarly, creating a 3-way stripe and one implicit parity device, RAID4 results in four metadata subvolumes (lv_rmeta_0, lv_rmeta_1, lv_rmeta_2, and lv_rmeta_3) and four data subvolumes (lv_rimage_0, lv_rimage_1, lv_rimage_2, and lv_rimage_3).

Integrity
You can lose data when a RAID device fails or when soft corruption occurs. Soft corruption in data storage implies that the data retrieved from a storage device is different from the data written to that device. Adding integrity to a RAID LV reduces or prevent soft corruption. For more information, see Creating a RAID LV with DM integrity.

9.2. RAID levels and linear support

The following are the supported configurations by RAID, including levels 0, 1, 4, 5, 6, 10, and linear:

Level 0

RAID level 0, often called striping, is a performance-oriented striped data mapping technique. This means the data being written to the array is broken down into stripes and written across the member disks of the array, allowing high I/O performance at low inherent cost but provides no redundancy.

RAID level 0 implementations only stripe the data across the member devices up to the size of the smallest device in the array. This means that if you have multiple devices with slightly different sizes, each device gets treated as though it was the same size as the smallest drive. Therefore, the common storage capacity of a level 0 array is the total capacity of all disks. If the member disks have a different size, then the RAID0 uses all the space of those disks using the available zones.

Level 1

RAID level 1, or mirroring, provides redundancy by writing identical data to each member disk of the array, leaving a mirrored copy on each disk. Mirroring remains popular due to its simplicity and high level of data availability. Level 1 operates with two or more disks, and provides very good data reliability and improves performance for read-intensive applications but at relatively high costs.

RAID level 1 is costly because you write the same information to all of the disks in the array, which provides data reliability, but in a much less space-efficient manner than parity based RAID levels such as level 5. However, this space inefficiency comes with a performance benefit, which is parity-based RAID levels that consume considerably more CPU power in order to generate the parity while RAID level 1 simply writes the same data more than once to the multiple RAID members with very little CPU overhead. As such, RAID level 1 can outperform the parity-based RAID levels on machines where software RAID is employed and CPU resources on the machine are consistently taxed with operations other than RAID activities.

The storage capacity of the level 1 array is equal to the capacity of the smallest mirrored hard disk in a hardware RAID or the smallest mirrored partition in a software RAID. Level 1 redundancy is the highest possible among all RAID types, with the array being able to operate with only a single disk present.

Level 4

Level 4 uses parity concentrated on a single disk drive to protect data. Parity information is calculated based on the content of the rest of the member disks in the array. This information can then be used to reconstruct data when one disk in the array fails. The reconstructed data can then be used to satisfy I/O requests to the failed disk before it is replaced and to repopulate the failed disk after it has been replaced.

Since the dedicated parity disk represents an inherent bottleneck on all write transactions to the RAID array, level 4 is seldom used without accompanying technologies such as write-back caching. Or it is used in specific circumstances where the system administrator is intentionally designing the software RAID device with this bottleneck in mind such as an array that has little to no write transactions once the array is populated with data. RAID level 4 is so rarely used that it is not available as an option in Anaconda. However, it could be created manually by the user if needed.

The storage capacity of hardware RAID level 4 is equal to the capacity of the smallest member partition multiplied by the number of partitions minus one. The performance of a RAID level 4 array is always asymmetrical, which means reads outperform writes. This is because write operations consume extra CPU resources and main memory bandwidth when generating parity, and then also consume extra bus bandwidth when writing the actual data to disks because you are not only writing the data, but also the parity. Read operations need only read the data and not the parity unless the array is in a degraded state. As a result, read operations generate less traffic to the drives and across the buses of the computer for the same amount of data transfer under normal operating conditions.

Level 5

This is the most common type of RAID. By distributing parity across all the member disk drives of an array, RAID level 5 eliminates the write bottleneck inherent in level 4. The only performance bottleneck is the parity calculation process itself. Modern CPUs can calculate parity very fast. However, if you have a large number of disks in a RAID 5 array such that the combined aggregate data transfer speed across all devices is high enough, parity calculation can be a bottleneck.

Level 5 has asymmetrical performance, and reads substantially outperforming writes. The storage capacity of RAID level 5 is calculated the same way as with level 4.

Level 6

This is a common level of RAID when data redundancy and preservation, and not performance, are the paramount concerns, but where the space inefficiency of level 1 is not acceptable. Level 6 uses a complex parity scheme to be able to recover from the loss of any two drives in the array. This complex parity scheme creates a significantly higher CPU burden on software RAID devices and also imposes an increased burden during write transactions. As such, level 6 is considerably more asymmetrical in performance than levels 4 and 5.

The total capacity of a RAID level 6 array is calculated similarly to RAID level 5 and 4, except that you must subtract two devices instead of one from the device count for the extra parity storage space.

Level 10

This RAID level attempts to combine the performance advantages of level 0 with the redundancy of level 1. It also reduces some of the space wasted in level 1 arrays with more than two devices. With level 10, it is possible, for example, to create a 3-drive array configured to store only two copies of each piece of data, which then allows the overall array size to be 1.5 times the size of the smallest devices instead of only equal to the smallest device, similar to a 3-device, level 1 array. This avoids CPU process usage to calculate parity similar to RAID level 6, but it is less space efficient.

The creation of RAID level 10 is not supported during installation. It is possible to create one manually after installation.

Linear RAID

Linear RAID is a grouping of drives to create a larger virtual drive.

In linear RAID, the chunks are allocated sequentially from one member drive, going to the next drive only when the first is completely filled. This grouping provides no performance benefit, as it is unlikely that any I/O operations split between member drives. Linear RAID also offers no redundancy and decreases reliability. If any one member drive fails, the entire array cannot be used and data can be lost. The capacity is the total of all member disks.

9.3. LVM RAID segment types

To create a RAID logical volume, you can specify a RAID type by using the --type argument of the lvcreate command. For most users, specifying one of the five available primary types, which are raid1, raid4, raid5, raid6, and raid10, should be sufficient.

The following table describes the possible RAID segment types.

Table 9.1. LVM RAID segment types
Segment typeDescription

raid1

RAID1 mirroring. This is the default value for the --type argument of the lvcreate command, when you specify the -m argument without specifying striping.

raid4

RAID4 dedicated parity disk.

raid5_la

  • RAID5 left asymmetric.
  • Rotating parity 0 with data continuation.

raid5_ra

  • RAID5 right asymmetric.
  • Rotating parity N with data continuation.

raid5_ls

  • RAID5 left symmetric.
  • It is same as raid5.
  • Rotating parity 0 with data restart.

raid5_rs

  • RAID5 right symmetric.
  • Rotating parity N with data restart.

raid6_zr

  • RAID6 zero restart.
  • It is same as raid6.
  • Rotating parity zero (left-to-right) with data restart.

raid6_nr

  • RAID6 N restart.
  • Rotating parity N (left-to-right) with data restart.

raid6_nc

  • RAID6 N continue.
  • Rotating parity N (left-to-right) with data continuation.

raid10

  • Striped mirrors. This is the default value for the --type argument of the lvcreate command if you specify the -m argument along with the number of stripes that is greater than 1.
  • Striping of mirror sets.

raid0/raid0_meta

Striping. RAID0 spreads logical volume data across multiple data subvolumes in units of stripe size. This is used to increase performance. Logical volume data is lost if any of the data subvolumes fail.

9.4. Creating RAID logical volumes

You can create RAID1 arrays with multiple numbers of copies, according to the value you specify for the -m argument. Similarly, you can specify the number of stripes for a RAID 0, 4, 5, 6, and 10 logical volume with the -i argument. You can also specify the stripe size with the -I argument. The following procedure describes different ways to create different types of RAID logical volume.

Procedure

  • Create a 2-way RAID. The following command creates a 2-way RAID1 array, named my_lv, in the volume group my_vg, that is 1G in size: 

    # lvcreate --type raid1 -m 1 -L 1G -n my_lv my_vg
Logical volume "my_lv" created.

  • Create a RAID5 array with stripes. The following command creates a RAID5 array with three stripes and one implicit parity drive, named my_lv, in the volume group my_vg, that is 1G in size. Note that you can specify the number of stripes similar to an LVM striped volume. The correct number of parity drives is added automatically. 

    # lvcreate --type raid5 -i 3 -L 1G -n my_lv my_vg

  • Create a RAID6 array with stripes. The following command creates a RAID6 array with three 3 stripes and two implicit parity drives, named my_lv, in the volume group my_vg, that is 1G one gigabyte in size: 

    # lvcreate --type raid6 -i 3 -L 1G -n my_lv my_vg

Verification

  • Display the LVM device my_vg/my_lv, which is a 2-way RAID1 array: 
# lvs -a -o name,copy_percent,devices _my_vg_
  LV                Copy%  Devices
  my_lv             6.25    my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]         /dev/sde1(0)
  [my_lv_rimage_1]         /dev/sdf1(1)
  [my_lv_rmeta_0]          /dev/sde1(256)
  [my_lv_rmeta_1]          /dev/sdf1(0)

Additional resources

  • lvcreate(8) and lvmraid(7) man pages

9.5. Creating a RAID0 striped logical volume

A RAID0 logical volume spreads logical volume data across multiple data subvolumes in units of stripe size. The following procedure creates an LVM RAID0 logical volume called mylv that stripes data across the disks.

Prerequisites

  1. You have created three or more physical volumes. For more information about creating physical volumes, see Creating LVM physical volume.
  2. You have created the volume group. For more information, see Creating LVM volume group.

Procedure

  1. Create a RAID0 logical volume from the existing volume group. The following command creates the RAID0 volume mylv from the volume group myvg, which is 2G in size, with three stripes and a stripe size of 4kB: 

    # lvcreate --type raid0 -L 2G --stripes 3 --stripesize 4 -n mylv my_vg
  Rounding size 2.00 GiB (512 extents) up to stripe boundary size 2.00 GiB(513 extents).
  Logical volume "mylv" created.

  2. Create a file system on the RAID0 logical volume. The following command creates an ext4 file system on the logical volume: 

    # mkfs.ext4 /dev/my_vg/mylv

  3. Mount the logical volume and report the file system disk space usage: 

    # mount /dev/my_vg/mylv /mnt

# df
Filesystem             1K-blocks     Used  Available  Use% Mounted on
/dev/mapper/my_vg-mylv   2002684     6168  1875072    1%   /mnt

Verification

  • View the created RAID0 stripped logical volume: 

    # lvs -a -o +devices,segtype my_vg
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices Type
  mylv my_vg rwi-a-r--- 2.00g mylv_rimage_0(0),mylv_rimage_1(0),mylv_rimage_2(0) raid0
  [mylv_rimage_0] my_vg iwi-aor--- 684.00m /dev/sdf1(0) linear
  [mylv_rimage_1] my_vg iwi-aor--- 684.00m /dev/sdg1(0) linear
  [mylv_rimage_2] my_vg iwi-aor--- 684.00m /dev/sdh1(0) linear

9.6. Configuring a stripe size for RAID LVM volumes by using the storage RHEL system role

With the storage system role, you can configure a stripe size for RAID LVM volumes on RHEL by using Red Hat Ansible Automation Platform. You can set up an Ansible playbook with the available parameters to configure an LVM pool with RAID.

Prerequisites

Procedure

  1. Create a playbook file, for example ~/playbook.yml, with the following content: 

    ---
- name: Configure stripe size for RAID LVM volumes
  hosts: managed-node-01.example.com
  roles:
    - rhel-system-roles.storage
  vars:
    storage_safe_mode: false
    storage_pools:
      - name: my_pool
        type: lvm
        disks: [sdh, sdi]
        volumes:
          - name: my_volume
            size: "1 GiB"
            mount_point: "/mnt/app/shared"
            fs_type: xfs
            raid_level: raid1
            raid_stripe_size: "256 KiB"
            state: present

  2. Validate the playbook syntax: 

    $ ansible-playbook --syntax-check ~/playbook.yml

    Note that this command only validates the syntax and does not protect against a wrong but valid configuration.

  3. Run the playbook: 

    $ ansible-playbook ~/playbook.yml

Additional resources

  • /usr/share/ansible/roles/rhel-system-roles.storage/README.md file
  • /usr/share/doc/rhel-system-roles/storage/ directory
  • Managing RAID

9.7. Parameters for creating a RAID0

You can create a RAID0 striped logical volume using the lvcreate --type raid0[meta] --stripes _Stripes --stripesize StripeSize VolumeGroup [PhysicalVolumePath] command.

The following table describes different parameters, which you can use while creating a RAID0 striped logical volume.

Table 9.2. Parameters for creating a RAID0 striped logical volume
ParameterDescription

--type raid0[_meta]

Specifying raid0 creates a RAID0 volume without metadata volumes. Specifying raid0_meta creates a RAID0 volume with metadata volumes. Since RAID0 is non-resilient, it does not store any mirrored data blocks as RAID1/10 or calculate and store any parity blocks as RAID4/5/6 do. Hence, it does not need metadata volumes to keep state about resynchronization progress of mirrored or parity blocks. Metadata volumes become mandatory on a conversion from RAID0 to RAID4/5/6/10. Specifying raid0_meta preallocates those metadata volumes to prevent a respective allocation failure.

--stripes Stripes

Specifies the number of devices to spread the logical volume across.

--stripesize StripeSize

Specifies the size of each stripe in kilobytes. This is the amount of data that is written to one device before moving to the next device.

VolumeGroup

Specifies the volume group to use.

PhysicalVolumePath

Specifies the devices to use. If this is not specified, LVM will choose the number of devices specified by the Stripes option, one for each stripe.

9.8. Soft data corruption

Soft corruption in data storage implies that the data retrieved from a storage device is different from the data written to that device. The corrupted data can exist indefinitely on storage devices. You might not discover this corrupted data until you retrieve and attempt to use this data.

Depending on the type of configuration, a Redundant Array of Independent Disks (RAID) logical volume(LV) prevents data loss when a device fails. If a device consisting of a RAID array fails, the data can be recovered from other devices that are part of that RAID LV. However, a RAID configuration does not ensure the integrity of the data itself. Soft corruption, silent corruption, soft errors, and silent errors are terms that describe data that has become corrupted, even if the system design and software continues to function as expected.

Device mapper (DM) integrity is used with RAID levels 1, 4, 5, 6, and 10 to mitigate or prevent data loss due to soft corruption. The RAID layer ensures that a non-corrupted copy of the data can fix the soft corruption errors. The integrity layer sits above each RAID image while an extra sub LV stores the integrity metadata or data checksums for each RAID image. When you retrieve data from an RAID LV with integrity, the integrity data checksums analyze the data for corruption. If corruption is detected, the integrity layer returns an error message, and the RAID layer retrieves a non-corrupted copy of the data from another RAID image. The RAID layer automatically rewrites non-corrupted data over the corrupted data to repair the soft corruption.

When creating a new RAID LV with DM integrity or adding integrity to an existing RAID LV, consider the following points:

  • The integrity metadata requires additional storage space. For each RAID image, every 500MB data requires 4MB of additional storage space because of the checksums that get added to the data.
  • While some RAID configurations are impacted more than others, adding DM integrity impacts performance due to latency when accessing the data. A RAID1 configuration typically offers better performance than RAID5 or its variants.
  • The RAID integrity block size also impacts performance. Configuring a larger RAID integrity block size offers better performance. However, a smaller RAID integrity block size offers greater backward compatibility.
  • There are two integrity modes available: bitmap or journal. The bitmap integrity mode typically offers better performance than journal mode.
Tip

If you experience performance issues, either use RAID1 with integrity or test the performance of a particular RAID configuration to ensure that it meets your requirements.

9.9. Creating a RAID LV with DM integrity

When you create a RAID LV with device mapper (DM) integrity or add integrity to an existing RAID LV, it mitigates the risk of losing data due to soft corruption. Wait for the integrity synchronization and the RAID metadata to complete before using the LV. Otherwise, the background initialization might impact the LV’s performance.

Procedure

  1. Create a RAID LV with DM integrity. The following example creates a new RAID LV with integrity named test-lv in the my_vg volume group, with a usable size of 256M and RAID level 1: 

    # lvcreate --type raid1 --raidintegrity y -L 256M -n test-lv my_vg
Creating integrity metadata LV test-lv_rimage_0_imeta with size 8.00 MiB.
Logical volume "test-lv_rimage_0_imeta" created.
Creating integrity metadata LV test-lv_rimage_1_imeta with size 8.00 MiB.
Logical volume "test-lv_rimage_1_imeta" created.
Logical volume "test-lv" created.
    Note

    To add DM integrity to an existing RAID LV, use the following command: 

    # lvconvert --raidintegrity y my_vg/test-lv

    Adding integrity to a RAID LV limits the number of operations that you can perform on that RAID LV.

  2. Optional: Remove the integrity before performing certain operations. 

    # lvconvert --raidintegrity n my_vg/test-lv
Logical volume my_vg/test-lv has removed integrity.

Verification

  • View information about the added DM integrity:

    • View information about the test-lv RAID LV that was created in the my_vg volume group: 

      # lvs -a my_vg
  LV                        VG      Attr       LSize   Origin                 Cpy%Sync
  test-lv                   my_vg rwi-a-r--- 256.00m                          2.10
  [test-lv_rimage_0]        my_vg gwi-aor--- 256.00m [test-lv_rimage_0_iorig] 93.75
  [test-lv_rimage_0_imeta]  my_vg ewi-ao----   8.00m
  [test-lv_rimage_0_iorig]  my_vg -wi-ao---- 256.00m
  [test-lv_rimage_1]        my_vg gwi-aor--- 256.00m [test-lv_rimage_1_iorig] 85.94
 [...]

      The following describes different options from this output:

      g attribute
      It is the list of attributes under the Attr column indicates that the RAID image is using integrity. The integrity stores the checksums in the _imeta RAID LV.
      Cpy%Sync column
      It indicates the synchronization progress for both the top level RAID LV and for each RAID image.
      RAID image
      It is is indicated in the LV column by raid_image_N.
      LV column
      It ensures that the synchronization progress displays 100% for the top level RAID LV and for each RAID image.

    • Display the type for each RAID LV: 

      # lvs -a my-vg -o+segtype
  LV                       VG      Attr       LSize   Origin                 Cpy%Sync Type
  test-lv                  my_vg rwi-a-r--- 256.00m                          87.96    raid1
  [test-lv_rimage_0]       my_vg gwi-aor--- 256.00m [test-lv_rimage_0_iorig] 100.00   integrity
  [test-lv_rimage_0_imeta] my_vg ewi-ao----   8.00m                                   linear
  [test-lv_rimage_0_iorig] my_vg -wi-ao---- 256.00m                                   linear
  [test-lv_rimage_1]       my_vg gwi-aor--- 256.00m [test-lv_rimage_1_iorig] 100.00   integrity
 [...]

    • There is an incremental counter that counts the number of mismatches detected on each RAID image. View the data mismatches detected by integrity from rimage_0 under my_vg/test-lv: 

      # lvs -o+integritymismatches my_vg/test-lv_rimage_0
  LV                 VG      Attr       LSize   Origin                    Cpy%Sync IntegMismatches
  [test-lv_rimage_0] my_vg gwi-aor--- 256.00m [test-lv_rimage_0_iorig]    100.00                 0

      In this example, the integrity has not detected any data mismatches and thus the IntegMismatches counter shows zero (0).

    • View the data integrity information in the /var/log/messages log files, as shown in the following examples:

      Example 9.1. Example of dm-integrity mismatches from the kernel message logs

      device-mapper: integrity: dm-12: Checksum failed at sector 0x24e7

      Example 9.2. Example of dm-integrity data corrections from the kernel message logs

      md/raid1:mdX: read error corrected (8 sectors at 9448 on dm-16)

Additional resources

  • lvcreate(8) and lvmraid(7) man pages

9.10. Minimum and maximum I/O rate options

When you create a RAID logical volumes, the background I/O required to initialize the logical volumes with the sync operation can expel other I/O operations to LVM devices, such as updates to volume group metadata, particularly when you are creating many RAID logical volumes. This can cause the other LVM operations to slow down.

You can control the rate at which a RAID logical volume is initialized by implementing recovery throttling. To control the rate at which sync operations are performed, set the minimum and maximum I/O rate for those operations with the --minrecoveryrate and --maxrecoveryrate options of the lvcreate command.

You can specify these options as follows:

--maxrecoveryrate Rate[bBsSkKmMgG]
Sets the maximum recovery rate for a RAID logical volume so that it will not expel nominal I/O operations. Specify the Rate as an amount per second for each device in the array. If you do not provide a suffix, then it assumes kiB/sec/device. Setting the recovery rate to 0 means it will be unbounded.
--minrecoveryrate Rate[bBsSkKmMgG]
Sets the minimum recovery rate for a RAID logical volume to ensure that I/O for sync operations achieves a minimum throughput, even when heavy nominal I/O is present. Specify the Rate as an amount per second for each device in the array. If you do not give a suffix, then it assumes kiB/sec/device.

For example, use the lvcreate --type raid10 -i 2 -m 1 -L 10G --maxrecoveryrate 128 -n my_lv my_vg command to create a 2-way RAID10 array my_lv, which is in the volume group my_vg with 3 stripes that is 10G in size with a maximum recovery rate of 128 kiB/sec/device. You can also specify minimum and maximum recovery rates for a RAID scrubbing operation.

9.11. Converting a linear device to a RAID logical volume

You can convert an existing linear logical volume to a RAID logical volume. To perform this operation, use the --type argument of the lvconvert command.

RAID logical volumes are composed of metadata and data subvolume pairs. When you convert a linear device to a RAID1 array, it creates a new metadata subvolume and associates it with the original logical volume on one of the same physical volumes that the linear volume is on. The additional images are added in a metadata/data subvolume pair. If the metadata image that pairs with the original logical volume cannot be placed on the same physical volume, the lvconvert fails.

Procedure

  1. View the logical volume device that needs to be converted: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV     Copy%  Devices
  my_lv         /dev/sde1(0)

  2. Convert the linear logical volume to a RAID device. The following command converts the linear logical volume my_lv in volume group __my_vg, to a 2-way RAID1 array: 

    # lvconvert --type raid1 -m 1 my_vg/my_lv
  Are you sure you want to convert linear LV my_vg/my_lv to raid1 with 2 images enhancing resilience? [y/n]: y
  Logical volume my_vg/my_lv successfully converted.

Verification

  • Ensure if the logical volume is converted to a RAID device: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv            6.25   my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]        /dev/sde1(0)
  [my_lv_rimage_1]        /dev/sdf1(1)
  [my_lv_rmeta_0]         /dev/sde1(256)
  [my_lv_rmeta_1]         /dev/sdf1(0)

Additional resources

  • The lvconvert(8) man page

9.12. Converting an LVM RAID1 logical volume to an LVM linear logical volume

You can convert an existing RAID1 LVM logical volume to an LVM linear logical volume. To perform this operation, use the lvconvert command and specify the -m0 argument. This removes all the RAID data subvolumes and all the RAID metadata subvolumes that make up the RAID array, leaving the top-level RAID1 image as the linear logical volume.

Procedure

  1. Display an existing LVM RAID1 logical volume: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]        /dev/sde1(1)
  [my_lv_rimage_1]        /dev/sdf1(1)
  [my_lv_rmeta_0]         /dev/sde1(0)
  [my_lv_rmeta_1]         /dev/sdf1(0)

  2. Convert an existing RAID1 LVM logical volume to an LVM linear logical volume. The following command converts the LVM RAID1 logical volume my_vg/my_lv to an LVM linear device: 

    # lvconvert -m0 my_vg/my_lv
  Are you sure you want to convert raid1 LV my_vg/my_lv to type linear losing all resilience? [y/n]: y
  Logical volume my_vg/my_lv successfully converted.

    When you convert an LVM RAID1 logical volume to an LVM linear volume, you can also specify which physical volumes to remove. In the following example, the lvconvert command specifies that you want to remove /dev/sde1, leaving /dev/sdf1 as the physical volume that makes up the linear device: 

    # lvconvert -m0 my_vg/my_lv /dev/sde1

Verification

  • Verify if the RAID1 logical volume was converted to an LVM linear device: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV    Copy%  Devices
  my_lv        /dev/sdf1(1)

Additional resources

  • The lvconvert(8) man page

9.13. Converting a mirrored LVM device to a RAID1 logical volume

You can convert an existing mirrored LVM device with a segment type mirror to a RAID1 LVM device. To perform this operation, use the lvconvert command with the --type raid1 argument. This renames the mirror subvolumes named mimage to RAID subvolumes named rimage.

In addition, it also removes the mirror log and creates metadata subvolumes named rmeta for the data subvolumes on the same physical volumes as the corresponding data subvolumes.

Procedure

  1. View the layout of a mirrored logical volume my_vg/my_lv: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv             15.20 my_lv_mimage_0(0),my_lv_mimage_1(0)
  [my_lv_mimage_0]        /dev/sde1(0)
  [my_lv_mimage_1]        /dev/sdf1(0)
  [my_lv_mlog]            /dev/sdd1(0)

  2. Convert the mirrored logical volume my_vg/my_lv to a RAID1 logical volume: 

    # lvconvert --type raid1 my_vg/my_lv
Are you sure you want to convert mirror LV my_vg/my_lv to raid1 type? [y/n]: y
Logical volume my_vg/my_lv successfully converted.

Verification

  • Verify if the mirrored logical volume is converted to a RAID1 logical volume: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]        /dev/sde1(0)
  [my_lv_rimage_1]        /dev/sdf1(0)
  [my_lv_rmeta_0]         /dev/sde1(125)
  [my_lv_rmeta_1]         /dev/sdf1(125)

Additional resources

  • The lvconvert(8) man page

9.14. Commands to resize a RAID logical volume

You can resize a RAID logical volume in the following ways:

  • You can increase the size of a RAID logical volume of any type with the lvresize or lvextend command. This does not change the number of RAID images. For striped RAID logical volumes, the same stripe rounding constraints apply when you create a striped RAID logical volume.
  • You can reduce the size of a RAID logical volume of any type with the lvresize or lvreduce command. This does not change the number of RAID images. As with the lvextend command, the same stripe rounding constraints apply when you create a striped RAID logical volume.
  • You can change the number of stripes on a striped RAID logical volume such as RAID4, RAID5, RAID6, or RAID10 with the --stripes N parameter of the lvconvert command. This increases or reduces the size of the RAID logical volume by the capacity of the stripes added or removed. Note that raid10 volumes are capable only of adding stripes. This capability is part of the RAID reshaping feature and with this feature, you can change attributes of a RAID logical volume while keeping the same RAID level.

9.15. Changing the number of images in an existing RAID1 device

You can change the number of images in an existing RAID1 array, similar to the way you can change the number of images in the implementation of LVM mirroring.

When you add images to a RAID1 logical volume with the lvconvert command, you can perform the following operations:

  • specify the total number of images for the resulting device,
  • how many images to add to the device, and
  • can optionally specify on which physical volumes the new metadata/data image pairs reside.

Procedure

  1. Display the LVM device my_vg/my_lv, which is a 2-way RAID1 array: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV                Copy%  Devices
  my_lv             6.25    my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]         /dev/sde1(0)
  [my_lv_rimage_1]         /dev/sdf1(1)
  [my_lv_rmeta_0]          /dev/sde1(256)
  [my_lv_rmeta_1]          /dev/sdf1(0)

    Metadata subvolumes named rmeta always exist on the same physical devices as their data subvolume counterparts rimage. The metadata/data subvolume pairs will not be created on the same physical volumes as those from another metadata/data subvolume pair in the RAID array unless you specify --alloc anywhere.

  2. Convert the 2-way RAID1 logical volume my_vg/my_lv to a 3-way RAID1 logical volume: 

    # lvconvert -m 2 my_vg/my_lv
Are you sure you want to convert raid1 LV my_vg/my_lv to 3 images enhancing resilience? [y/n]: y
Logical volume my_vg/my_lv successfully converted.

    The following are a few examples of changing the number of images in an existing RAID1 device:

    • You can also specify which physical volumes to use while adding an image to RAID. The following command converts the 2-way RAID1 logical volume my_vg/my_lv to a 3-way RAID1 logical volume by specifying the physical volume /dev/sdd1 to use for the array: 

      # lvconvert -m 2 my_vg/my_lv /dev/sdd1

    • Convert the 3-way RAID1 logical volume into a 2-way RAID1 logical volume: 

      # lvconvert -m1 my_vg/my_lv
Are you sure you want to convert raid1 LV my_vg/my_lv to 2 images reducing resilience? [y/n]: y
Logical volume my_vg/my_lv successfully converted.

    • Convert the 3-way RAID1 logical volume into a 2-way RAID1 logical volume by specifying the physical volume /dev/sde1, which contains the image to remove: 

      # lvconvert -m1 my_vg/my_lv /dev/sde1

      Additionally, when you remove an image and its associated metadata subvolume volume, any higher-numbered images will be shifted down to fill the slot. Removing lv_rimage_1 from a 3-way RAID1 array that consists of lv_rimage_0, lv_rimage_1, and lv_rimage_2 results in a RAID1 array that consists of lv_rimage_0 and lv_rimage_1. The subvolume lv_rimage_2 will be renamed and take over the empty slot, becoming lv_rimage_1.

Verification

  • View the RAID1 device after changing the number of images in an existing RAID1 device: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV Cpy%Sync Devices
  my_lv 100.00 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0] /dev/sdd1(1)
  [my_lv_rimage_1] /dev/sde1(1)
  [my_lv_rimage_2] /dev/sdf1(1)
  [my_lv_rmeta_0] /dev/sdd1(0)
  [my_lv_rmeta_1] /dev/sde1(0)
  [my_lv_rmeta_2] /dev/sdf1(0)

Additional resources

  • The lvconvert(8) man page

9.16. Splitting off a RAID image as a separate logical volume

You can split off an image of a RAID logical volume to form a new logical volume. When you are removing a RAID image from an existing RAID1 logical volume or removing a RAID data subvolume and its associated metadata subvolume from the middle of the device, any higher numbered images will be shifted down to fill the slot. The index numbers on the logical volumes that make up a RAID array will thus be an unbroken sequence of integers.

Note

You cannot split off a RAID image if the RAID1 array is not yet in sync.

Procedure

  1. Display the LVM device my_vg/my_lv, which is a 2-way RAID1 array: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv             12.00 my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]        /dev/sde1(1)
  [my_lv_rimage_1]        /dev/sdf1(1)
  [my_lv_rmeta_0]         /dev/sde1(0)
  [my_lv_rmeta_1]         /dev/sdf1(0)

  2. Split the RAID image into a separate logical volume:

    • The following example splits a 2-way RAID1 logical volume, my_lv, into two linear logical volumes, my_lv and new: 

      # lvconvert --splitmirror 1 -n new my_vg/my_lv
Are you sure you want to split raid1 LV my_vg/my_lv losing all resilience? [y/n]: y

    • The following example splits a 3-way RAID1 logical volume, my_lv, into a 2-way RAID1 logical volume, my_lv, and a linear logical volume, new: 

      # lvconvert --splitmirror 1 -n new my_vg/my_lv

Verification

  • View the logical volume after you split off an image of a RAID logical volume: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV      Copy%  Devices
  my_lv          /dev/sde1(1)
  new            /dev/sdf1(1)

Additional resources

  • The lvconvert(8) man page

9.17. Splitting and merging a RAID Image

You can temporarily split off an image of a RAID1 array for read-only use while tracking any changes by using the --trackchanges argument with the --splitmirrors argument of the lvconvert command. Using this feature, you can merge the image into an array at a later time while resyncing only those portions of the array that have changed since the image was split.

When you split off a RAID image with the --trackchanges argument, you can specify which image to split but you cannot change the name of the volume being split. In addition, the resulting volumes have the following constraints:

  • The new volume you create is read-only.
  • You cannot resize the new volume.
  • You cannot rename the remaining array.
  • You cannot resize the remaining array.
  • You can activate the new volume and the remaining array independently.

You can merge an image that was split off. When you merge the image, only the portions of the array that have changed since the image was split are resynced.

Procedure

  1. Create a RAID logical volume: 

    # lvcreate --type raid1 -m 2 -L 1G -n my_lv my_vg
  Logical volume "my_lv" created

  2. Optional: View the created RAID logical volume: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv          100.00 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]        /dev/sdb1(1)
  [my_lv_rimage_1]        /dev/sdc1(1)
  [my_lv_rimage_2]        /dev/sdd1(1)
  [my_lv_rmeta_0]         /dev/sdb1(0)
  [my_lv_rmeta_1]         /dev/sdc1(0)
  [my_lv_rmeta_2]         /dev/sdd1(0)

  3. Split an image from the created RAID logical volume and track the changes to the remaining array: 

    # lvconvert --splitmirrors 1 --trackchanges my_vg/my_lv
  my_lv_rimage_2 split from my_lv for read-only purposes.
  Use 'lvconvert --merge my_vg/my_lv_rimage_2' to merge back into my_lv

  4. Optional: View the logical volume after splitting the image: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]        /dev/sdc1(1)
  [my_lv_rimage_1]          /dev/sdd1(1)
  [my_lv_rmeta_0]         /dev/sdc1(0)
  [my_lv_rmeta_1]         /dev/sdd1(0)

  5. Merge the volume back into the array: 

    # lvconvert --merge my_vg/my_lv_rimage_1
  my_vg/my_lv_rimage_1 successfully merged back into my_vg/my_lv

Verification

  • View the merged logical volume: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]        /dev/sdc1(1)
  [my_lv_rimage_1]        /dev/sdd1(1)
  [my_lv_rmeta_0]         /dev/sdc1(0)
  [my_lv_rmeta_1]         /dev/sdd1(0)

Additional resources

  • The lvconvert(8) man page

9.18. Setting a RAID fault policy

Based on the raid_fault_policy field preferences in the /etc/lvm/lvm.conf file, LVM RAID automatically handles device failures. You can set raid_fault_policy field to any one of the following parameter depending on the requirement:

warn

You can this parameter to manually repair the failed device and display warnings by using system logs.

By default, the value of the raid_fault_policy field is warn in lvm.conf. If enough devices are operational, the RAID logical volume continues to operate.

allocate
You can use this parameter to automatically replace the failed device.

9.18.1. Setting the RAID fault policy to allocate

You can set the raid_fault_policy field to the allocate parameter in the /etc/lvm/lvm.conf file. With this preference, the system attempts to replace the failed device with a spare device from the volume group. If there is no spare device, the system log includes this information.

Procedure

  1. View the RAID logical volume: 

    # lvs -a -o name,copy_percent,devices my_vg

  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]        /dev/sdb1(1)
  [my_lv_rimage_1]        /dev/sdc1(1)
  [my_lv_rimage_2]        /dev/sdd1(1)
  [my_lv_rmeta_0]         /dev/sdb1(0)
  [my_lv_rmeta_1]         /dev/sdc1(0)
  [my_lv_rmeta_2]         /dev/sdd1(0)

  2. View the RAID logical volume if the /dev/sdb device fails: 

    # lvs --all --options name,copy_percent,devices my_vg

  /dev/sdb: open failed: No such device or address
  Couldn't find device with uuid A4kRl2-vIzA-uyCb-cci7-bOod-H5tX-IzH4Ee.
  WARNING: Couldn't find all devices for LV my_vg/my_lv_rimage_1 while checking used and assumed devices.
  WARNING: Couldn't find all devices for LV my_vg/my_lv_rmeta_1 while checking used and assumed devices.
  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]        [unknown](1)
  [my_lv_rimage_1]        /dev/sdc1(1)
  [...]

    You can also view the system log for the error messages if the /dev/sdb device fails.

  3. Set the raid_fault_policy field to allocate in the lvm.conf file: 

     # vi /etc/lvm/lvm.conf
 raid_fault_policy = "allocate"
    Note

    If you set raid_fault_policy to allocate but there are no spare devices, the allocation fails, leaving the logical volume as it is. If the allocation fails, you can fix and replace the failed device by using the lvconvert --repair command. For more information, see Replacing a failed RAID device in a logical volume.

Verification

  • Verify if the failed device is now replaced with a new device from the volume group: 

    # lvs -a -o name,copy_percent,devices my_vg
  Couldn't find device with uuid 3lugiV-3eSP-AFAR-sdrP-H20O-wM2M-qdMANy.
  LV            Copy%  Devices
  lv            100.00 lv_rimage_0(0),lv_rimage_1(0),lv_rimage_2(0)
  [lv_rimage_0]        /dev/sdh1(1)
  [lv_rimage_1]        /dev/sdc1(1)
  [lv_rimage_2]        /dev/sdd1(1)
  [lv_rmeta_0]         /dev/sdh1(0)
  [lv_rmeta_1]         /dev/sdc1(0)
  [lv_rmeta_2]         /dev/sdd1(0)
    Note

    Even though the failed device is now replaced, the display still indicates that LVM could not find the failed device because the device is not yet removed from the volume group. You can remove the failed device from the volume group by executing the vgreduce --removemissing my_vg command.

Additional resources

  • lvm.conf(5) man page

9.18.2. Setting the RAID fault policy to warn

You can set the raid_fault_policy field to the warn parameter in the lvm.conf file. With this preference, the system adds a warning to the system log that indicates a failed device. Based on the warning, you can determine the further steps.

Procedure

  1. View the RAID logical volume: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]        /dev/sdb1(1)
  [my_lv_rimage_1]        /dev/sdc1(1)
  [my_lv_rimage_2]        /dev/sdd1(1)
  [my_lv_rmeta_0]         /dev/sdb1(0)
  [my_lv_rmeta_1]         /dev/sdc1(0)
  [my_lv_rmeta_2]         /dev/sdd1(0)

  2. Set the raid_fault_policy field to warn in the lvm.conf file: 

    # vi /etc/lvm/lvm.conf
 # This configuration option has an automatic default value.
 raid_fault_policy = "warn"

  3. View the system log to display error messages if the /dev/sdb device fails: 

    # grep lvm /var/log/messages

Apr 14 18:48:59 virt-506 kernel: sd 25:0:0:0: rejecting I/O to offline device
Apr 14 18:48:59 virt-506 kernel: I/O error, dev sdb, sector 8200 op 0x1:(WRITE) flags 0x20800 phys_seg 0 prio class 2
[...]
Apr 14 18:48:59 virt-506 dmeventd[91060]: WARNING: VG my_vg is missing PV 9R2TVV-bwfn-Bdyj-Gucu-1p4F-qJ2Q-82kCAF (last written to /dev/sdb).
Apr 14 18:48:59 virt-506 dmeventd[91060]: WARNING: Couldn't find device with uuid 9R2TVV-bwfn-Bdyj-Gucu-1p4F-qJ2Q-82kCAF.
Apr 14 18:48:59 virt-506 dmeventd[91060]: Use 'lvconvert --repair my_vg/ly_lv' to replace failed device.

    If the /dev/sdb device fails, the system log displays error messages. In this case, however, LVM will not automatically attempt to repair the RAID device by replacing one of the images. Instead, if the device has failed you can replace the device with the --repair argument of the lvconvert command. For more information, see Replacing a failed RAID device in a logical volume.

Additional resources

  • lvm.conf(5) man page

9.19. Replacing a RAID device in a logical volume

You can replace a RAID device in a logical volume depending on the following scenarios:

  • Replacing a working RAID device.
  • Replacing a failed RAID device in a logical volume.

9.19.1. Replacing a working RAID device

You can replace a working RAID device in a logical volume by using the --replace argument of the lvconvert command.

Warning

In the case of RAID device failure, the following commands do not work.

Prerequisites

  • The RAID device has not failed.

Procedure

  1. Create a RAID1 array: 

    # lvcreate --type raid1 -m 2 -L 1G -n my_lv my_vg
  Logical volume "my_lv" created

  2. Examine the created RAID1 array: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv            100.00 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]        /dev/sdb1(1)
  [my_lv_rimage_1]        /dev/sdb2(1)
  [my_lv_rimage_2]        /dev/sdc1(1)
  [my_lv_rmeta_0]         /dev/sdb1(0)
  [my_lv_rmeta_1]         /dev/sdb2(0)
  [my_lv_rmeta_2]         /dev/sdc1(0)

  3. Replace the RAID device with any of the following methods depending on your requirements:

    1. Replace a RAID1 device by specifying the physical volume that you want to replace: 

      # lvconvert --replace /dev/sdb2 my_vg/my_lv

    2. Replace a RAID1 device by specifying the physical volume to use for the replacement: 

      # lvconvert --replace /dev/sdb1 my_vg/my_lv /dev/sdd1

    3. Replace multiple RAID devices at a time by specifying multiple replace arguments: 

      # lvconvert --replace /dev/sdb1 --replace /dev/sdc1 my_vg/my_lv

Verification

  1. Examine the RAID1 array after specifying the physical volume that you wanted to replace: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv             37.50 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]        /dev/sdb1(1)
  [my_lv_rimage_1]        /dev/sdc2(1)
  [my_lv_rimage_2]        /dev/sdc1(1)
  [my_lv_rmeta_0]         /dev/sdb1(0)
  [my_lv_rmeta_1]         /dev/sdc2(0)
  [my_lv_rmeta_2]         /dev/sdc1(0)

  2. Examine the RAID1 array after specifying the physical volume to use for the replacement: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv             28.00 my_lv_rimage_0(0),my_lv_rimage_1(0)
  [my_lv_rimage_0]        /dev/sda1(1)
  [my_lv_rimage_1]        /dev/sdd1(1)
  [my_lv_rmeta_0]         /dev/sda1(0)
  [my_lv_rmeta_1]         /dev/sdd1(0)

  3. Examine the RAID1 array after replacing multiple RAID devices at a time: 

    # lvs -a -o name,copy_percent,devices my_vg
  LV               Copy%  Devices
  my_lv             60.00 my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]        /dev/sda1(1)
  [my_lv_rimage_1]        /dev/sdd1(1)
  [my_lv_rimage_2]        /dev/sde1(1)
  [my_lv_rmeta_0]         /dev/sda1(0)
  [my_lv_rmeta_1]         /dev/sdd1(0)
  [my_lv_rmeta_2]         /dev/sde1(0)

Additional resources

  • lvconvert(8) man page

9.19.2. Replacing a failed RAID device in a logical volume

RAID is not similar to traditional LVM mirroring. In case of LVM mirroring, remove the failed devices. Otherwise, the mirrored logical volume would hang while RAID arrays continue running with failed devices. For RAID levels other than RAID1, removing a device would mean converting to a lower RAID level, for example, from RAID6 to RAID5, or from RAID4 or RAID5 to RAID0.

Instead of removing a failed device and allocating a replacement, with LVM, you can replace a failed device that serves as a physical volume in a RAID logical volume by using the --repair argument of the lvconvert command.

Prerequisites

  • The volume group includes a physical volume that provides enough free capacity to replace the failed device.

    If no physical volume with enough free extents is available on the volume group, add a new, sufficiently large physical volume by using the vgextend utility.

Procedure

  1. View the RAID logical volume: 

    # lvs --all --options name,copy_percent,devices my_vg
  LV               Cpy%Sync Devices
  my_lv            100.00   my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]          /dev/sde1(1)
  [my_lv_rimage_1]          /dev/sdc1(1)
  [my_lv_rimage_2]          /dev/sdd1(1)
  [my_lv_rmeta_0]           /dev/sde1(0)
  [my_lv_rmeta_1]           /dev/sdc1(0)
  [my_lv_rmeta_2]           /dev/sdd1(0)

  2. View the RAID logical volume after the /dev/sdc device fails: 

    # lvs --all --options name,copy_percent,devices my_vg
  /dev/sdc: open failed: No such device or address
  Couldn't find device with uuid A4kRl2-vIzA-uyCb-cci7-bOod-H5tX-IzH4Ee.
  WARNING: Couldn't find all devices for LV my_vg/my_lv_rimage_1 while checking used and assumed devices.
  WARNING: Couldn't find all devices for LV my_vg/my_lv_rmeta_1 while checking used and assumed devices.
  LV               Cpy%Sync Devices
  my_lv            100.00   my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]          /dev/sde1(1)
  [my_lv_rimage_1]          [unknown](1)
  [my_lv_rimage_2]          /dev/sdd1(1)
  [my_lv_rmeta_0]           /dev/sde1(0)
  [my_lv_rmeta_1]           [unknown](0)
  [my_lv_rmeta_2]           /dev/sdd1(0)

  3. Replace the failed device: 

    # lvconvert --repair my_vg/my_lv
  /dev/sdc: open failed: No such device or address
  Couldn't find device with uuid A4kRl2-vIzA-uyCb-cci7-bOod-H5tX-IzH4Ee.
  WARNING: Couldn't find all devices for LV my_vg/my_lv_rimage_1 while checking used and assumed devices.
  WARNING: Couldn't find all devices for LV my_vg/my_lv_rmeta_1 while checking used and assumed devices.
Attempt to replace failed RAID images (requires full device resync)? [y/n]: y
Faulty devices in my_vg/my_lv successfully replaced.

  4. Optional: Manually specify the physical volume that replaces the failed device: 

    # lvconvert --repair my_vg/my_lv replacement_pv

  5. Examine the logical volume with the replacement: 

    # lvs --all --options name,copy_percent,devices my_vg

  /dev/sdc: open failed: No such device or address
  /dev/sdc1: open failed: No such device or address
  Couldn't find device with uuid A4kRl2-vIzA-uyCb-cci7-bOod-H5tX-IzH4Ee.
  LV               Cpy%Sync Devices
  my_lv            43.79    my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]          /dev/sde1(1)
  [my_lv_rimage_1]          /dev/sdb1(1)
  [my_lv_rimage_2]          /dev/sdd1(1)
  [my_lv_rmeta_0]           /dev/sde1(0)
  [my_lv_rmeta_1]           /dev/sdb1(0)
  [my_lv_rmeta_2]           /dev/sdd1(0)

    Until you remove the failed device from the volume group, LVM utilities still indicate that LVM cannot find the failed device.

  6. Remove the failed device from the volume group: 

    # vgreduce --removemissing my_vg

Verification

  1. View the available physical volumes after removing the failed device: 

    # pvscan
PV /dev/sde1 VG rhel_virt-506 lvm2 [<7.00 GiB / 0 free]
PV /dev/sdb1 VG my_vg lvm2 [<60.00 GiB / 59.50 GiB free]
PV /dev/sdd1 VG my_vg lvm2 [<60.00 GiB / 59.50 GiB free]
PV /dev/sdd1 VG my_vg lvm2 [<60.00 GiB / 59.50 GiB free]

  2. Examine the logical volume after the replacing the failed device: 

    # lvs --all --options name,copy_percent,devices my_vg
my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
  [my_lv_rimage_0]          /dev/sde1(1)
  [my_lv_rimage_1]          /dev/sdb1(1)
  [my_lv_rimage_2]          /dev/sdd1(1)
  [my_lv_rmeta_0]           /dev/sde1(0)
  [my_lv_rmeta_1]           /dev/sdb1(0)
  [my_lv_rmeta_2]           /dev/sdd1(0)

Additional resources

  • lvconvert(8) and vgreduce(8) man pages

9.20. Checking data coherency in a RAID logical volume

LVM provides scrubbing support for RAID logical volumes. RAID scrubbing is the process of reading all the data and parity blocks in an array and checking to see whether they are coherent. The lvchange --syncaction repair command initiates a background synchronization action on the array. The following attributes provide details about data coherency:

  • The raid_sync_action field displays the current synchronization action that the RAID logical volume is performing. It can be one of the following values:

    idle
    Completed all sync actions (doing nothing).
    resync
    Initializing or resynchronizing an array after an unclean machine shutdown.
    recover
    Replacing a device in the array.
    check
    Looking for array inconsistencies.
    repair
    Looking for and repairing inconsistencies.
  • The raid_mismatch_count field displays the number of discrepancies found during a check action.
  • The Cpy%Sync field displays the progress of the sync actions.

  • The lv_attr field provides additional indicators. Bit 9 of this field displays the health of the logical volume, and it supports the following indicators:

    m or mismatches
    Indicates that there are discrepancies in a RAID logical volume. You can see this character after the scrubbing operation detects the portions of the RAID, which are not coherent.
    r or refresh
    Indicates a failed device in a RAID array, even though LVM can read the device label and considers the device to be operational. Refresh the logical volume to notify the kernel that the device is now available, or replace the device if you suspect that it failed.

Procedure

  1. Optional: Limit the I/O bandwidth that the scrubbing process uses. When you perform a RAID scrubbing operation, the background I/O required by the sync actions can crowd out other I/O to LVM devices, such as updates to volume group metadata. This might cause the other LVM operations to slow down.

    You can control the rate of the scrubbing operation by implementing recovery throttling. You can set the recovery rate using --maxrecoveryrate Rate[bBsSkKmMgG] or --minrecoveryrate Rate[bBsSkKmMgG] with the lvchange --syncaction commands. For more information, see Minimum and maximum I/O rate options.

    Specify the Rate value as an amount per second for each device in the array. If you provide no suffix, the options assume kiB per second per device.

  2. Display the number of discrepancies in the array, without repairing them: 

    # lvchange --syncaction check my_vg/my_lv

    This command initiates a background synchronization action on the array.

  3. Optional: View the var/log/syslog file for the kernel messages.

  4. Correct the discrepancies in the array: 

    # lvchange --syncaction repair my_vg/my_lv

    This command repairs or replaces failed devices in a RAID logical volume. You can view the var/log/syslog file for the kernel messages after executing this command.

Verification

  1. Display information about the scrubbing operation: 

    # lvs -o +raid_sync_action,raid_mismatch_count my_vg/my_lv
LV    VG    Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert SyncAction Mismatches
my_lv my_vg rwi-a-r--- 500.00m                                    100.00           idle        0

Additional resources

9.21. Converting a RAID logical volume to another RAID level

LVM supports RAID takeover, which means converting a RAID logical volume from one RAID level to another, for example, from RAID 5 to RAID 6. You can change the RAID level to increase or decrease resilience to device failures.

Procedure

  1. Create a RAID logical volume: 

    # lvcreate --type raid5 -i 3 -L 500M -n my_lv my_vg
Using default stripesize 64.00 KiB.
Rounding size 500.00 MiB (125 extents) up to stripe boundary size 504.00 MiB (126 extents).
Logical volume "my_lv" created.

  2. View the RAID logical volume: 

    # lvs -a -o +devices,segtype
  LV               VG            Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices                                                                 Type
  my_lv            my_vg         rwi-a-r--- 504.00m                                    100.00           my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0),my_lv_rimage_3(0) raid5
  [my_lv_rimage_0] my_vg         iwi-aor--- 168.00m                                                     /dev/sda(1)                                                             linear

  3. Convert the RAID logical volume to another RAID level: 

    # lvconvert --type raid6 my_vg/my_lv
Using default stripesize 64.00 KiB.
Replaced LV type raid6 (same as raid6_zr) with possible type raid6_ls_6.
Repeat this command to convert to raid6 after an interim conversion has finished.
Are you sure you want to convert raid5 LV my_vg/my_lv to raid6_ls_6 type? [y/n]: y
Logical volume my_vg/my_lv successfully converted.

  4. Optional: If this command prompts to repeat the conversion, run: 

    # lvconvert --type raid6 my_vg/my_lv

Verification

  1. View the RAID logical volume with the converted RAID level: 

    # lvs -a -o +devices,segtype
  LV               VG            Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices                                                                                   Type
  my_lv            my_vg         rwi-a-r--- 504.00m                                    100.00           my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0),my_lv_rimage_3(0),my_lv_rimage_4(0) raid6
  [my_lv_rimage_0] my_vg         iwi-aor--- 172.00m                                                     /dev/sda(1)                                                                               linear

Additional resources

  • lvconvert(8) and lvmraid(8) man pages

9.22. I/O Operations on a RAID1 logical volume

You can control the I/O operations for a device in a RAID1 logical volume by using the --writemostly and --writebehind parameters of the lvchange command. The following is the format for using these parameters:

--[raid]writemostly PhysicalVolume[:{t|y|n}]

Marks a device in a RAID1 logical volume as write-mostly and avoids all read actions to these drives unless necessary. Setting this parameter keeps the number of I/O operations to the drive to a minimum. Use the lvchange --writemostly /dev/sdb my_vg/ly_lv command to set this parameter.

You can set the writemostly attribute in the following ways:

:y
By default, the value of the writemostly attribute is yes for the specified physical volume in the logical volume.
:n
To remove the writemostly flag, append :n to the physical volume.
:t
To toggle the value of the writemostly attribute, specify the --writemostly argument. You can use this argument more than one time in a single command, to toggle the writemostly attributes for all the physical volumes in a logical volume at once.
--[raid]writebehind IOCount

Specifies the maximum number of pending writes marked as writemostly. These are the number of write operations applicable to devices in a RAID1 logical volume. After the value of this parameter exceeds, all write actions to the constituent devices complete synchronously before the RAID array notifies for completion of all write actions.

You can set this parameter by using the lvchange --writebehind 100 my_vg/ly_lv command. Setting the writemostly attribute’s value to zero clears the preference. With this setting, the system chooses the value arbitrarily.

9.23. Reshaping a RAID volume

RAID reshaping means changing attributes of a RAID logical volume without changing the RAID level. Some attributes that you can change include RAID layout, stripe size, and number of stripes.

Procedure

  1. Create a RAID logical volume: 

    # lvcreate --type raid5 -i 2 -L 500M -n my_lv my_vg

Using default stripesize 64.00 KiB.
Rounding size 500.00 MiB (125 extents) up to stripe boundary size 504.00 MiB (126 extents).
Logical volume "my_lv" created.

  2. View the RAID logical volume: 

    # lvs -a -o +devices

LV               VG    Attr       LSize   Pool   Origin Data% Meta% Move Log Cpy%Sync Convert Devices
my_lv            my_vg rwi-a-r--- 504.00m                                    100.00            my_lv_rimage_0(0),my_lv_rimage_1(0),my_lv_rimage_2(0)
[my_lv_rimage_0] my_vg iwi-aor--- 252.00m                                                      /dev/sda(1)
[my_lv_rimage_1] my_vg iwi-aor--- 252.00m                                                      /dev/sdb(1)
[my_lv_rimage_2] my_vg iwi-aor--- 252.00m                                                      /dev/sdc(1)
[my_lv_rmeta_0]  my_vg ewi-aor---   4.00m                                                      /dev/sda(0)
[my_lv_rmeta_1]  my_vg ewi-aor---   4.00m                                                      /dev/sdb(0)
[my_lv_rmeta_2]  my_vg ewi-aor---   4.00m                                                      /dev/sdc(0)

  3. Optional: View the stripes images and stripesize of the RAID logical volume: 

    # lvs -o stripes my_vg/my_lv
  #Str
     3
    # lvs -o stripesize my_vg/my_lv
  Stripe
  64.00k

  4. Modify the attributes of the RAID logical volume by using the following ways depending on your requirement:

    1. Modify the stripes images of the RAID logical volume: 

      # lvconvert --stripes 3 my_vg/my_lv
Using default stripesize 64.00 KiB.
WARNING: Adding stripes to active logical volume my_vg/my_lv will grow it from 126 to 189 extents!
Run "lvresize -l126 my_vg/my_lv" to shrink it or use the additional capacity.
Are you sure you want to add 1 images to raid5 LV my_vg/my_lv? [y/n]: y
Logical volume my_vg/my_lv successfully converted.

    2. Modify the stripesize of the RAID logical volume: 

      # lvconvert --stripesize 128k my_vg/my_lv
  Converting stripesize 64.00 KiB of raid5 LV my_vg/my_lv to 128.00 KiB.
Are you sure you want to convert raid5 LV my_vg/my_lv? [y/n]: y
  Logical volume my_vg/my_lv successfully converted.

    3. Modify the maxrecoveryrate and minrecoveryrate attributes: 

      # lvchange --maxrecoveryrate 4M my_vg/my_lv
  Logical volume my_vg/my_lv changed.
      # lvchange --minrecoveryrate 1M my_vg/my_lv
  Logical volume my_vg/my_lv changed.

    4. Modify the syncaction attribute: 

      # lvchange --syncaction check my_vg/my_lv

    5. Modify the writemostly and writebehind attributes: 

      # lvchange --writemostly /dev/sdb my_vg/my_lv
  Logical volume my_vg/my_lv changed.
      # lvchange --writebehind 100 my_vg/my_lv
  Logical volume my_vg/my_lv changed.

Verification

  1. View the stripes images and stripesize of the RAID logical volume: 

    # lvs -o stripes my_vg/my_lv
  #Str
     4
    # lvs -o stripesize my_vg/my_lv
  Stripe
  128.00k

  2. View the RAID logical volume after modifying the maxrecoveryrate attribute: 

    # lvs -a -o +raid_max_recovery_rate
  LV               VG       Attr        LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert MaxSync
  my_lv            my_vg    rwi-a-r---  10.00g                                     100.00           4096
  [my_lv_rimage_0] my_vg    iwi-aor---  10.00g
 [...]

  3. View the RAID logical volume after modifying the minrecoveryrate attribute: 

    # lvs -a -o +raid_min_recovery_rate
  LV               VG     Attr        LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert MinSync
  my_lv            my_vg  rwi-a-r---  10.00g                                     100.00           1024
  [my_lv_rimage_0] my_vg  iwi-aor---  10.00g
  [...]

  4. View the RAID logical volume after modifying the syncaction attribute: 

    # lvs -a
  LV               VG      Attr        LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  my_lv            my_vg   rwi-a-r---  10.00g                                     2.66
  [my_lv_rimage_0] my_vg   iwi-aor---  10.00g
  [...]

Additional resources

  • lvconvert(8) and lvmraid(8) man pages

9.24. Changing the region size on a RAID logical volume

When you create a RAID logical volume, the raid_region_size parameter from the /etc/lvm/lvm.conf file represents the region size for the RAID logical volume. After you created a RAID logical volume, you can change the region size of the volume. This parameter defines the granularity to keep track of the dirty or clean state. Dirty bits in the bitmap define the work set to synchronize after a dirty shutdown of a RAID volume, for example, a system failure.

If you set raid_region_size to a higher value, it reduces the size of bitmap as well as the congestion. But it impacts the write operation during resynchronizing the region because writes to RAID are postponed until synchronizing the region finishes.

Procedure

  1. Create a RAID logical volume: 

    # lvcreate --type raid1 -m 1 -L 10G test
  Logical volume "lvol0" created.

  2. View the RAID logical volume: 

    # lvs -a -o +devices,region_size

LV                VG      Attr     LSize Pool Origin Data% Meta% Move Log   Cpy%Sync Convert Devices                              Region
lvol0             test rwi-a-r--- 10.00g                                    100.00           lvol0_rimage_0(0),lvol0_rimage_1(0)  2.00m
[lvol0_rimage_0]  test iwi-aor--- 10.00g                                                     /dev/sde1(1)                            0
[lvol0_rimage_1]  test iwi-aor--- 10.00g                                                     /dev/sdf1(1)                            0
[lvol0_rmeta_0]   test ewi-aor---  4.00m                                                     /dev/sde1(0)                            0
[lvol0_rmeta_1]   test ewi-aor---  4.00m

    The Region column indicates the raid_region_size parameter’s value.

  3. Optional: View the raid_region_size parameter’s value: 

    # cat /etc/lvm/lvm.conf | grep raid_region_size

# Configuration option activation/raid_region_size.
	# raid_region_size = 2048

  4. Change the region size of a RAID logical volume: 

    # lvconvert -R 4096K my_vg/my_lv

Do you really want to change the region_size 512.00 KiB of LV my_vg/my_lv to 4.00 MiB? [y/n]: y
  Changed region size on RAID LV my_vg/my_lv to 4.00 MiB.

  5. Resynchronize the RAID logical volume: 

    # lvchange --resync my_vg/my_lv

Do you really want to deactivate logical volume my_vg/my_lv to resync it? [y/n]: y

Verification

  1. View the RAID logical volume: 

    # lvs -a -o +devices,region_size

LV               VG   Attr        LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices                              Region
lvol0            test rwi-a-r--- 10.00g                                    6.25           lvol0_rimage_0(0),lvol0_rimage_1(0)  4.00m
[lvol0_rimage_0] test iwi-aor--- 10.00g                                                   /dev/sde1(1)                            0
[lvol0_rimage_1] test iwi-aor--- 10.00g                                                   /dev/sdf1(1)                            0
[lvol0_rmeta_0]  test ewi-aor---  4.00m                                                   /dev/sde1(0)                            0

    The Region column indicates the changed value of the raid_region_size parameter.

  2. View the raid_region_size parameter’s value in the lvm.conf file: 

    # cat /etc/lvm/lvm.conf | grep raid_region_size

# Configuration option activation/raid_region_size.
	# raid_region_size = 4096

Additional resources

  • lvconvert(8) man page
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.