Chapter 3. Testing VDO space savings

Procedure

1. Create a VDO volume:

   # vdo create --name=vdo-test \
                --device=/dev/sdb \
                --vdoLogicalSize=1T \
                --writePolicy=policy \
                --verbose

   • Replace /dev/sdb with the path to a block device.
   • To test the VDO async mode on top of asynchronous storage, create an asynchronous volume using the --writePolicy=async option.
   • To test the VDO sync mode on top of synchronous storage, create a synchronous volume using the --writePolicy=sync option.

2. Format the new volume with an XFS or ext4 file system.

   • For XFS:

     # mkfs.xfs -K /dev/mapper/vdo-test

   • For ext4:

     # mkfs.ext4 -E nodiscard /dev/mapper/vdo-test

3. Mount the formatted volume:

   # mkdir /mnt/vdo-test
   
   # mount /dev/mapper/vdo-test /mnt/vdo-test && \
     chmod a+rwx /mnt/vdo-test

Procedure

1. Write 32 GiB of random data to the VDO volume:

   $ dd if=/dev/urandom of=/mnt/vdo-test/testfile bs=4096 count=8388608

2. Read the data from the VDO volume and write it to another volume:

   $ dd if=/mnt/vdo-test/testfile of=another-location/testfile bs=4096

   • Replace another-location with any directory where you have write access that is not on the VDO test volume. For example, you can use your home directory.

3. Compare the two files:

   $ diff --report-identical-files /mnt/vdo-test/testfile another-location/testfile

   The command should report that the files are the same.

4. Copy the file back to a new location on the VDO volume:

   $ dd if=another-location/testfile of=/mnt/vdo-test/testfile2 bs=4096

5. Compare the third file to the second file:

   $ diff --report-identical-files /mnt/vdo-test/testfile2 another-location/testfile

   The command should report that the files are the same.

Procedure

1. Unmount the file system created on the VDO volume:

   # umount /mnt/vdo-test

2. Remove the VDO test volume from the system:

   # vdo remove --name=vdo-test

Procedure

1. Prepare a table where you can record the test results:

   Statistic	Bare file system	After seed	After 10 copies
   File system used size
   VDO data used
   VDO logical used

2. Create 10 directories on the VDO volume to hold 10 copies of the test data set:

   $ mkdir /mnt/vdo-test/vdo{01..10}

3. Examine the disk usage reported by the file system:

   $ df --human-readable /mnt/vdo-test

   Example 3.1. Disk usage

   Filesystem            Size  Used Avail Use% Mounted on
   /dev/mapper/vdo-test  1.5T  198M  1.4T   1% /mnt/vdo-test

4. Record the following values:

   # vdostats --verbose | grep "blocks used"

   Example 3.2. Used blocks

   data blocks used                : 1090
   overhead blocks used            : 538846
   logical blocks used             : 6059434

   • The data blocks used value is the number of blocks used by user data after optimization on the physical device running under VDO.
   • The logical blocks used value is the number of blocks used before optimization. It will be used as the starting point for measurements.

5. Create a data source file on the VDO volume:

   $ dd if=/dev/urandom of=/mnt/vdo-test/sourcefile bs=4096 count=1048576
   
   4294967296 bytes (4.3 GB) copied, 540.538 s, 7.9 MB/s

6. Re-examine the amount of used physical disk space:

   $ df --human-readable /mnt/vdo-test

   Example 3.3. Disk usage with the data source file

   Filesystem            Size  Used Avail Use% Mounted on
   /dev/mapper/vdo-test  1.5T  4.2G  1.4T   1% /mnt/vdo-test

   # vdostats --verbose | grep "blocks used"

   Example 3.4. Used blocks with the data source file

   data blocks used                : 1050093  # Increased by 4GiB
   overhead blocks used            : 538846   # Did not significantly change
   logical blocks used             : 7108036  # Increased by 4GiB

   This command should show an increase in the number of blocks used, corresponding to the size of the written file.

7. Copy the file to each of the 10 subdirectories:

   $ for i in {01..10}; do
     cp /mnt/vdo-test/sourcefile /mnt/vdo-test/vdo$i
     done

8. Re-examine the amount of used physical disk space:

   $ df -h /mnt/vdo-test

   Example 3.5. Disk usage after copying the file

   Filesystem            Size  Used Avail Use% Mounted on
   /dev/mapper/vdo-test  1.5T   45G  1.3T   4% /mnt/vdo-test

   # vdostats --verbose | grep "blocks used"

   Example 3.6. Used blocks after copying the file

   data blocks used                : 1050836   # Increased by 3 MiB
   overhead blocks used            : 538846
   logical blocks used             : 17594127  # Increased by 41 GiB

   The data blocks used value should be similar to the result of the earlier listing, with only a slight increase due to file system journaling and metadata.

9. Subtract this new value of the space used by the file system from the value found before writing the test data. This is the amount of space consumed by this test from the perspective of the file system.

10. Observe the space savings in your recorded statistics:

    Example 3.7. Recorded values

    Statistic	Bare file system	After seed	After 10 copies
    File system used size	198 MiB	4.2 GiB	45 GiB
    VDO data used	4 MiB	4.1 GiB	4.1 GiB
    VDO logical used	23.6 GiB (file system overhead for 1.6 TiB formatted drive)	27.8 GiB	68.7 GiB

    Note

    In the table, values have been converted to MiB or GiB. Blocks in the vdostats output are 4,096 B in size.

Procedure

1. Disable deduplication and enable compression on the VDO test volume:

   # vdo disableDeduplication --name=vdo-test
   # vdo enableCompression --name=vdo-test

2. Synchronize the VDO volume to complete any unfinished compression:

   # sync && dmsetup message vdo-test 0 sync-dedupe

3. Inspect VDO statistics before the transfer:

   # vdostats --verbose | grep "blocks used"

   Make note of the data blocks used and logical blocks used values.

4. VDO optimizes file system overhead as well as actual user data. Calculate the number of 4 KiB blocks saved by compression for the empty file system as logical blocks used minus data blocks used.

5. Copy the content of the /lib directory to the VDO volume:

   # cp --verbose --recursive /lib /mnt/vdo-test
   
   ...
   sent 152508960 bytes  received 60448 bytes  61027763.20 bytes/sec
   total size is 152293104  speedup is 1.00

   Record the total size of the copied data.

6. Synchronize Linux caches and the VDO volume:

   # sync && dmsetup message vdo-test 0 sync-dedupe

7. Inspect VDO statistics again:

   # vdostats --verbose | grep "blocks used"

   Observe the logical blocks used and data blocks used values.

8. Calculate the amount of bytes saved by compression using the following formula:

   saved_bytes = (logical_blocks_used - data_blocks_used) * 4096

Procedure

1. Create and mount a VDO volume as described in Section 3.4, “Creating a VDO test volume”.
2. Perform the tests described in Measuring VDO deduplication and Measuring VDO compression on the same volume without removing it. Observe changes to space savings in the vdostats output.
3. Experiment with your own datasets.

Procedure

1. Prepare a table where you can record the test results:

   Step	File space used (MB)	Data blocks used	Logical blocks used
   Initial
   Add 1 GiB file
   Run fstrim
   Delete 1 GiB file
   Run fstrim

2. Trim the file system to remove unneeded blocks:

   # fstrim /mnt/vdo-test

   The command might take a long time.

3. Record the initial space usage in the file system:

   $ df -m /mnt/vdo-test

4. See how many physical and logical data blocks the VDO volume uses:

   # vdostats --verbose | grep "blocks used"

5. Create a 1 GiB file with non-duplicate data on the VDO volume:

   $ dd if=/dev/urandom of=/mnt/vdo-test/file bs=1M count=1K

6. Record the space usage again:

   $ df -m /mnt/vdo-test
   
   # vdostats --verbose | grep "blocks used"

   The file system should use an additional 1 GiB. The data blocks used and logical blocks used values should increase similarly.

7. Trim the file system again:

   # fstrim /mnt/vdo-test

8. Inspect the space usage again to confirm that the trim had no impact on the physical volume usage:

   $ df -m /mnt/vdo-test
   
   # vdostats --verbose | grep "blocks used"

9. Delete the 1 GiB file:

   $ rm /mnt/vdo-test/file

10. Check and record the space usage again:

    $ df -m /mnt/vdo-test
    
    # vdostats --verbose | grep "blocks used"

    The file system is aware that a file has been deleted, but there is no change to the number of physical or logical blocks because the file deletion has not been communicated to the underlying storage.

11. Trim the file system again:

    # fstrim /mnt/vdo-test

12. Check and record the space usage again:

    $ df -m /mnt/vdo-test
    
    # vdostats --verbose | grep "blocks used"

    The fstrim utility looks for free blocks in the file system and sends a TRIM command to the VDO volume for unused addresses, which releases the associated logical blocks. VDO processes the TRIM command to release the underlying physical blocks.