Chapter 4. Reusing bricks and reconstructing existing brick configuration

Procedure

1. Set new keys and key files.

   1. Store the passphrase for the LUKS encrypted disk in a temporary file in the /root directory.

      # echo passphrase /root/key

      If each disk has a separate passphrase, save them separately.

      # echo passphraseA /root/sda_key
      # echo passphraseB /root/sdb_key
      # echo passphraseC /root/sdc_key
      # echo passphraseD /root/sdd_key

   2. Generate new key files.

      1. Generate a random key file for each disk.

         # for disk in sda sdb sdc sdd; do dd if=/dev/urandom of=/etc/${disk}_keyfile bs=1024 count=8192

      2. Set appropriate permissions on the new keyfiles.

         # chown 400 /etc/*_keyfile

   3. Set the new key for each disk.

      # cryptsetup luksAddKey /etc/sda_keyfile --key-file /root/sda_key
      # cryptsetup luksAddKey /etc/sdb_keyfile --key-file /root/sdb_key
      # cryptsetup luksAddKey /etc/sdc_keyfile --key-file /root/sdc_key
      # cryptsetup luksAddKey /etc/sdd_keyfile --key-file /root/sdd_key

2. Verify each device can be opened with its key file.

   1. Determine the LUKS UUID for each device.

      # cryptsetup luksUUID /dev/sdX

   2. Open each device using its key file and UUID.

      # cryptsetup luksOpen UUID=sdX-UUID luks_sdX -d /etc/sdX_keyfile

      For example:

      # cryptsetup luksOpen UUID=a28a19c7-6028-44df-b0b8-e5245944710c luks_sda -d /etc/sda_keyfile

3. Configure automatic decryption at boot time.

   Add a line for each device to the /etc/crypttab file using the following format.

   # echo luks_sdX UUID=sdX-UUID /etc/sdX_keyfile >> /etc/crypttab

   For example:

   # echo luks_sda UUID=a28a19c7-6028-44df-b0b8-e5245944710c /etc/sda_keyfile >> /etc/crypttab

4. Set up Network-Bound Disk Encryption on the root disk.

   1. Change into the hc-ansible-deployment directory:

      # cd /etc/ansible/roles/gluster.ansible/playbooks/hc-ansible-deployment

   2. Create the inventory file.

      1. Make a copy of the luks_tang_inventory.yml file for future reference.

         cp luks_tang_inventory.yml luks_tang_inventory.yml.backup

      2. Define your configuration in the luks_tang_inventory.yml file.

         Use the example luks_tang_inventory.yml file to define the details of disk encryption on each host. A complete outline of this file is available in Understanding the luks_tang_inventory.yml file.

   3. Encrypt the luks_tang_inventory.yml file and specify a password using ansible-vault.

      The required variables in luks_tang_inventory.yml include password values, so it is important to encrypt the file to protect the password values.

      # ansible-vault encrypt luks_tang_inventory.yml

      Enter and confirm a new vault password when prompted.

   4. Execute the luks_tang_setup.yml playbook with the bindtang tag.

      # ansible-playbook -i luks_tang_inventory.yml tasks/luks_tang_setup.yml --tags=bindtang --ask-vault-pass

      Enter the vault password for this file when prompted to start disk encryption configuration.

Procedure

1. Copy the /etc/vdoconf.yml file from a healthy node to the replacement node.

   # scp /etc/vdoconf.yml root@new-node.example.com:/etc/

2. Edit the indicated values in the /etc/vdoconf.yml file to provide the correct values for your replacement node.

   Important

   Be careful when editing this file. Editing this file by hand is supported only when reconstructing deduplication and compression configuration without a backup file.

   vdo_sd*

   Change this parameter to match the name of your VDO device.

   device

   Specify the VDO device using its by-id path. For normal volumes, this is something like /dev/disk/by-id/scsi-xxx. For encrypted volumes, this is something like /dev/disk/by-id/dm-uuid-CRYPT-LUKS2-xxxxx.

   For example:

   # cat /etc/vdoconf.yml
   
   config: !Configuration
     vdos:
       vdo_sdc: !VDOService
         ...
         device: /dev/disk/by-id/scsi-360030480197f830125618adb17bac04c
         ...
         logicalSize: 180T
         ...
         physicalSize: 18625G
         ...

3. Restart the VDO service.

   # systemctl restart vdo.service

Procedure

1. Scan existing physical volumes, volume groups, and logical volumes.

   # pvscan
   # vgscan
   # lvscan

2. Determine the UUID of each gluster brick.

   # blkid lv_name

3. Add a line to the /etc/fstab file for each gluster brick, using the UUID.

   # echo "UUID=64dfd1b1-4333-4ef6-8835-1053c6904d93 /gluster_bricks/engine xfs inode64,noatime,nodiratime,_netdev,x-systemd.device-timeout=0 0 0" >> /etc/fstab

   Volumes that use deduplication and compression need additional mount options, as shown:

   # echo "UUID=64dfd1b1-4333-4ef6-8835-1053c6904d93 /gluster_bricks/vmstore xfs inode64,noatime,nodiratime,_netdev,x-systemd.device-timeout=0,x-systemd.requires=vdo.service 0 0" >> /etc/fstab

4. Create mount directories based on information from volumes.

   # mkdir -p /gluster_bricks/{engine,vmstore,data}

5. Mount all bricks.

   # mount -a

6. Set the required SELinux labels on all brick mount points.

   # semanage fcontext -a -t glusterd_brick_t /gluster_bricks/engine
   # semanage fcontext -a -t glusterd_brick_t /gluster_bricks/vmstore
   # semanage fcontext -a -t glusterd_brick_t /gluster_bricks/data
   # restorecon -Rv /gluster_bricks/engine
   # restorecon -Rv /gluster_bricks/vmstore
   # restorecon -Rv /gluster_bricks/data