Chapter 2. Configuring the Block Storage service (cinder)

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

3. To enable active-active Block Storage service volumes, add this environment file to the stack with your other environment files and deploy the overcloud:

   /usr/share/openstack-tripleo-heat-templates/environments/cinder-volume-active-active.yaml

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Run the following command to verify whether all of the services for a cluster are running:

   $ cinder cluster-list --detailed

   Alternatively, run the cluster show command.

3. If any services are not running, run the following command to identify those specific services:

   $ cinder service-list

4. Run the following command to trigger the server cleanup:

   $ cinder --os-volume-api-version 3.24 work-cleanup [--cluster <cluster-name>] [--host <hostname>] [--binary <binary>] [--is-up <True|true|False|false>] [--disabled <True|true|False|false>] [--resource-id <resource-id>] [--resource-type <Volume|Snapshot>]

   Note

   Filters, such as --cluster, --host, and --binary, define what the command cleans. You can filter on cluster name, host name, type of service, and resource type, including a specific resource. If you do not apply filtering, the command attempts to clean everything that can be cleaned.

   The following example filters by cluster name:

   $ cinder --os-volume-api-version 3.24 work-cleanup --cluster tripleo@tripleo_ceph

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Determine the host of cinder-volume:

   $ cinder service-list

   This command will return a list containing the host of each Block Storage service (cinder-backup, cinder-scheduler, and cinder-volume). For example:

   +------------------+---------------------------+------+---------
   |      Binary      |            Host           | Zone |  Status ...
   +------------------+---------------------------+------+---------
   |  cinder-backup   |   localhost.localdomain   | nova | enabled ...
   | cinder-scheduler |   localhost.localdomain   | nova | enabled ...
   |  cinder-volume   | *localhost.localdomain@lvm* | nova | enabled ...
   +------------------+---------------------------+------+---------

3. Display the driver capabilities to determine the supported Extra Specs of a Block Storage service:

   $ cinder get-capabilities <volsvchost>

   • Replace <volsvchost> with the host of cinder-volume. For example:

     $ cinder get-capabilities localhost.localdomain@lvm
         +---------------------+-----------------------------------------+
         |     Volume stats    |                        Value            |
         +---------------------+-----------------------------------------+
         |     description     |                         None            |
         |     display_name    |                         None            |
         |    driver_version   |                        3.0.0            |
         |      namespace      | OS::Storage::Capabilities::localhost.loc...
         |      pool_name      |                         None            |
         |   storage_protocol  |                        iSCSI            |
         |     vendor_name     |                     Open Source         |
         |      visibility     |                         None            |
         | volume_backend_name |                         lvm             |
         +---------------------+-----------------------------------------+
         +--------------------+------------------------------------------+
         | Backend properties |                        Value             |
         +--------------------+------------------------------------------+
         |    compression     |      {u'type': u'boolean', u'description'...
         |        qos         |              {u'type': u'boolean', u'des ...
         |    replication     |      {u'type': u'boolean', u'description'...
         | thin_provisioning  | {u'type': u'boolean', u'description': u'S...
         +--------------------+------------------------------------------+

     The Backend properties column shows a list of Extra Spec Keys that you can set, while the Value column provides information on valid corresponding values.

Procedure

1. Log into the dashboard as an admin user.
2. Select Admin > Volumes > Volume Types.
3. Click Create Volume Type.
4. Enter the volume type name in the Name field.
5. Click Create Volume Type. The new type appears in the Volume Types table.
6. Select the volume type’s View Extra Specs action.
7. Click Create and specify the Key and Value. The key-value pair must be valid; otherwise, specifying the volume type during volume creation will result in an error.
8. Click Create. The associated setting (key-value pair) now appears in the Extra Specs table.

Procedure

1. Log into the dashboard as an admin user.
2. Select Admin > Volumes > Volume Types.
3. In the Volume Types table, select the volume type’s View Extra Specs action.

4. On the Extra Specs table of this page, you can:

   • Add a new setting to the volume type. To do this, click Create and specify the key/value pair of the new setting you want to associate to the volume type.
   • Edit an existing setting associated with the volume type by selecting the setting’s Edit action.
   • Delete existing settings associated with the volume type by selecting the extra specs' check box and clicking Delete Extra Specs in this and the next dialog screen.

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Create a new cinder volume type and set the is-public flag to false:

   $ cinder type-create --is-public false <type_name>

   • Replace <type_name> with the name that you want to call this new private volume type.

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Define, clear, or list the default volume type for a project:

   Note

   You must replace <project_id> in these commands, with the ID of the required project. To find the ID and name of each tenant, run the openstack project list command.

   • To define the default volume type for a project:

     $ cinder --os-volume-api-version 3.62 default-type-set <volume_type> <project_id>

     • Replace <volume_type> with the name or ID of the required volume type. You can run the cinder type-list command to list the name and ID of all the volume types.

   • To clear the default volume type for a project:

     $ cinder --os-volume-api-version 3.62 default-type-unset <project_id>

   • To list the default volume type for a project:

     $ cinder --os-volume-api-version 3.62 default-type-list --project <project_id>

Procedure

1. Log into the dashboard as an admin user.
2. Select Admin > Volumes > Volume Types.
3. On the QoS Specs table, click Create QoS Spec.
4. Enter a name for the QoS Spec.
5. In the Consumer field, choose the consumer of this QoS specification. For more information, see Consumers of QoS specifications.
6. Click Create. The new QoS specification is displayed in the QoS Specs table.
7. In the QoS Specs table, select the Manage Specs action of your new QoS specification to open the Specs window, where you add the QoS performance limits.
8. Click Create in the Specs window to open the Create Extra Specs window.

9. Specify the QoS property key for a QoS performance limit in the Key field, and set the performance limit value in the Value field. For more information on the available property keys, see Block Storage QoS property keys.

   Important

   Ensure that you spell the QoS property keys correctly. The volume performance limits that contain incorrectly spelled property keys are ignored.

10. Click Create to add the QoS limit to your QoS specification.
11. Repeat steps 7 to 10 for each QoS limit that you want to add to your QoS specification.

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Create the QoS specification:

   $ openstack volume qos create [--consumer <qos_spec_consumer>] <qos_spec_name>

   • Optional: Replace <qos_spec_consumer> with the required consumer of this QoS specification. If not specified, the consumer defaults to both. For more information, see Consumers of QoS specifications.
   • Replace <qos_spec_name> with the name of your QoS specification.

3. Add the performance limits to the QoS specification, by specifying a separate --property <key=value> argument for each QoS limit that you want to add:

   $ openstack volume qos set --property <key>=<value> <qos_spec_name>

   • Replace <key> with the QoS property key of the required performance constraint. For more information, see Block Storage QoS property keys.

     Important

     Ensure that you spell the QoS property keys correctly. The volume performance limits that contain incorrectly spelled property keys are ignored.

   • Replace <value> with your deployment-specific limit for this performance constraint, in the measurement unit required by the QoS property key.

   • Replace <qos_spec_name> with the name or ID of your QoS specification.

     Example:

     $ openstack volume qos set \
      --property read_iops_sec=5000 \
      --property write_iops_sec=7000 \
      myqoslimits

4. Review the QoS specification:

   $ openstack volume qos list
   +--------------------------------------+---------+-----------+--------------+-----------------------------------------------------+
   | ID                                   | Name    | Consumer  | Associations | Properties                                          |
   +--------------------------------------+---------+-----------+--------------+-----------------------------------------------------+
   | 204c6ba2-c67c-4ac8-918a-03f101811235 | myqoslimits | front-end |              | read_iops_sec='5000', write_iops_sec='7000' |
   +--------------------------------------+---------+-----------+--------------+-----------------------------------------------------+

   This command provides a table of the configuration details of all the configured QoS specifications.

Procedure

1. Log into the dashboard as an admin user.
2. Select Admin > Volumes > Volume Types.
3. In the Volume Types table, select the Manage QoS Spec Association action of the required volume type.
4. Select the required QoS specification from the QoS Spec to be associated list.
5. Click Associate. The QoS specification is added to the Associated QoS Spec column of the edited volume type.

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Associate the required QoS specification with the required volume type:

   $ openstack volume qos associate <qos_spec_name> <volume_type>

   • Replace <qos_spec_name> with the name or ID of the QoS specification. You can run the openstack volume qos list command to list the name and ID of all the QoS specifications.
   • Replace <volume_type> with the name or ID of the volume type. You can run the cinder type-list command to list the name and ID of all the volume types.

3. Verify that the QoS specification has been associated:

   $ openstack volume qos list

   The Associations column of the output table shows which volume types are associated with this QoS specification.

Procedure

1. Log into the dashboard as an admin user.
2. Select Admin > Volumes > Volume Types.
3. In the Volume Types table, select the Manage QoS Spec Association action of the required volume type.
4. Select None from the QoS Spec to be associated list.

5. Click Associate.

   The QoS specification should be removed from the Associated QoS Spec column of the edited volume type.

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Disassociate the volume types associated with the QoS specification. You can either disassociate a specific volume type, or all volumes types when more than one volume type is associated to the same QoS specification:

   • To disassociate a specific volume type associated with the QoS specification:

     $ openstack volume qos disassociate <qos_spec_name> --volume-type <volume_type>

     • Replace <qos_spec_name> with the name or ID of the QoS specification. You can run the openstack volume qos list command to list the name and ID of all the QoS specifications.
     • Replace <volume_type> with the name or ID of the volume type associated with this QoS specification. You can run the cinder type-list command to list the name and ID of all the volume types.

   • To disassociate all volume types associated with the QoS specification:

     $ openstack volume qos disassociate <qos_spec_name> --all

3. Verify that the QoS specification has been disassociated:

   $ openstack volume qos list

   The Associations column of this QoS specification should either not specify the volume type or be empty.

Procedure

1. Log into the dashboard as an admin user.
2. Select Admin > Volumes > Volume Types.
3. In the Actions column of the volume to be encrypted, select Create Encryption to launch the Create Volume Type Encryption wizard.

4. From there, configure the Provider, Control Location, Cipher, and Key Size settings of the volume type’s encryption. The Description column describes each setting.

   Important

   The values listed below are the only supported options for Provider, Cipher, and Key Size.

   1. Enter luks for Provider.
   2. Enter aes-xts-plain64 for Cipher.
   3. Enter 256 for Key Size.
5. Click Create Volume Type Encryption.

1. Select Update Encryption from the Actions column of the volume type to launch the Update Volume Type Encryption wizard.
2. In Project > Compute > Volumes, check the Encrypted column in the Volumes table to determine whether the volume is encrypted.
3. If the volume is encrypted, click Yes in that column to view the encryption settings.

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. Create a volume type:

   $ cinder type-create myEncType

3. Configure the cipher, key size, control location, and provider settings:

   $ cinder encryption-type-create --cipher aes-xts-plain64 --key-size 256 --control-location front-end myEncType luks

4. Create an encrypted volume:

   $ cinder --debug create 1 --volume-type myEncType --name myEncVol

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

3. Edit an environment file and add a new entry to the parameter_defaults section. This ensures that the entries are updated in the containers and are retained whenever the environment is re-deployed by director with the openstack overcloud deploy command.

4. Add a new section to an environment file using CinderApiPolicies to set the consistency group settings. The equivalent parameter_defaults section with the default settings from the JSON file appear in the following way:

   parameter_defaults:
     CinderApiPolicies: { \
        cinder-consistencygroup_create: { key: 'consistencygroup:create', value: 'group:nobody' }, \
        cinder-consistencygroup_delete: { key: 'consistencygroup:delete', value: 'group:nobody' },  \
        cinder-consistencygroup_update: { key: 'consistencygroup:update', value: 'group:nobody' }, \
        cinder-consistencygroup_get: { key: 'consistencygroup:get', value: 'group:nobody' }, \
        cinder-consistencygroup_get_all: { key: 'consistencygroup:get_all', value: 'group:nobody' }, \
        cinder-consistencygroup_create_cgsnapshot: { key: 'consistencygroup:create_cgsnapshot', value: 'group:nobody' }, \
        cinder-consistencygroup_delete_cgsnapshot: { key: 'consistencygroup:delete_cgsnapshot', value: 'group:nobody' }, \
        cinder-consistencygroup_get_cgsnapshot: { key: 'consistencygroup:get_cgsnapshot', value: 'group:nobody' }, \
        cinder-consistencygroup_get_all_cgsnapshots: { key: 'consistencygroup:get_all_cgsnapshots', value: 'group:nobody' }, \
    }

5. The value ‘group:nobody’ determines that no group can use this feature so it is effectively disabled. To enable it, change the group to another value.

6. For increased security, set the permissions for both consistency group API and volume type management API to be identical. The volume type management API is set to "rule:admin_or_owner" by default in the same /etc/cinder/policy.json_ file:

   "volume_extension:types_manage": "rule:admin_or_owner",

7. To make the consistency groups feature available to all users, set the API policy entries to allow users to create, use, and manage their own consistency groups. To do so, use rule:admin_or_owner:

   CinderApiPolicies: { \
        cinder-consistencygroup_create: { key: 'consistencygroup:create', value: 'rule:admin_or_owner' }, \
        cinder-consistencygroup_delete: { key: 'consistencygroup:delete', value: 'rule:admin_or_owner' },  \
        cinder-consistencygroup_update: { key: 'consistencygroup:update', value: 'rule:admin_or_owner' }, \
        cinder-consistencygroup_get: { key: 'consistencygroup:get', value: 'rule:admin_or_owner' }, \
        cinder-consistencygroup_get_all: { key: 'consistencygroup:get_all', value: 'rule:admin_or_owner' }, \
        cinder-consistencygroup_create_cgsnapshot: { key: 'consistencygroup:create_cgsnapshot', value: 'rule:admin_or_owner' }, \
        cinder-consistencygroup_delete_cgsnapshot: { key: 'consistencygroup:delete_cgsnapshot', value: 'rule:admin_or_owner' }, \
        cinder-consistencygroup_get_cgsnapshot: { key: 'consistencygroup:get_cgsnapshot', value: 'rule:admin_or_owner' }, \
        cinder-consistencygroup_get_all_cgsnapshots: { key: 'consistencygroup:get_all_cgsnapshots', value: 'rule:admin_or_owner’ }, \
    }

8. Save the updates to your environment file.
9. Add your environment file to the stack with your other environment files and deploy the overcloud.

Procedure

1. Log into the dashboard as an admin user or a volume owner.
2. Select Project > Compute > Volumes > Volume Consistency Groups.
3. Click Create Consistency Group.
4. In the Consistency Group Information tab of the wizard, enter a name and description for your consistency group. Then, specify its Availability Zone.
5. You can also add volume types to your consistency group. When you create volumes within the consistency group, the Block Storage service will apply compatible settings from those volume types. To add a volume type, click its + button from the All available volume types list.
6. Click Create Consistency Group. It appears next in the Volume Consistency Groups table.

Procedure

1. Log into the dashboard as an admin user.
2. Select Project > Compute > Volumes > Volume Consistency Groups.
3. Optional: You can change the name or description of a consistency group by selecting Edit Consistency Group from its Action column.

4. To add or remove volumes from a consistency group directly, find the consistency group you want to configure. In the Actions column of that consistency group, select Manage Volumes. This launches the Add/Remove Consistency Group Volumes wizard.

   1. To add a volume to the consistency group, click its + button from the All available volumes list.
   2. To remove a volume from the consistency group, click its - button from the Selected volumes list.
5. Click Edit Consistency Group.

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. List all available consistency groups and their respective IDs:

   $ cinder consisgroup-list

3. Create snapshots using the consistency group:

   $ cinder cgsnapshot-create [--name <cgsnapname>] [--description "<description>"] <cgnameid>

   • Replace <cgsnapname> with the name of the snapshot.
   • Replace <description> with a description of the snapshot.
   • Replace <cgnameid> with the name or ID of the consistency group.

4. Display a list of all available consistency group snapshots:

   # cinder cgsnapshot-list

Procedure

1. Source the overcloud credentials file:

   $ source ~/<credentials_file>

   • Replace <credentials_file> with the name of your credentials file, for example, overcloudrc.

2. To clone an existing consistency group:

   $ cinder consisgroup-create-from-src --source-cg <cgnameid> [--name <cgname>] [--description "<description>"]

   • Replace <cgnameid> with the name or ID of the consistency group you want to clone.
   • Replace <cgname> with the name of your consistency group.
   • Replace <description> with a description of your consistency group.

3. To create a consistency group from a consistency group snapshot:

   $ cinder consisgroup-create-from-src --cgsnapshot <cgsnapname> [--name <cgname>] [--description "<description>"]

   • Replace <cgsnapname> with the name or ID of the snapshot you are using to create the consistency group.

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

3. Use an overrides environment file or create a new one, for example multipath_overrides.yaml. Add and set the following parameter:

   parameter_defaults:
     ExtraConfig:
       cinder::config::cinder_config:
         backend_defaults/use_multipath_for_image_xfer:
           value: true

   Note

   The default settings will generate a basic multipath configuration that works for most environments. However, check with your storage vendor for recommendations, because some vendors have optimized configurations that are specific to their hardware. For more information about multipath, see Configuring device mapper multipath.

4. Optional: If you have a multipath configuration file for your overcloud deployment, then you can use the MultipathdCustomConfigFile parameter to specify the location of this file:

   parameter_defaults:
     MultipathdCustomConfigFile: <config_file_directory>/<config_file_name>

   In the following example, /home/stack is the directory of the multipath configuration file and multipath.conf is the name of this file:

   parameter_defaults:
     MultipathdCustomConfigFile: /home/stack/multipath.conf

   Note

   Other TripleO multipath parameters override any corresponding value in the local custom configuration file. For example, if MultipathdEnableUserFriendlyNames is False, the files on the overcloud nodes are updated to match, even if the setting is enabled in the local custom file.

   For more information about multipath parameters, see Multipath heat template parameters.

5. Save the updates to your overrides environment file.

6. Add your overrides environment file to the stack with your other environment files, such as:

   ----
   /usr/share/openstack-tripleo-heat-templates/environments/multipathd.yaml
   ----

7. Deploy the overcloud.

Procedure

1. Create an instance.
2. Attach a non-encrypted volume to the instance.

3. Get the name of the Compute node that contains the instance:

   $ nova show <instance> | grep OS-EXT-SRV-ATTR:host

   Replace <instance> with the name of the instance that you created.

4. Retrieve the virsh name of the instance:

   $ nova show <instance> | grep instance_name

5. Get the IP address of the Compute node:

   $ . stackrc
   $ metalsmith list | grep <compute_name>

   Replace <compute_name> with the name from the output of the nova show <instance> command to display two rows, from a table of six columns.

   Find the row in which <compute_name> is in the fourth column. The IP address of <compute_name> is in the last column of this row.

   In the following example, the IP address of compute-0 is 192.168.24.15 because compute-0 is in the fourth column of the second row:

   $ . stackrc
   $ metalsmith list | grep compute-0
   | 3b1bf72e-c425-494c-9717-d0b89bb66580 | compute-0    | 95b21d3e-36be-470d-ba5c-70d5dcd6d0b3 | compute-1    | ACTIVE | ctlplane=192.168.24.49 |
   | 72a24883-25f9-435c-bf71-a20e66be172d | compute-1    | a59f79f7-006e-4f38-a9ad-8164da47d58e | compute-0    | ACTIVE | ctlplane=192.168.24.15 |

6. SSH into the Compute node that runs the instance:

   $ ssh tripleo-admin@<compute_node_ip>

   Replace <compute_node_ip> with the IP address of the Compute node.

7. Log in to the container that runs virsh:

   $ podman exec -it nova_libvirt /bin/bash

8. Enter the following command on a Compute node instance to verify that it is using multipath in the cinder volume host location:

   virsh domblklist <virsh_instance_name> | grep /dev/dm

   Replace <virsh_instance_name> with the output of the nova show <instance> | grep instance_name command.

   If the instance shows a value other than /dev/dm-, the connection is non-multipath and you must refresh the connection info with the nova shelve and nova unshelve commands:

   $ nova shelve <instance>
   $ nova unshelve <instance>

   Note

   If you have more than one type of back end, you must verify the instances and volumes on all back ends, because connection info that each back end returns might vary.