Chapter 2. Block Storage and Volumes

The cinder-volume-active-active.yaml file enables you to deploy the Block Storage service in an active-active configuration. This file ensures director uses the non-Pacemaker cinder-volume heat template and adds the etcd service to the deployment as a distributed lock manager (DLM).

The cinder-volume-active-active.yaml file also defines the active-active cluster name by assigning a value to the CinderVolumeCluster parameter. CinderVolumeCluster is a global Block Storage parameter. Therefore, you cannot include clustered (active-active) and non-clustered back ends in the same deployment.

Procedure

To enable active-active Block Storage service volumes, include the following environment file in your overcloud deployment:

-e /usr/share/openstack-tripleo-heat-templates/environments/cinder-volume-active-active.yaml

After deploying an active-active configuration, there are several commands you can use to interact with the environment when using API version 3.17 and later.

The unmanage and manage mechanisms facilitate moving volumes from one service using version X to another service using version X+1. Both services remain running during this process.

In API version 3.17 or later, you can see lists of volumes and snapshots that are available for management in Block Storage clusters. To see these lists, use the --cluster argument with cinder manageable-list or cinder snapshot-manageable-list.

In API version 3.16 and later, the cinder manage command also accepts the optional --cluster argument so that you can add previously unmanaged volumes to a Block Storage cluster.

With API version 3.16 and later, the cinder migrate and cinder-manage commands accept the --cluster argument to define the destination for active-active deployments.

When you migrate a volume on a Block Storage clustered service, pass the optional --cluster argument and omit the host positional argument, because the arguments are mutually exclusive.

All Block Storage volume services perform their own maintenance when they start. In an environment with multiple volume services grouped in a cluster, you can clean up services that are not currently running.

The command work-cleanup triggers server cleanups. The command returns:

Alternatively, you can query the Block Storage host directly to determine which well-defined standard Extra Specs are supported by its driver. Start by logging in (through the command line) to the node hosting the Block Storage service. Then:

# cinder service-list

This command will return a list containing the host of each Block Storage service (cinder-backup, cinder-scheduler, and cinder-volume). For example:

+------------------+---------------------------+------+---------
|      Binary      |            Host           | Zone |  Status ...
+------------------+---------------------------+------+---------
|  cinder-backup   |   localhost.localdomain   | nova | enabled ...
| cinder-scheduler |   localhost.localdomain   | nova | enabled ...
|  cinder-volume   | *localhost.localdomain@lvm* | nova | enabled ...
+------------------+---------------------------+------+---------

To display the driver capabilities (and, in turn, determine the supported Extra Specs) of a Block Storage service, run:

# cinder get-capabilities _VOLSVCHOST_

Where VOLSVCHOST is the complete name of the cinder-volume's host. For example:

# cinder get-capabilities localhost.localdomain@lvm
    +---------------------+-----------------------------------------+
    |     Volume stats    |                        Value            |
    +---------------------+-----------------------------------------+
    |     description     |                         None            |
    |     display_name    |                         None            |
    |    driver_version   |                        3.0.0            |
    |      namespace      | OS::Storage::Capabilities::localhost.loc...
    |      pool_name      |                         None            |
    |   storage_protocol  |                        iSCSI            |
    |     vendor_name     |                     Open Source         |
    |      visibility     |                         None            |
    | volume_backend_name |                         lvm             |
    +---------------------+-----------------------------------------+
    +--------------------+------------------------------------------+
    | Backend properties |                        Value             |
    +--------------------+------------------------------------------+
    |    compression     |      {u'type': u'boolean', u'description'...
    |        qos         |              {u'type': u'boolean', u'des ...
    |    replication     |      {u'type': u'boolean', u'description'...
    | thin_provisioning  | {u'type': u'boolean', u'description': u'S...
    +--------------------+------------------------------------------+

The Backend properties column shows a list of Extra Spec Keys that you can set, while the Value column provides information on valid corresponding values.

By default, all volume types are accessible to all OpenStack projects. If you need to create volume types with restricted access, you will need to do so through the CLI. For instructions, see Section 2.2.2.5, “Create and Configure Private Volume Types”.

To delete a volume type, select its corresponding check boxes from the Volume Types table and click Delete Volume Types.

By default, all volume types are available to all projects. You can create a restricted volume type by marking it private. To do so, set the type’s is-public flag to false.

Private volume types are useful for restricting access to volumes with certain attributes. Typically, these are settings that should only be usable by specific projects; examples include new back ends or ultra-high performance configurations that are being tested.

To create a private volume type, run:

$ cinder type-create --is-public false  <TYPE-NAME>

By default, private volume types are only accessible to their creators. However, admin users can find and view private volume types using the following command:

$ cinder type-list --all

This command lists both public and private volume types, and it also includes the name and ID of each one. You need the volume type’s ID to provide access to it.

Access to a private volume type is granted at the project level. To grant a project access to a private volume type, run:

$ cinder  type-access-add --volume-type <TYPE-ID> --project-id <TENANT-ID>

To view which projects have access to a private volume type, run:

$ cinder  type-access-list --volume-type <TYPE-ID>

To remove a project from the access list of a private volume type, run:

$ cinder  type-access-remove --volume-type <TYPE-ID> --project-id <TENANT-ID>

You can define performance limits for volumes on a per-volume basis using basic volume QOS values. The Block Storage service supports the following options:

As an administrator, you can create and configure a QOS Spec through the QOS Specs table. You can associate more than one key/value pair to the same QOS Spec.

You can use volume types to implement capacity-derived Quality-of-Service (QoS) limits on volumes. This will allow you to set a deterministic IOPS throughput based on the size of provisioned volumes. Doing this simplifies how storage resources are provided to users — namely, providing a user with pre-determined (and, ultimately, highly predictable) throughput rates based on the volume size they provision.

In particular, the Block Storage service allows you to set how much IOPS to allocate to a volume based on the actual provisioned size. This throughput is set on an IOPS per GB basis through the following QoS keys:

read_iops_sec_per_gb
write_iops_sec_per_gb
total_iops_sec_per_gb

These keys allow you to set read, write, or total IOPS to scale with the size of provisioned volumes. For example, if the volume type uses read_iops_sec_per_gb=500, then a provisioned 3GB volume would automatically have a read IOPS of 1500.

Capacity-derived QoS limits are set per volume type, and configured like any normal QoS spec. In addition, these limits are supported by the underlying Block Storage service directly, and is not dependent on any particular driver.

For more information about volume types, see Section 2.2.2, “Group Volume Settings with Volume Types” and Section 2.2.2.2, “Create and Configure a Volume Type”. For instructions on how to set QoS specs, Section 2.2.5, “Use Quality-of-Service Specifications”.

See Section 2.3.12, “Changing the volume type (volume re-typing)” for information about volume re-typing.

As an administrator, you can associate a QOS Spec to an existing volume type using the Volume Types table.

To create encrypted volumes, you first need an encrypted volume type. Encrypting a volume type involves setting what provider class, cipher, and key size it should use:

Once you have an encrypted volume type, you can invoke it to automatically create encrypted volumes. For more information on creating a volume type, see Section 2.2.2.2, “Create and Configure a Volume Type”. Specifically, select the encrypted volume type from the Type drop-down list in the Create Volume window (see Section 2.3, “Basic volume usage and configuration”).

To configure an encrypted volume type through the CLI, see Section 2.2.6.2, “Configure Volume Type Encryption Through the CLI”.

You can also re-configure the encryption settings of an encrypted volume type.

To configure Block Storage volume encryption, do the following:

The Block Storage service (cinder) creates an encryption key in the Key Management service (barbican) when it uploads an encrypted volume to the Image service (glance). This creates a 1:1 relationship between an encryption key and a stored image.

Encryption key deletion prevents unlimited resource consumption of the Key Management service. The Block Storage, Key Management, and Image services automatically manage the key for an encrypted volume, including the deletion of the key.

The Block Storage service automatically adds two properties to a volume image:

When you create a volume image, the Block Storage service sets the cinder_encryption_key_deletion_policy property to on_image_deletion. When you delete a volume image, the Image service deletes the corresponding encryption key if the cinder_encryption_key_deletion_policy equals on_image_deletion.

For additional information, refer to the Manage secrets with the OpenStack Key Manager guide.

By default, Block Storage security policy disables consistency groups APIs. You need to enable it here before using the feature. The related consistency group entries in /etc/cinder/policy.json of the node hosting the Block Storage API service (namely, openstack-cinder-api) list the default settings:

"consistencygroup:create" : "group:nobody",
​"consistencygroup:delete": "group:nobody",
​"consistencygroup:update": "group:nobody",
​"consistencygroup:get": "group:nobody",
​"consistencygroup:get_all": "group:nobody",
​"consistencygroup:create_cgsnapshot" : "group:nobody",
​"consistencygroup:delete_cgsnapshot": "group:nobody",
​"consistencygroup:get_cgsnapshot": "group:nobody",
​"consistencygroup:get_all_cgsnapshots": "group:nobody",

These settings need to be changed in an environment file and then deployed to the overcloud using the openstack overcloud deploy command. If you edit the JSON file directly, the changes will be overwritten next time the overcloud is deployed.

To enable the consistency groups, edit an environment file and add a new entry to the parameter_defaults section. This will ensure that the entries are updated in the containers and are retained whenever the environment is re-deployed using director with the openstack overcloud deploy command.

Add a new section to an environment file using CinderApiPolicies to set the consistency group settings. The equivalent parameter_defaults section showing the default settings from the JSON file would look like this:

parameter_defauts:
  CinderApiPolicies: { \
     cinder-consistencygroup_create: { key: 'consistencygroup:create', value: 'group:nobody' }, \
     cinder-consistencygroup_delete: { key: 'consistencygroup:delete', value: 'group:nobody' },  \
     cinder-consistencygroup_update: { key: 'consistencygroup:update', value: 'group:nobody' }, \
     cinder-consistencygroup_get: { key: 'consistencygroup:get', value: 'group:nobody' }, \
     cinder-consistencygroup_get_all: { key: 'consistencygroup:get_all', value: 'group:nobody' }, \
     cinder-consistencygroup_create_cgsnapshot: { key: 'consistencygroup:create_cgsnapshot', value: 'group:nobody' }, \
     cinder-consistencygroup_delete_cgsnapshot: { key: 'consistencygroup:delete_cgsnapshot', value: 'group:nobody' }, \
     cinder-consistencygroup_get_cgsnapshot: { key: 'consistencygroup:get_cgsnapshot', value: 'group:nobody' }, \
     cinder-consistencygroup_get_all_cgsnapshots: { key: 'consistencygroup:get_all_cgsnapshots', value: 'group:nobody' }, \
 }

The value ‘group:nobody’ determines that no group can use this feature, effectively disabling it. To enable it, you will need to change the group to another value.

For increased security, set the permissions for both consistency group API and volume type management API be identical. The volume type management API is set to "rule:admin_or_owner" by default (in the same /etc/cinder/policy.json file):

"volume_extension:types_manage": "rule:admin_or_owner",

You can make the consistency groups feature available to all users by setting the API policy entries to allow users to create, use, and manage their own consistency groups. To do so, use rule:admin_or_owner:

CinderApiPolicies: { \
     cinder-consistencygroup_create: { key: 'consistencygroup:create', value: 'rule:admin_or_owner' }, \
     cinder-consistencygroup_delete: { key: 'consistencygroup:delete', value: 'rule:admin_or_owner' },  \
     cinder-consistencygroup_update: { key: 'consistencygroup:update', value: 'rule:admin_or_owner' }, \
     cinder-consistencygroup_get: { key: 'consistencygroup:get', value: 'rule:admin_or_owner' }, \
     cinder-consistencygroup_get_all: { key: 'consistencygroup:get_all', value: 'rule:admin_or_owner' }, \
     cinder-consistencygroup_create_cgsnapshot: { key: 'consistencygroup:create_cgsnapshot', value: 'rule:admin_or_owner' }, \
     cinder-consistencygroup_delete_cgsnapshot: { key: 'consistencygroup:delete_cgsnapshot', value: 'rule:admin_or_owner' }, \
     cinder-consistencygroup_get_cgsnapshot: { key: 'consistencygroup:get_cgsnapshot', value: 'rule:admin_or_owner' }, \
     cinder-consistencygroup_get_all_cgsnapshots: { key: 'consistencygroup:get_all_cgsnapshots', value: 'rule:admin_or_owner’ }, \
 }

Once you have created the environment file file in /home/stack/templates/, log in as the stack user. Then, deploy the configuration by running:

$ openstack overcloud deploy --templates \
-e /home/stack/templates/<ENV_FILE>.yaml

Where ENV_FILE.yaml is the name of the file with the ExtraConfig settings added earlier.

For additional information on the openstack overcloud deploy command, refer to Creating the Overcloud with the CLI Tools section in the Director Installation and Usage Guide.

After enabling the consistency groups API, you can then start creating consistency groups. To do so:

You can change the name or description of a consistency group by selecting Edit Consistency Group from its Action column.

In addition, you can also add or remove volumes from a consistency group directly. To do so:

After adding volumes to a consistency group, you can now create snapshots from it. Before doing so, first log in as admin user from the command line on the node hosting the openstack-cinder-api and run:

# export OS_VOLUME_API_VERSION=2

Doing so will configure the client to use version 2 of openstack-cinder-api.

To list all available consistency groups (along with their respective IDs, which you will need later):

# cinder consisgroup-list

To create snapshots using the consistency group, run:

# cinder cgsnapshot-create --name CGSNAPNAME --description "DESCRIPTION" CGNAMEID

Where:

To display a list of all available consistency group snapshots, run:

` # cinder cgsnapshot-list `

Consistency groups can also be used to create a whole batch of pre-configured volumes simultaneously. You can do this by cloning an existing consistency group or restoring a consistency group snapshot. Both processes use the same command.

To clone an existing consistency group:

# cinder consisgroup-create-from-src --source-cg CGNAMEID --name CGNAME --description "DESCRIPTION"

Where: - CGNAMEID is the name or ID of the consistency group you want to clone. - CGNAME is the name of your consistency group (optional). - DESCRIPTION is a description of your consistency group (optional).

To create a consistency group from a consistency group snapshot:

# cinder consisgroup-create-from-src --cgsnapshot CGSNAPNAME --name CGNAME --description "DESCRIPTION"

Replace CGSNAPNAME with the name or ID of the snapshot you are using to create the consistency group.

To attach a volume to multiple instances, set the multiattach flag to <is>True in the volume extra specs. When you create a multi-attach volume type, the volume inherits the flag and becomes a multi-attach volume.

You can retype a volume to be multi-attach capable or retype a multi-attach capable volume to make it incapable of attaching to multiple instances. However, you can retype a volume only when it is not in use and its status is available.

When you attach a multi-attach volume, some hypervisors require special considerations, such as when you disable caching. Currently, there is no way to safely update an attached volume while keeping it attached the entire time. Retyping fails if you attempt to retype a volume that is attached to multiple instances.

After you create a multi-attach volume type, create a multi-attach volume.

You can now attach the volume to multiple instances. For information about how to attach a volume to an instance, see Attach a volume to an instance.

The Block Storage back end must support multi-attach. For information about supported back ends, contact Red Hat Support.

Create a volume transfer from the dashboard

Accept a volume transfer from the dashboard

When using Red Hat Ceph Storage as a back end for your OpenStack deployment, you can set a snapshot to protected in the back end. Attempting to delete protected snapshots through OpenStack (as in, through the dashboard or the cinder snapshot-delete command) will fail.

When this occurs, set the snapshot to unprotected in the Red Hat Ceph back end first. Afterwards, you should be able to delete the snapshot through OpenStack as normal.

For related instructions, see Protecting a Snapshot and Unprotecting a Snapshot.

When you migrate a volume between hosts, both hosts must reside on the same back end. Use the dashboard to migrate a volume between hosts:

Migrating a volume between back ends involves volume re-typing. This means that in order to migrate to a new back end, the following must be true:

For more information, see:

When you define the back end as an Extra Spec, use volume_backend_name as the Key. Its corresponding value is the name of the back end, as defined in the Block Storage configuration file, /etc/cinder/cinder.conf. In this file, each back end is defined in its own section, and its corresponding name is set in the volume_backend_name parameter.

After you have a back end defined in a target volume type, you can migrate a volume to that back end by using the re-typing procedure. This involves re-applying the target volume type to a volume, thereby applying the new back end settings. For more information, see Section 2.3.12, “Changing the volume type (volume re-typing)”.