Configuring Red Hat OpenStack with Red Hat Storage

Procedure 4.1. Configure Openstack Block Storage to Use Red Hat Storage Volume

1. Update the cinder.conf File

   Set the glusterFS default configuration parameters in the cinder.conf file as follows:

   # openstack-config --set /etc/cinder/cinder.conf DEFAULT volume_driver cinder.volume.drivers.glusterfs.GlusterfsDriver
   # openstack-config --set /etc/cinder/cinder.conf DEFAULT glusterfs_shares_config /etc/cinder/shares.conf
   # openstack-config --set /etc/cinder/cinder.conf DEFAULT glusterfs_mount_point_base /var/lib/cinder/volumes

2. Create the shares.conf File

   Create the /etc/cinder/shares.conf file. Add a list of Red Hat Storage Volumes from different trusted storage pools with the client mount point for each. Each storage volume and mount point must be on a new line, as follows:

   STORAGEHOST:VOLUME
   STORAGEHOST1:NEXTVOLUME
   STORAGEHOSTHOST2:ANOTHERVOLUME

   An example of this configuration is as follows:

   10.1.1.11:/cinder-vol
   10.1.2.12:/nextcinder-vol
   10.1.3.13:/anothercinder-vol

   When listing the storage volume and mount point, Red Hat recommends that you use the backup-volfile-servers=volfile_server2:volfile_server3:...:volfile_serverN option to list the backup volfile servers to mount the client. If this option is specified while mounting the FUSE client, and if the first volfile server fails, the servers specified in the backup-volfile-servers option are used as volfile servers to mount the client until the mount operation is successful. Add this option to the storage volume and mount point list in /etc/cinder/shares.conf file as shown below:

   10.1.1.11:/cinder-vol -o backup-volfile-servers=volfile_server2:volfile_server3

3. Restart Services

   Restart the cinder services as follows:

   # service openstack-cinder-api restart
   # service openstack-cinder-volume restart
   # service openstack-cinder-scheduler restart

Procedure 4.2. Configure the OpenStack Image Service to Use Red Hat Storage Volumes

1. Specify the location for filesystem_store_datadir as follows:

   # ============ Filesystem Store Options===================
   # Directory that the Filesystem backend store
   # writes image data to
   filesystem_store_datadir = /mnt/gluster/glance/images/

2. Create a directory named images as follows:

   # mkdir -p /mnt/gluster/glance/images

3. Change the ownership of the the Red Hat Storage volume as follows:

   chown -R glance:glance/mnt/gluster/glance/

4. Mount the Red Hat Storage volume on filesystem_store_datadir as follows:

   mount -t glusterfs 10.1.1.11:/glance-vol /mnt/gluster

   When mounting the volume, Red Hat recommends you to use backup-volfile-servers=volfile_server2:volfile_server3:...:volfile_serverN option to list the backup volfile servers to mount the client. If this option is specified while mounting the fuse client, when the first volfile server fails, the servers specified in backup-volfile-servers option are used as volfile servers to mount the client until the mount is successful.
5. To persistently mount the glance volume, add the following line to the /etc/fstab file:

   10.1.1.11:/glance-vol images glusterfs defaults,_netdev 0 0

6. Restart the OpenStack glance service as follows:

   # service openstack-glance-api restart

Procedure 4.3. Configure the OpenStack Compute Service to Use Red Hat Storage Volumes

1. Configure the Compute service to the use Red Hat Storage volume by editing the instances_path in the /etc/nova/nova.conf file:

   #instances_path = /mnt/gluster/instance

2. Create a directory named instance as follows:

   # mkdir -p /mnt/gluster/instance

   If you are using a different volume other than the glance volume for compute service, then you must mount the volume using mount -t glusterfs 10.1.1.11:/nova-vol /mnt/nova command.
   When mounting the volume, Red Hat recommends you to use the backup-volfile-servers=volfile_server2:volfile_server3:...:volfile_serverN option to list the backup volfile servers to mount the client. If this option is specified while mounting the FUSE client, when the first volfile server fails, the servers specified in backup-volfile-servers option are used as volfile servers to mount the client until the mount operation is successful.
3. Change the ownership of the Red Hat Storage volume as follows:

   chown -R nova:nova /mnt/gluster/instance

4. Restart the OpenStack Compute service as follows:

   # service openstack-nova-compute restart

Procedure 4.4. Verify the Integrated Setup

1. Verify that the Red Hat Storage Volumes are successfully mounted as follows:

   # mount | grep gluster
   10.1.1.11:/glance-vol on /mnt/gluster/glance/images type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
   10.1.1.11:cinder-vol on /var/lib/cinder/volumes/586c24173ac3ab5d1d43aed1f113d9f6 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)

2. Verify that the Cinder volume is mounted on the Red Hat Storage volumes as follows:

   # getfattr -d -etext -m. -n trusted.glusterfs.pathinfo /var/lib/cinder/volumes/586c24173ac3ab5d1d43aed1f113d9f6
   
   # file: var/lib/cinder/volumes/586c24173ac3ab5d1d43aed1f113d9f6
   trusted.glusterfs.pathinfo="((<DISTRIBUTE:cinder-vol-dht> (<REPLICATE:cinder-vol-replicate-0> <POSIX(/rhs/brick1/cinder-vol):dhcpzz-183.example.com:/rhs/brick1/cinder-vol/> <POSIX(/rhs/brick1/cinder-
   vol):dhcpzz-223.example.com:/rhs/brick1/cinder-vol/>) (<REPLICATE:cinder-vol-replicate-1> <POSIX(/rhs/brick2/cinder-vol):dhcpzz-223.example.com:/rhs/brick2/cinder-vol/> <POSIX(/rhs/brick2/
   cinder-vol):dhcpzz-183.example.com:/rhs/brick2/cinder-vol/>)) (cinder-vol-dht-layout (cinder-vol-replicate-0 0 2147483646) (cinder-vol-replicate-1 2147483647 4294967295)))"

3. Verify that the glance image was created on the Red Hat Storage volumes as follows:

   # glance image-create --name="test" --is-public=true --container-format=ovf --disk-format=qcow2 < f17-x86_64-openstack-sda.qcow2                    
   
                     
   +------------------+--------------------------------------+
   | Property         | Value                                |
   +------------------+--------------------------------------+
   | checksum         | 1f104b5667768964d5df8c4ad1d7cd27     |
   | container_format | ovf                                  |
   | created_at       | 2013-07-30T12:56:05                  |
   | deleted          | False                                |
   | deleted_at       | None                                 |
   | disk_format      | qcow2                                |
   | id               | a66213ee-1a76-4d4a-959d-5df3f8f320ac |
   | is_public        | True                                 |
   | min_disk         | 0                                    |
   | min_ram          | 0                                    |
   | name             | test                                 |
   | owner            | 84e2f0fac93d402287a8eb97b6ba9711     |
   | protected        | False                                |
   | size             | 251985920                            |
   | status           | active                               |
   | updated_at       | 2013-07-30T12:56:49                  |
   +------------------+--------------------------------------+
   
   # ls -l /mnt/gluster/glance/images/
   total 246080
   -rw-r-----. 1 glance glance 251985920 Jul 30 18:26 a66213ee-1a76-4d4a-959d-5df3f8f320ac
   
   
   ~(keystone_admin)]# getfattr -d -etext -m. -n trusted.glusterfs.pathinfo /mnt/gluster/glance/images/a66213ee-1a76-4d4a-959d-5df3f8f320ac                                                            
   getfattr: Removing leading '/' from absolute path names           # file: var/lib/glance/images/a66213ee-1a76-4d4a-959d-5df3f8f320ac
   trusted.glusterfs.pathinfo="(<DISTRIBUTE:glance-vol-dht> (<REPLICATE:glance-vol-replicate-1> <POSIX(/rhs/brick4/glance-vol):dhcpzz-223.example.com:/rhs/brick4/glance-vol/a66213ee-1a76-4d4a-959d-5df3f
   8f320ac> <POSIX(/rhs/brick4/glance-vol):dhcpzz-183.example.com:/rhs/brick4/glance-vol/a66213ee-1a76-4d4a-959d-5df3f8f320ac>))"

4. Verify that the nova instance was created on the Red Hat Storage volumes as follows:

   # nova list
   +--------------------------------------+-----------+--------+------------+-------------+---------------------+
   | ID                                   | Name      | Status | Task State | Power State | Networks            |
   +--------------------------------------+-----------+--------+------------+-------------+---------------------+
   | 9b8f4140-665b-4959-abcd-5970c979e9df | instance1 | ACTIVE | None       | Running     | public=172.24.4.227 |
   +--------------------------------------+-----------+--------+------------+-------------+---------------------+
   
   
   # getfattr -d -etext -m. -n trusted.glusterfs.pathinfo /mnt/gluster/instance/9b8f4140-665b-4959-abcd-5970c979e9df/disk
   getfattr: Removing leading '/' from absolute path names
   
   # file: mnt/gluster/instance/9b8f4140-665b-4959-abcd-5970c979e9df
   trusted.glusterfs.pathinfo="(<DISTRIBUTE:glance-vol-dht> (<REPLICATE:glance-vol-replicate-5> <POSIX(/rhs/brick4/glance-vol):rhs-vm4:/rhs/brick4/glance-vol/glance/images/ 9b8f4140-665b-4959-abcd-5970c979e9df> <POSIX(/rhs/brick4/glance-vol):rhs-vm3:/rhs/brick4/glance-vol/glance/images/9b8f4140-665b-4959-abcd-5970c979e9df>))"