Chapter 4. Troubleshooting the Block Storage backup service

Procedure

1. Run the openstack volume service list command:

   # openstack volume service list
   +------------------+-------------------------+------+---------+-------+----------------------------+
   | Binary           | Host                    | Zone | Status  | State | Updated At                 |
   +------------------+-------------------------+------+---------+-------+----------------------------+
   | cinder-scheduler | controller-0            | nova | enabled | up    | 2023-06-21T13:07:42.000000 |
   | cinder-scheduler | controller-1            | nova | enabled | up    | 2023-06-21T13:07:42.000000 |
   | cinder-scheduler | controller-2            | nova | enabled | up    | 2023-06-21T13:07:42.000000 |
   | cinder-backup    | controller-0            | nova | enabled | up    | 2023-06-21T13:07:46.000000 |
   | cinder-backup    | controller-1            | nova | enabled | up    | 2023-06-21T13:07:46.000000 |
   | cinder-backup    | controller-2            | nova | enabled | up    | 2023-06-21T13:07:46.000000 |
   | cinder-volume    | hostgroup@tripleo_iscsi | nova | enabled | up    | 2023-06-21T13:07:47.000000 |
   +------------------+-------------------------+------+---------+-------+----------------------------+

2. Verify that the State entry of every service is up. If not, examine the relevant log files. For more information about the location of these log files, see Block Storage (cinder) Log Files in Managing overcloud observability.
3. Verify that the Block Storage backup service is deployed successfully, by backing up any Block Storage volume and ensuring that the backup succeeds. For more information, see Troubleshooting backups.

1. The user issues a request to the cinder API, which is a REST API, to back up a Block Storage volume.
2. The cinder API receives the request from HAProxy and validates the request, the user credentials, and other information.
3. The cinder API creates the backup record in the SQL database.
4. The cinder API makes an asynchronous RPC call to the cinder-backup service via AMQP to back up the volume.
5. The cinder API returns the current backup record, with an ID, to the API caller.
6. An RPC create message arrives on one of the backup services.
7. The cinder-backup service performs a synchronous RPC call to get_backup_device.
8. The cinder-volume service ensures that the correct device is returned to the caller. Normally, it is the same volume, but if the volume is in use, the service returns a temporary cloned volume or a temporary snapshot, depending on the configuration.
9. The cinder-backup service issues another synchronous RPC to cinder-volume to expose the source device.
10. The cinder-volume service exports and maps the source device (volume or snapshot) and returns the appropriate connection information.
11. The cinder-backup service attaches the source device by using the connection information.
12. The cinder-backup service calls the backup back end driver, with the device already attached, which begins the data transfer to the backup repository.
13. The source device is detached from the Backup host.
14. The cinder-backup service issues a synchronous RPC to cinder-volume to disconnect the source device.
15. The cinder-volume service unmaps and removes the export for the device.
16. If a temporary volume or temporary snapshot was created, cinder-backup calls cinder-volume to remove it.
17. The cinder-volume service removes the temporary volume.
18. When the backup is completed, the backup record is updated in the database.

1. The user issues a request to the cinder API, which is a REST API, to restore a Block Storage backup.
2. The cinder API receives the request from HAProxy and validates the request, the user credentials, and other information.
3. If the request does not contain an existing volume as the destination, the cinder API makes an asynchronous RPC call to create a new volume and polls the status of the volume until it becomes available.
4. The cinder-scheduler selects a volume service and makes the RPC call to create the volume.
5. The selected cinder-volume service creates the volume.
6. When the cinder API detects that the volume is available, the backup record is created in the database.
7. The cinder API makes an asynchronous RPC call to the backup service via AMQP to restore the backup.
8. The cinder API returns the current volume ID, backup ID, and volume name to the API caller.
9. An RPC create message arrives on one of the backup services.
10. The cinder-backup service performs a synchronous RPC call to cinder-volume to expose the volume.
11. The cinder-volume service exports and maps the volume returning the appropriate connection information.
12. The cinder-backup service attaches the volume by using the connection information.
13. The cinder-backup service calls the back end driver with the volume already attached, which begins the data restoration to the volume.
14. The volume is detached from the backup host.
15. The cinder-backup service issues a synchronous RPC to cinder-volume to disconnect the volume.
16. The cinder-volume service unmaps and removes the export for the volume.
17. When the volume is restored, the backup record is updated in the database.