Chapter 7. Working with volume snapshots
Cluster administrators can use volume snapshots to help protect against data loss by using the supported MicroShift logical volume manager storage (LVMS) Container Storage Interface (CSI) provider. Familiarity with persistent volumes is required.
A snapshot represents the state of the storage volume in a cluster at a particular point in time. Volume snapshots can also be used to provision new volumes. Snapshots are created as read-only logical volumes (LVs) located on the same device as the original data.
A cluster administrator can complete the following tasks using CSI volume snapshots:
- Create a snapshot of an existing persistent volume claim (PVC).
- Back up a volume snapshot to a secure location.
- Restore a volume snapshot as a different PVC.
- Delete an existing volume snapshot.
Only the logical volume manager storage (LVMS) plugin CSI driver is supported by MicroShift.
7.1. About LVM thin volumes Copy linkLink copied to clipboard!
To use advanced storage features such as creating volume snapshots or volume cloning, you must perform the following actions:
- Configure both the logical volume manager storage (LVMS) provider and the cluster.
- Provision a logical volume manager (LVM) thin-pool on the RHEL for Edge host.
- Attach LVM thin-pools to a volume group.
To create Container Storage Interface (CSI) snapshots, you must configure thin volumes on the RHEL for Edge host. The CSI does not support volume shrinking.
For LVMS to manage thin logical volumes (LVs), a thin-pool device-class
array must be specified in the etc/lvmd.yaml
configuration file. Multiple thin-pool device classes are permitted.
If additional storage pools are configured with device classes, then additional storage classes must also exist to expose the storage pools to users and workloads. To enable dynamic provisioning on a thin-pool, a StorageClass
resource must be present on the cluster. The StorageClass
resource specifies the source device-class
array in the topolvm.io/device-class
parameter.
Example lvmd.yaml
file that specifies a single device class for a thin-pool
- 1
- String. The UNIX domain socket endpoint of gRPC. Defaults to
/run/lvmd/lvmd.socket
. - 2
- A list of maps for the settings for each
device-class
. - 3
- String. The unique name of the
device-class
. - 4
- Unsigned 64-bit integer. Storage capacity in GB to be left unallocated in the volume group. Defaults to
0
. - 5
- If you have multiple thin-provisioned devices that share the same pool, then these devices can be over-provisioned. Over-provisioning requires a float value of 1 or greater.
- 6
- Thin provisioning is required to create volume snapshots.
- 7
- String. The group where the
device-class
creates the logical volumes.
When multiple PVCs are created simultaneously, a race condition prevents LVMS from accurately tracking the allocated space and preserving the storage capacity for a device class. Use separate volume groups and device classes to protect the storage of highly dynamic workloads from each other.
7.1.1. Storage classes Copy linkLink copied to clipboard!
Storage classes provide the workload layer interface for selecting a device class. The following storage class parameters are supported in MicroShift:
-
The
csi.storage.k8s.io/fstype
parameter selects the file system types. Bothxfs
andext4
file system types are supported. -
The
topolvm.io/device-class
parameter is the name of the device class. If a device class is not provided, the default device class is assumed.
Multiple storage classes can refer to the same device class. You can provide varying sets of parameters for the same backing device class, such as xfs
and ext4
variants.
Example MicroShift default storage class resource
- 1
- An example of the default storage class. If a PVC does not specify a storage class, this class is assumed. There can only be one default storage class in a cluster. Having no value assigned to this annotation is also supported.
- 2
- Specifies which file system to provision on the volume. Options are "xfs" and "ext4".
- 3
- Identifies which provisioner should manage this class.
- 4
- Specifies whether to provision the volume before a client pod is present or immediately. Options are
WaitForFirstConsumer
andImmediate
.WaitForFirstConsumer
is recommended to ensure that storage is only provisioned for pods that can be scheduled. - 5
- Specifies if PVCs provisioned from the
StorageClass
permit expansion. The MicroShift LVMS CSI plugin does support volume expansion, but if this value is set tofalse
, expansion is blocked.
7.2. Volume snapshot classes Copy linkLink copied to clipboard!
Snapshotting is a CSI storage feature supported by LVMS. To enable dynamic snapshotting, at least one VolumeSnapshotClass
configuration file must be present on the cluster.
You must enable thin logical volumes to take logical volume snapshots.
Example VolumeSnapshotClass
configuration file
- 1
- Determines which
VolumeSnapshotClass
configuration file to use when none is specified byVolumeSnapshot
, which is a request for snapshot of a volume by a user. - 2
- Identifies which snapshot provisioner should manage the requests for snapshots of a volume by a user for this class.
- 3
- Determines whether
VolumeSnapshotContent
objects and the backing snapshots are kept or deleted when a boundVolumeSnapshot
is deleted. Valid values areRetain
orDelete
.
7.3. About volume snapshots Copy linkLink copied to clipboard!
You can use volume snapshots with logical volume manager (LVM) thin volumes to help protect against data loss from applications running in a MicroShift cluster. MicroShift only supports the logical volume manager storage (LVMS) Container Storage Interface (CSI) provider.
LVMS only supports the volumeBindingMode
of the storage class being set to WaitForFirstConsumer
. This setting means the storage volume is not provisioned until a pod is ready to mount it.
Example workload that deploys a single pod and PVC
7.3.1. Creating a volume snapshot Copy linkLink copied to clipboard!
To create a snapshot of a MicroShift storage volume, you must first configure RHEL for Edge and the cluster. In the following example procedure, the pod that the source volume is mounted to is deleted. Deleting the pod prevents data from being written to it during snapshot creation. Ensuring that no data is being written during a snapshot is crucial to creating a viable snapshot.
Prerequisites
- User has root access to a MicroShift cluster.
- A MicroShift cluster is running.
- A device class defines an LVM thin-pool.
-
A
volumeSnapshotClass
specifiesdriver: topolvm.io
. - Any workload attached to the source PVC is paused or deleted. This helps avoid data corruption.
All writes to the volume must be halted while you are creating the snapshot. If you do not halt writes, your data might be corrupted.
Procedure
Prevent data from being written to the volume during snapshotting by using one of the two following steps:
Delete the pod to ensure that no data is written to the volume during snapshotting by running the following command:
oc delete my-pod
$ oc delete my-pod
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Scale the replica count to zero on a pod that is managed with a replication controller. Setting the count to zero prevents the instant creation of a new pod when one is deleted.
After all writes to the volume are halted, run a command similar to the example that follows. Insert your own configuration details.
Example snapshot configuration
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Wait for the storage driver to finish creating the snapshot by running the following command:
oc wait volumesnapshot/<snapshot_name> --for=jsonpath\='{.status.readyToUse}=true'
$ oc wait volumesnapshot/<snapshot_name> --for=jsonpath\='{.status.readyToUse}=true'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Next steps
-
When the
volumeSnapshot
object is in aReadyToUse
state, you can restore it as a volume for future PVCs. Restart the pod or scale the replica count back up to the desired number. - After you have created the volume snapshot, you can remount the source PVC to a new pod.
Volume snapshots are located on the same devices as the original data. To use the volume snapshots as backups, move the snapshots to a secure location.
7.3.2. Backing up a volume snapshot Copy linkLink copied to clipboard!
Snapshots of data from applications running on a MicroShift cluster are created as read-only logical volumes (LVs) located on the same devices as the original data. You must manually mount local volumes before they can be copied as persistent volumes (PVs) and used as backup copies. To use a snapshot of a MicroShift storage volume as a backup, find it on the local host and then move it to a secure location.
To find specific snapshots and copy them, use the following procedure.
Prerequisites
- You have root access to the host machine.
- You have an existing volume snapshot.
Procedure
Get the name of the volume snapshot by running the following command:
oc get volumesnapshot -n <namespace> <snapshot_name> -o 'jsonpath={.status.volumeSnapshotContentName}'
$ oc get volumesnapshot -n <namespace> <snapshot_name> -o 'jsonpath={.status.volumeSnapshotContentName}'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Get the unique identity of the volume created on the storage backend by using the following command and inserting the name retrieved in the previous step:
oc get volumesnapshotcontent snapcontent-<retrieved_volume_identity> -o 'jsonpath={.status.snapshotHandle}'
$ oc get volumesnapshotcontent snapcontent-<retrieved_volume_identity> -o 'jsonpath={.status.snapshotHandle}'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Display the snapshots by using the unique identity of the volume you retrieved in the previous step to determine which one you want to backup by running the following command:
sudo lvdisplay <retrieved_snapshot_handle>
$ sudo lvdisplay <retrieved_snapshot_handle>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a directory to use for mounting the LV by running the following command:
sudo mkdir /mnt/snapshot
$ sudo mkdir /mnt/snapshot
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Mount the LV using the device name for the retrieved snapshot handle by running the following command:
sudo mount /dev/<retrieved_snapshot_handle> /mnt/snapshot
$ sudo mount /dev/<retrieved_snapshot_handle> /mnt/snapshot
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the files from the mounted location and store them in a secure location by running the following command:
sudo cp -r /mnt/snapshot <destination>
$ sudo cp -r /mnt/snapshot <destination>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.3.3. Restoring a volume snapshot Copy linkLink copied to clipboard!
The following workflow demonstrates snapshot restoration. In this example, the verification steps are also given to ensure that data written to a source persistent volume claim (PVC) is preserved and restored on a new PVC.
A snapshot must be restored to a PVC of exactly the same size as the source volume of the snapshot. You can resize the PVC after the snapshot is restored successfully if a larger PVC is needed.
Procedure
Restore a snapshot by specifying the
VolumeSnapshot
object as the data source in a persistent volume claim by entering the following command:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Wait for the pod to reach the
Ready
state:oc wait --for=condition=Ready pod/base
$ oc wait --for=condition=Ready pod/base
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - When the new pod is ready, verify that the data from your application is correct in the snapshot.
7.3.4. Deleting a volume snapshot Copy linkLink copied to clipboard!
You can configure how Red Hat build of MicroShift deletes volume snapshots.
Procedure
Specify the deletion policy that you require in the
VolumeSnapshotClass
object, as shown in the following example:volumesnapshotclass.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- When deleting the volume snapshot, if the
Delete
value is set, the underlying snapshot is deleted along with theVolumeSnapshotContent
object. If theRetain
value is set, both the underlying snapshot andVolumeSnapshotContent
object remain.
If theRetain
value is set and theVolumeSnapshot
object is deleted without deleting the correspondingVolumeSnapshotContent
object, the content remains. The snapshot itself is also retained in the storage back end.
Delete the volume snapshot by entering the following command:
oc delete volumesnapshot <volumesnapshot_name>
$ oc delete volumesnapshot <volumesnapshot_name>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
volumesnapshot.snapshot.storage.k8s.io "mysnapshot" deleted
volumesnapshot.snapshot.storage.k8s.io "mysnapshot" deleted
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the deletion policy is set to
Retain
, delete the volume snapshot content by entering the following command:oc delete volumesnapshotcontent <volumesnapshotcontent_name>
$ oc delete volumesnapshotcontent <volumesnapshotcontent_name>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Optional: If the
VolumeSnapshot
object is not successfully deleted, enter the following command to remove any finalizers for the leftover resource so that the delete operation can continue:ImportantOnly remove the finalizers if you are confident that there are no existing references from either persistent volume claims or volume snapshot contents to the
VolumeSnapshot
object. Even with the--force
option, the delete operation does not delete snapshot objects until all finalizers are removed.oc patch -n $PROJECT volumesnapshot/$NAME --type=merge -p '{"metadata": {"finalizers":null}}'
$ oc patch -n $PROJECT volumesnapshot/$NAME --type=merge -p '{"metadata": {"finalizers":null}}'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
volumesnapshotclass.snapshot.storage.k8s.io "csi-ocs-rbd-snapclass" deleted
volumesnapshotclass.snapshot.storage.k8s.io "csi-ocs-rbd-snapclass" deleted
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The finalizers are removed and the volume snapshot is deleted.
7.4. About LVM volume cloning Copy linkLink copied to clipboard!
The logical volume manager storage (LVMS) supports persistent volume claim (PVC) cloning for logical volume manager (LVM) thin volumes. A clone is a duplicate of an existing volume that can be used like any other volume. When provisioned, an exact duplicate of the original volume is created if the data source references a source PVC in the same namespace. After a cloned PVC is created, it is considered a new object and completely separate from the source PVC. The clone represents the data from the source at the moment in time it was created.
Cloning is only possible when the source and destination PVCs are in the same namespace. To create PVC clones, you must configure thin volumes on the RHEL for Edge host.