Chapter 7. Working with volume snapshots
Cluster administrators can use volume snapshots to help protect against data loss by using the supported MicroShift logical volume manager storage (LVMS) Container Storage Interface (CSI) provider. Familiarity with persistent volumes is required.
A snapshot represents the state of the storage volume in a cluster at a particular point in time. Volume snapshots can also be used to provision new volumes. Snapshots are created as read-only logical volumes (LVs) located on the same device as the original data.
A cluster administrator can complete the following tasks using CSI volume snapshots:
- Create a snapshot of an existing persistent volume claim (PVC).
- Back up a volume snapshot to a secure location.
- Restore a volume snapshot as a different PVC.
- Delete an existing volume snapshot.
Only the logical volume manager storage (LVMS) plugin CSI driver is supported by MicroShift.
Additional resources
7.1. About LVM thin volumes
To use advanced storage features such as creating volume snapshots or volume cloning, you must perform the following actions:
- Configure both the logical volume manager storage (LVMS) provider and the cluster.
- Provision a logical volume manager (LVM) thin-pool on the RHEL for Edge host.
- Attach LVM thin-pools to a volume group.
To create Container Storage Interface (CSI) snapshots, you must configure thin volumes on the RHEL for Edge host. The CSI does not support volume shrinking.
When using thin provisioning, it is important that you monitor the storage pool and add more capacity as the available physical space runs out. You can configure the storage pool to auto expand when there is available space within the volume group (VG). See Creating thinly provisioned logical volumes.
For LVMS to manage thin logical volumes (LVs), a thin-pool device-class
array must be specified in the etc/lvmd.yaml
configuration file. Multiple thin-pool device classes are permitted.
If additional storage pools are configured with device classes, then additional storage classes must also exist to expose the storage pools to users and workloads. To enable dynamic provisioning on a thin-pool, a StorageClass
resource must be present on the cluster. The StorageClass
resource specifies the source device-class
array in the topolvm.io/device-class
parameter.
Example lvmd.yaml
file that specifies a single device class for a thin-pool
socket-name: 1 device-classes: 2 - name: thin 3 default: true spare-gb: 0 4 thin-pool: name: thin overprovision-ratio: 1 5 type: thin 6 volume-group: ssd 7
- 1
- String. The UNIX domain socket endpoint of gRPC. Defaults to
/run/lvmd/lvmd.socket
. - 2
- A list of maps for the settings for each
device-class
. - 3
- String. The unique name of the
device-class
. - 4
- Unsigned 64-bit integer. Storage capacity in GB to be left unallocated in the volume group. Defaults to
0
. - 5
- Specify a float factor by which you can provision additional storage based on the available storage in the thin pool. For example, if this field is set to 10, you can provision up to 10 times the amount of available storage in the thin pool. To disable over-provisioning, set this field to
1
. - 6
- Thin provisioning is required to create volume snapshots.
- 7
- String. The group where the
device-class
creates the logical volumes.
When multiple PVCs are created simultaneously, a race condition prevents LVMS from accurately tracking the allocated space and preserving the storage capacity for a device class. Use separate volume groups and device classes to protect the storage of highly dynamic workloads from each other.
Additional resources
7.1.1. Storage classes
Storage classes provide the workload layer interface for selecting a device class. The following storage class parameters are supported in MicroShift:
-
The
csi.storage.k8s.io/fstype
parameter selects the file system types. Bothxfs
andext4
file system types are supported. -
The
topolvm.io/device-class
parameter is the name of the device class. If a device class is not provided, the default device class is assumed.
Multiple storage classes can refer to the same device class. You can provide varying sets of parameters for the same backing device class, such as xfs
and ext4
variants.
Example MicroShift default storage class resource
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: annotations: storageclass.kubernetes.io/is-default-class: "true" 1 name: topolvm-provisioner parameters: "csi.storage.k8s.io/fstype": "xfs" 2 provisioner: topolvm.io 3 reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer 4 allowVolumeExpansion: 5
- 1
- An example of the default storage class. If a PVC does not specify a storage class, this class is assumed. There can only be one default storage class in a cluster. Having no value assigned to this annotation is also supported.
- 2
- Specifies which file system to provision on the volume. Options are "xfs" and "ext4".
- 3
- Identifies which provisioner should manage this class.
- 4
- Specifies whether to provision the volume before a client pod is present or immediately. Options are
WaitForFirstConsumer
andImmediate
.WaitForFirstConsumer
is recommended to ensure that storage is only provisioned for pods that can be scheduled. - 5
- Specifies if PVCs provisioned from the
StorageClass
permit expansion. The MicroShift LVMS CSI plugin does support volume expansion, but if this value is set tofalse
, expansion is blocked.
Additional resources
7.2. Volume snapshot classes
Snapshotting is a CSI storage feature supported by LVMS. To enable dynamic snapshotting, at least one VolumeSnapshotClass
configuration file must be present on the cluster.
You must enable thin logical volumes to take logical volume snapshots.
Example VolumeSnapshotClass
configuration file
apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: topolvm-snapclass annotations: snapshot.storage.kubernetes.io/is-default-class: "true" 1 driver: topolvm.io 2 deletionPolicy: Delete 3
- 1
- Determines which
VolumeSnapshotClass
configuration file to use when none is specified byVolumeSnapshot
, which is a request for snapshot of a volume by a user. - 2
- Identifies which snapshot provisioner should manage the requests for snapshots of a volume by a user for this class.
- 3
- Determines whether
VolumeSnapshotContent
objects and the backing snapshots are kept or deleted when a boundVolumeSnapshot
is deleted. Valid values areRetain
orDelete
.
Additional resources
7.3. About volume snapshots
You can use volume snapshots with logical volume manager (LVM) thin volumes to help protect against data loss from applications running in a MicroShift cluster. MicroShift only supports the logical volume manager storage (LVMS) Container Storage Interface (CSI) provider.
LVMS only supports the volumeBindingMode
of the storage class being set to WaitForFirstConsumer
. This setting means the storage volume is not provisioned until a pod is ready to mount it.
Example workload that deploys a single pod and PVC
$ oc apply -f - <<EOF apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-claim-thin spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: topolvm-provisioner-thin --- apiVersion: v1 kind: Pod metadata: name: base spec: containers: - command: - nginx - -g - 'daemon off;' image: registry.redhat.io/rhel8/nginx-122@sha256:908ebb0dec0d669caaf4145a8a21e04fdf9ebffbba5fd4562ce5ab388bf41ab2 name: test-container securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL volumeMounts: - mountPath: /vol name: test-vol securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault volumes: - name: test-vol persistentVolumeClaim: claimName: test-claim-thin EOF
7.3.1. Creating a volume snapshot
To create a snapshot of a MicroShift storage volume, you must first configure RHEL for Edge and the cluster. In the following example procedure, the pod that the source volume is mounted to is deleted. Deleting the pod prevents data from being written to it during snapshot creation. Ensuring that no data is being written during a snapshot is crucial to creating a viable snapshot.
Prerequisites
- User has root access to a MicroShift cluster.
- A MicroShift cluster is running.
- A device class defines an LVM thin-pool.
-
A
volumeSnapshotClass
specifiesdriver: topolvm.io
. - Any workload attached to the source PVC is paused or deleted. This helps avoid data corruption.
All writes to the volume must be halted while you are creating the snapshot. If you do not halt writes, your data might be corrupted.
Procedure
Prevent data from being written to the volume during snapshotting by using one of the two following steps:
Delete the pod to ensure that no data is written to the volume during snapshotting by running the following command:
$ oc delete my-pod
- Scale the replica count to zero on a pod that is managed with a replication controller. Setting the count to zero prevents the instant creation of a new pod when one is deleted.
After all writes to the volume are halted, run a command similar to the example that follows. Insert your own configuration details.
Example snapshot configuration
# oc apply -f <<EOF apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot 1 metadata: name: <snapshot_name> 2 spec: volumeSnapshotClassName: topolvm-snapclass 3 source: persistentVolumeClaimName: test-claim-thin 4 EOF
Wait for the storage driver to finish creating the snapshot by running the following command:
$ oc wait volumesnapshot/<snapshot_name> --for=jsonpath\='{.status.readyToUse}=true'
Next steps
-
When the
volumeSnapshot
object is in aReadyToUse
state, you can restore it as a volume for future PVCs. Restart the pod or scale the replica count back up to the desired number. - After you have created the volume snapshot, you can remount the source PVC to a new pod.
Volume snapshots are located on the same devices as the original data. To use the volume snapshots as backups, move the snapshots to a secure location.
7.3.2. Backing up a volume snapshot
Snapshots of data from applications running on a MicroShift cluster are created as read-only logical volumes (LVs) located on the same devices as the original data. You must manually mount local volumes before they can be copied as persistent volumes (PVs) and used as backup copies. To use a snapshot of a MicroShift storage volume as a backup, find it on the local host and then move it to a secure location.
To find specific snapshots and copy them, use the following procedure.
Prerequisites
- You have root access to the host machine.
- You have an existing volume snapshot.
Procedure
Get the name of the volume snapshot by running the following command:
$ oc get volumesnapshot -n <namespace> <snapshot_name> -o 'jsonpath={.status.volumeSnapshotContentName}'
Get the unique identity of the volume created on the storage backend by using the following command and inserting the name retrieved in the previous step:
$ oc get volumesnapshotcontent snapcontent-<retrieved_volume_identity> -o 'jsonpath={.status.snapshotHandle}'
Display the snapshots by using the unique identity of the volume you retrieved in the previous step to determine which one you want to backup by running the following command:
$ sudo lvdisplay <retrieved_snapshot_handle>
Example output
--- Logical volume --- LV Path /dev/rhel/732e45ff-f220-49ce-859e-87ccca26b14c LV Name 732e45ff-f220-49ce-859e-87ccca26b14c VG Name rhel LV UUID 6Ojwc0-YTfp-nKJ3-F9FO-PvMR-Ic7b-LzNGSx LV Write Access read only LV Creation host, time rhel-92.lab.local, 2023-08-07 14:45:26 -0500 LV Pool name thinpool LV Thin origin name a2d2dcdc-747e-4572-8c83-56cd873d3b07 LV Status available # open 0 LV Size 1.00 GiB Mapped size 1.04% Current LE 256 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:11
Create a directory to use for mounting the LV by running the following command:
$ sudo mkdir /mnt/snapshot
Mount the LV using the device name for the retrieved snapshot handle by running the following command:
$ sudo mount /dev/<retrieved_snapshot_handle> /mnt/snapshot
Copy the files from the mounted location and store them in a secure location by running the following command:
$ sudo cp -r /mnt/snapshot <destination>
7.3.3. Restoring a volume snapshot
The following workflow demonstrates snapshot restoration. In this example, the verification steps are also given to ensure that data written to a source persistent volume claim (PVC) is preserved and restored on a new PVC.
A snapshot must be restored to a PVC of exactly the same size as the source volume of the snapshot. You can resize the PVC after the snapshot is restored successfully if a larger PVC is needed.
Procedure
Restore a snapshot by specifying the
VolumeSnapshot
object as the data source in a persistent volume claim by entering the following command:$ oc apply -f <<EOF apiVersion: v1 kind: PersistentVolumeClaim metadata: name: snapshot-restore spec: accessModes: - ReadWriteOnce dataSource: apiGroup: snapshot.storage.k8s.io kind: VolumeSnapshot name: my-snap resources: requests: storage: 1Gi storageClassName: topolvm-provisioner-thin --- apiVersion: v1 kind: Pod metadata: name: base spec: containers: - command: - nginx - -g - 'daemon off;' image: registry.redhat.io/rhel8/nginx-122@sha256:908ebb0dec0d669caaf4145a8a21e04fdf9ebffbba5fd4562ce5ab388bf41ab2 name: test-container securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL volumeMounts: - mountPath: /vol name: test-vol securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault volumes: - name: test-vol persistentVolumeClaim: claimName: snapshot-restore EOF
Verification
Wait for the pod to reach the
Ready
state:$ oc wait --for=condition=Ready pod/base
- When the new pod is ready, verify that the data from your application is correct in the snapshot.
Additional resources
7.3.4. Deleting a volume snapshot
You can configure how MicroShift deletes volume snapshots.
Procedure
Specify the deletion policy that you require in the
VolumeSnapshotClass
object, as shown in the following example:volumesnapshotclass.yaml
apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: csi-hostpath-snap driver: hostpath.csi.k8s.io deletionPolicy: Delete 1
- 1
- When deleting the volume snapshot, if the
Delete
value is set, the underlying snapshot is deleted along with theVolumeSnapshotContent
object. If theRetain
value is set, both the underlying snapshot andVolumeSnapshotContent
object remain.
If theRetain
value is set and theVolumeSnapshot
object is deleted without deleting the correspondingVolumeSnapshotContent
object, the content remains. The snapshot itself is also retained in the storage back end.
Delete the volume snapshot by entering the following command:
$ oc delete volumesnapshot <volumesnapshot_name>
Example output
volumesnapshot.snapshot.storage.k8s.io "mysnapshot" deleted
If the deletion policy is set to
Retain
, delete the volume snapshot content by entering the following command:$ oc delete volumesnapshotcontent <volumesnapshotcontent_name>
Optional: If the
VolumeSnapshot
object is not successfully deleted, enter the following command to remove any finalizers for the leftover resource so that the delete operation can continue:ImportantOnly remove the finalizers if you are confident that there are no existing references from either persistent volume claims or volume snapshot contents to the
VolumeSnapshot
object. Even with the--force
option, the delete operation does not delete snapshot objects until all finalizers are removed.$ oc patch -n $PROJECT volumesnapshot/$NAME --type=merge -p '{"metadata": {"finalizers":null}}'
Example output
volumesnapshotclass.snapshot.storage.k8s.io "csi-ocs-rbd-snapclass" deleted
The finalizers are removed and the volume snapshot is deleted.
7.4. About LVM volume cloning
The logical volume manager storage (LVMS) supports persistent volume claim (PVC) cloning for logical volume manager (LVM) thin volumes. A clone is a duplicate of an existing volume that can be used like any other volume. When provisioned, an exact duplicate of the original volume is created if the data source references a source PVC in the same namespace. After a cloned PVC is created, it is considered a new object and completely separate from the source PVC. The clone represents the data from the source at the moment in time it was created.
Cloning is only possible when the source and destination PVCs are in the same namespace. To create PVC clones, you must configure thin volumes on the RHEL for Edge host.
Additional resources
- CSI volume cloning
- LVMS volume cloning for Single-Node OpenShift
- To configure the host to enable cloning, see About LVM thin volumes