This documentation is for a release that is no longer maintained
See documentation for the latest supported version 3 or the latest supported version 4.4.3. CSI volume snapshots
This document describes how to use volume snapshots with supported Container Storage Interface (CSI) drivers to help protect against data loss in OpenShift Container Platform. Familiarity with persistent volumes is suggested.
CSI volume snapshot is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview/.
4.3.1. Overview of CSI volume snapshots 复制链接链接已复制到粘贴板!
A snapshot represents the state of the storage volume in a cluster at a particular point in time. Volume snapshots can be used to provision a new volume.
OpenShift Container Platform supports CSI volume snapshots by default. However, a specific CSI driver is required.
With CSI volume snapshots, a cluster administrator can:
- Deploy a third-party CSI driver that supports snapshots.
- Create a new persistent volume claim (PVC) from an existing volume snapshot.
- Take a snapshot of an existing PVC.
- Restore a snapshot as a different PVC.
- Delete an existing volume snapshot.
With CSI volume snapshots, an app developer can:
- Use volume snapshots as building blocks for developing application- or cluster-level storage backup solutions.
- Rapidly rollback to a previous development version.
- Use storage more efficiently by not having to make a full copy each time.
Be aware of the following when using volume snapshots:
- Support is only available for CSI drivers. In-tree and FlexVolumes are not supported.
- OpenShift Container Platform only ships with select CSI drivers. For CSI drivers that are not provided by an OpenShift Container Platform Driver Operator, it is recommended to use the CSI drivers provided by community or storage vendors. Follow the installation instructions provided by the CSI driver.
-
CSI drivers may or may not have implemented the volume snapshot functionality. CSI drivers that have provided support for volume snapshots will likely use the
csi-external-snapshottersidecar. See documentation provided by the CSI driver for details. - OpenShift Container Platform 4.5 supports version 1.1.0 of the CSI specification.
4.3.2. CSI snapshot controller and sidecar 复制链接链接已复制到粘贴板!
OpenShift Container Platform provides a snapshot controller that is deployed into the control plane. In addition, your CSI driver vendor provides the CSI snapshot sidecar as a helper container that is installed during the CSI driver installation.
The CSI snapshot controller and sidecar provide volume snapshotting through the OpenShift Container Platform API. These external components run in the cluster.
The external controller is deployed by the CSI Snapshot Controller Operator.
4.3.2.1. External controller 复制链接链接已复制到粘贴板!
The CSI snapshot controller binds VolumeSnapshot and VolumeSnapshotContent objects. The controller manages dynamic provisioning by creating and deleting VolumeSnapshotContent objects.
4.3.2.2. External sidecar 复制链接链接已复制到粘贴板!
Your CSI driver vendor provides the csi-external-snapshotter sidecar. This is a separate helper container that is deployed with the CSI driver. The sidecar manages snapshots by triggering CreateSnapshot and DeleteSnapshot operations. Follow the installation instructions provided by your vendor.
4.3.3. About the CSI Snapshot Controller Operator 复制链接链接已复制到粘贴板!
The CSI Snapshot Controller Operator runs in the openshift-cluster-storage-operator namespace. It is installed by the Cluster Version Operator (CVO) in all clusters by default.
The CSI Snapshot Controller Operator installs the CSI snapshot controller, which runs in the openshift-cluster-storage-operator namespace.
4.3.3.1. Volume snapshot CRDs 复制链接链接已复制到粘贴板!
During OpenShift Container Platform installation, the CSI Snapshot Controller Operator creates the following snapshot custom resource definitions (CRDs) in the snapshot.storage.k8s.io/v1beta1 API group:
VolumeSnapshotContentA snapshot taken of a volume in the cluster that has been provisioned by a cluster administrator.
Similar to the
PersistentVolumeobject, theVolumeSnapshotContentCRD is a cluster resource that points to a real snapshot in the storage back end.For manually pre-provisioned snapshots, a cluster administrator creates a number of
VolumeSnapshotContentCRDs. These carry the details of the real volume snapshot in the storage system.The
VolumeSnapshotContentCRD is not namespaced and is for use by a cluster administrator.VolumeSnapshotSimilar to the
PersistentVolumeClaimobject, theVolumeSnapshotCRD defines a developer request for a snapshot. The CSI Snapshot Controller Operator runs the CSI snapshot controller, which handles the binding of aVolumeSnapshotCRD with an appropriateVolumeSnapshotContentCRD. The binding is a one-to-one mapping.The
VolumeSnapshotCRD is namespaced. A developer uses the CRD as a distinct request for a snapshot.VolumeSnapshotClassAllows a cluster administrator to specify different attributes belonging to a
VolumeSnapshotCRD. These attributes may differ among snapshots taken of the same volume on the storage system, in which case they would not be expressed by using the same storage class of a persistent volume claim.The
VolumeSnapshotClassCRD defines the parameters for thecsi-external-snapshottersidecar to use when creating a snapshot. This allows the storage back end to know what kind of snapshot to dynamically create if multiple options are supported.Dynamically provisioned snapshots use the
VolumeSnapshotClassCRD to specify storage-provider-specific parameters to use when creating a snapshot.The
VolumeSnapshotContentClassCRD is not namespaced and is for use by a cluster administrator to enable global configuration options for their storage back end.
4.3.4. Volume snapshot provisioning 复制链接链接已复制到粘贴板!
There are two ways to provision snapshots: dynamically and manually.
4.3.4.1. Dynamic provisioning 复制链接链接已复制到粘贴板!
Instead of using a preexisting snapshot, you can request that a snapshot be taken dynamically from a persistent volume claim. Parameters are specified using a VolumeSnapshotClass CRD.
4.3.4.2. Manual provisioning 复制链接链接已复制到粘贴板!
As a cluster administrator, you can manually pre-provision a number of VolumeSnapshotContent objects. These carry the real volume snapshot details available to cluster users.
4.3.5. Creating a volume snapshot 复制链接链接已复制到粘贴板!
When you create a VolumeSnapshot object, OpenShift Container Platform creates a volume snapshot.
Prerequisites
- Logged in to a running OpenShift Container Platform cluster.
-
A PVC created using a CSI driver that supports
VolumeSnapshotobjects. - A storage class to provision the storage back end.
No pods are using the persistent volume claim (PVC) that you want to take a snapshot of.
注意Do not create a volume snapshot of a PVC if a pod is using it. Doing so might cause data corruption because the PVC is not quiesced (paused). Be sure to first tear down a running pod to ensure consistent snapshots.
Procedure
To dynamically create a volume snapshot:
Create a file with the
VolumeSnapshotClassobject described by the following YAML:volumesnapshotclass.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Allows you to specify different attributes belonging to a volume snapshot.
Create the object you saved in the previous step by entering the following command:
oc create -f volumesnapshotclass.yaml
$ oc create -f volumesnapshotclass.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
VolumeSnapshotobject:volumesnapshot-dynamic.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- The request for a particular class by the volume snapshot. If
volumeSnapshotClassNameis empty, then no snapshot is created. - 2
- The name of the
PersistentVolumeClaimobject bound to a persistent volume. This defines what you want to create a snapshot of. Required for dynamically provisioning a snapshot.
Create the object you saved in the previous step by entering the following command:
oc create -f volumesnapshot-dynamic.yaml
$ oc create -f volumesnapshot-dynamic.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
To manually provision a snapshot:
Provide a value for the
volumeSnapshotContentNameparameter as the source for the snapshot, in addition to defining volume snapshot class as shown above.volumesnapshot-manual.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- The
volumeSnapshotContentNameparameter is required for pre-provisioned snapshots.
Create the object you saved in the previous step by entering the following command:
oc create -f volumesnapshot-manual.yaml
$ oc create -f volumesnapshot-manual.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
After the snapshot has been created in the cluster, additional details about the snapshot are available.
To display details about the volume snapshot that was created, enter the following command:
oc describe volumesnapshot mysnap
$ oc describe volumesnapshot mysnapCopy to Clipboard Copied! Toggle word wrap Toggle overflow The following example displays details about the
mysnapvolume snapshot:volumesnapshot.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- The pointer to the actual storage content that was created by the controller.
- 2
- The time when the snapshot was created. The snapshot contains the volume content that was available at this indicated time.
- 3
- If the value is set to
true, the snapshot can be used to restore as a new PVC.
If the value is set tofalse, the snapshot was created. However, the storage back end needs to perform additional tasks to make the snapshot usable so that it can be restored as a new volume. For example, Amazon Elastic Block Store data might be moved to a different, less expensive location, which can take several minutes.
To verify that the volume snapshot was created, enter the following command:
oc get volumesnapshotcontent
$ oc get volumesnapshotcontentCopy to Clipboard Copied! Toggle word wrap Toggle overflow The pointer to the actual content is displayed. If the
boundVolumeSnapshotContentNamefield is populated, aVolumeSnapshotContentobject exists and the snapshot was created.-
To verify that the snapshot is ready, confirm that the
VolumeSnapshotobject hasreadyToUse: true.
4.3.6. Deleting a volume snapshot 复制链接链接已复制到粘贴板!
You can configure how OpenShift Container Platform deletes volume snapshots.
Procedure
Specify the deletion policy that you require in the
VolumeSnapshotClassobject, as shown in the following example:volumesnapshot.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- When deleting the volume snapshot, if the
Deletevalue is set, the underlying snapshot is deleted along with theVolumeSnapshotContentobject. If theRetainvalue is set, both the underlying snapshot andVolumeSnapshotContentobject remain.
If theRetainvalue is set and theVolumeSnapshotobject is deleted without deleting the correspondingVolumeSnapshotContentobject, the content remains. The snapshot itself is also retained in the storage back end.
Delete the volume snapshot by entering the following command:
oc delete volumesnapshot <volumesnapshot_name>
$ oc delete volumesnapshot <volumesnapshot_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
volumesnapshot.snapshot.storage.k8s.io "mysnapshot" deleted
volumesnapshot.snapshot.storage.k8s.io "mysnapshot" deletedCopy to Clipboard Copied! Toggle word wrap Toggle overflow If the deletion policy is set to
Retain, delete the volume snapshot content by entering the following command:oc delete volumesnapshotcontent <volumesnapshotcontent_name>
$ oc delete volumesnapshotcontent <volumesnapshotcontent_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Optional: If the
VolumeSnapshotobject is not successfully deleted, enter the following command to remove any finalizers for the leftover resource so that the delete operation can continue:重要Only remove the finalizers if you are confident that there are no existing references from either persistent volume claims or volume snapshot contents to the
VolumeSnapshotobject. Even with the--forceoption, the delete operation does not delete snapshot objects until all finalizers are removed.oc patch -n $PROJECT volumesnapshot/$NAME --type=merge -p '{"metadata": {"finalizers":null}}'$ oc patch -n $PROJECT volumesnapshot/$NAME --type=merge -p '{"metadata": {"finalizers":null}}'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
volumesnapshotclass.snapshot.storage.k8s.io "csi-ocs-rbd-snapclass" deleted
volumesnapshotclass.snapshot.storage.k8s.io "csi-ocs-rbd-snapclass" deletedCopy to Clipboard Copied! Toggle word wrap Toggle overflow The finalizers are removed and the volume snapshot is deleted.
4.3.7. Restoring a volume snapshot 复制链接链接已复制到粘贴板!
The VolumeSnapshot CRD content can be used to restore the existing volume to a previous state.
After your VolumeSnapshot CRD is bound and the readyToUse value is set to true, you can use that resource to provision a new volume that is pre-populated with data from the snapshot.
Prerequisites
- Logged in to a running OpenShift Container Platform cluster.
- A persistent volume claim (PVC) created using a Container Storage Interface (CSI) driver that supports volume snapshots.
- A storage class to provision the storage back end.
Procedure
Specify a
VolumeSnapshotdata source on a PVC as shown in the following:pvc-restore.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a PVC by entering the following command:
oc create -f pvc-restore.yaml
$ oc create -f pvc-restore.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the restored PVC has been created by entering the following command:
oc get pvc
$ oc get pvcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Two different PVCs are displayed.