Chapter 4. Configuring cluster storage
4.1. About persistent storage Copy linkLink copied to clipboard!
OpenShift AI uses persistent storage to support workbenches, project data, and model training.
Persistent storage is provisioned through OpenShift storage classes and persistent volumes. Volume provisioning and data access are determined by access modes.
Understanding storage classes and access modes can help you choose the right storage for your use case and avoid potential risks when sharing data across multiple workbenches.
4.1.1. Storage classes in OpenShift AI Copy linkLink copied to clipboard!
Storage classes in OpenShift AI are available from the underlying OpenShift cluster. A storage class defines how persistent volumes are provisioned, including which storage backend is used and what access modes the provisioned volumes can support. For more information, see Dynamic provisioning in the OpenShift documentation.
Cluster administrators create and configure storage classes in the OpenShift cluster. These storage classes provision persistent volumes that support one or more access modes, depending on the capabilities of the storage backend. OpenShift AI administrators then enable specific storage classes and access modes for use in OpenShift AI.
When adding cluster storage to your project or workbench, you can choose from any enabled storage classes and access modes.
4.1.2. Access modes Copy linkLink copied to clipboard!
Storage classes create persistent volumes that can support different access modes, depending on the storage backend. Access modes control how a volume can be mounted and used by one or more workbenches. If a storage class allows more than one access mode, you can select the one that best fits your needs when you request storage. All persistent volumes support ReadWriteOnce (RWO) by default.
| Access mode | Description |
|---|---|
|
|
The storage can be attached to a single workbench or pod at a time and is ideal for most individual workloads. |
|
|
The storage can be attached to many workbenches simultaneously. |
|
|
The storage can be attached to many workbenches as read-only. |
|
|
The storage can be attached to a single pod on a single node with read-write permissions. |
You can enable access modes that are required. A warning is displayed if you request an access mode with unknown support, but you can continue to select Save to create the storage class with the selected access mode.
4.2. Adding cluster storage to your project Copy linkLink copied to clipboard!
For projects that require data to be retained, you can add cluster storage to the project. Additionally, you can also connect cluster storage to a specific project’s workbench.
Prerequisites
- You have logged in to Red Hat OpenShift AI.
- You have created a project that you can add cluster storage to.
Procedure
From the OpenShift AI dashboard, click Projects.
The Projects page opens.
Click the name of the project that you want to add the cluster storage to.
A project details page opens.
- Click the Cluster storage tab.
Click Add cluster storage.
The Add cluster storage dialog opens.
- In the Name field, enter a unique name for the cluster storage.
- Optional: In the Description field, enter a description for the cluster storage.
From the Storage class list, select the type of cluster storage.
NoteYou cannot change the storage class after you add the cluster storage to the project.
For storage classes that support multiple access modes, select an Access mode to define how the volume can be accessed. For more information, see About persistent storage.
Only the access modes that have been enabled for the storage class by your cluster and OpenShift AI administrators are visible.
- In the Persistent storage size section, specify a new size in gibibytes or mebibytes.
Optional: If you want to connect the cluster storage to an existing workbench:
- In the Workbench connections section, click Add workbench.
- In the Name field, select an existing workbench from the list.
-
In the Path format field, select Standard if your storage directory begins with
/opt/app-root/src, otherwise select Custom. -
In the Mount path field, enter the path to a model or directory within a container where a volume is mounted and accessible. The path must consist of lowercase alphanumeric characters or
-. Use/to indicate subdirectories.
- Click Add storage.
Verification
- The cluster storage that you added is displayed on the Cluster storage tab for the project.
- A new persistent volume claim (PVC) is created with the storage size that you defined.
- The persistent volume claim (PVC) is visible as an attached storage on the Workbenches tab for the project.
4.3. Updating cluster storage Copy linkLink copied to clipboard!
If your data science work requires you to change the identifying information of a project’s cluster storage or the workbench that the storage is connected to, you can update your project’s cluster storage to change these properties.
You cannot directly change the storage class for cluster storage that is already configured for a workbench or project. To switch to a different storage class, you need to migrate your data to a new cluster storage instance that uses the required storage class. For more information, see Changing the storage class for an existing cluster storage instance.
Prerequisites
- You have logged in to Red Hat OpenShift AI.
- You have created a project that contains cluster storage.
Procedure
From the OpenShift AI dashboard, click Projects.
The Projects page opens.
Click the name of the project whose storage you want to update.
A project details page opens.
- Click the Cluster storage tab.
Click the action menu (⋮) beside the storage that you want to update and then click Edit storage.
The Update cluster storage page opens.
- Optional: Edit the Name field to change the display name for your storage.
- Optional: Edit the Description field to change the description of your storage.
Optional: In the Persistent storage size section, specify a new size in gibibytes or mebibytes.
Note that you can only increase the storage size. Updating the storage size restarts the workbench and makes it unavailable for a period of time that is usually proportional to the size change.
Optional: If you want to connect the cluster storage to a different workbench:
- In the Workbench connections section, click Add workbench.
- In the Name field, select an existing workbench from the list.
-
In the Path format field, select Standard if your storage directory begins with
/opt/app-root/src, otherwise select Custom. -
In the Mount path field, enter the path to a model or directory within a container where a volume is mounted and accessible. The path must consist of lowercase alphanumeric characters or
-. Use/to indicate subdirectories.
- Click Update storage.
If you increased the storage size, the workbench restarts and is unavailable for a period of time that is usually proportional to the size change.
Verification
- The storage that you updated is displayed on the Cluster storage tab for the project.
4.4. Changing the storage class for an existing cluster storage instance Copy linkLink copied to clipboard!
When you create a workbench with cluster storage, the cluster storage is tied to a specific storage class. Later, if your data science work requires a different storage class, or if the current storage class has been deprecated, you cannot directly change the storage class on the existing cluster storage instance. Instead, you must migrate your data to a new cluster storage instance that uses the storage class that you want to use.
Prerequisites
- You have logged in to Red Hat OpenShift AI.
- You have created a workbench or project that contains cluster storage.
Procedure
Stop the workbench with the storage class that you want to change.
From the OpenShift AI dashboard, click Projects.
The Projects page opens.
Click the name of the project with the cluster storage instance that uses the storage class you want to change.
The project details page opens.
- Click the Workbenches tab.
In the Status column for the relevant workbench, click Stop.
Wait until the Status column for the relevant workbench changes from Running to Stopped.
Add a new cluster storage instance that uses the needed storage class.
- Click the Cluster storage tab.
Click Add cluster storage.
The Add cluster storage dialog opens.
- Enter a name for the cluster storage.
- Optional: Enter a description for the cluster storage.
- Select the needed storage class for the cluster storage.
For storage classes that support multiple access modes, select an Access mode to define how the volume can be accessed. For more information, see About persistent storage.
Only the access modes that have been enabled for the storage class by your cluster and OpenShift AI administrators are visible.
- Under Persistent storage size, enter a size in gibibytes or mebibytes.
- In the Workbench connections section, click Add workbench.
- In the Name field, select an existing workbench from the list.
-
In the Path format field, select Standard if your storage directory begins with
/opt/app-root/src, otherwise select Custom. -
In the Mount path field, enter the path to a model or directory within a container where a volume is mounted and accessible. For example,
backup. - Click Add storage.
Copy the data from the existing cluster storage instance to the new cluster storage instance.
- Click the Workbenches tab.
- In the Status column for the relevant workbench, click Start.
- When the workbench status is Running, click Open to open the workbench.
-
In JupyterLab, click File
New Terminal. Copy the data to the new storage directory. Replace <mount_folder_name> with the storage directory of your new cluster storage instance.
rsync -avO --exclude='/opt/app-root/src/__<mount_folder_name>__' /opt/app-root/src/ /opt/app-root/src/__<mount_folder_name>__/
rsync -avO --exclude='/opt/app-root/src/__<mount_folder_name>__' /opt/app-root/src/ /opt/app-root/src/__<mount_folder_name>__/Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example:
rsync -avO --exclude='/opt/app-root/src/backup' /opt/app-root/src/ /opt/app-root/src/backup/
rsync -avO --exclude='/opt/app-root/src/backup' /opt/app-root/src/ /opt/app-root/src/backup/Copy to Clipboard Copied! Toggle word wrap Toggle overflow - After the data has finished copying, log out of JupyterLab.
Stop the workbench.
- Click the Workbenches tab.
In the Status column for the relevant workbench, click Stop.
Wait until the Status column for the relevant workbench changes from Running to Stopped.
Remove the original cluster storage instance from the workbench.
- Click the Cluster storage tab.
- Click the action menu (⋮) beside the existing cluster storage instance, and then click Edit storage.
- Under Existing connected workbenches, remove the workbench.
- Click Update.
Update the mount folder of the new cluster storage instance by removing it and re-adding it to the workbench.
- On the Cluster storage tab, click the action menu (⋮) beside the new cluster storage instance, and then click Edit storage.
- Under Existing connected workbenches, remove the workbench.
- Click Update.
- Click the Workbenches tab.
- Click the action menu (⋮) beside the workbench and then click Edit workbench.
- In the Cluster storage section, under Use existing persistent storage, select the new cluster storage instance.
- Click Update workbench.
Restart the workbench.
- Click the Workbenches tab.
- In the Status column for the relevant workbench, click Start.
- Optional: The initial cluster storage that uses the previous storage class is still visible on the Cluster storage tab. If you no longer need this cluster storage (for example, if the storage class is deprecated), you can delete it.
-
Optional: You can delete the mount folder of your new cluster storage instance (for example, the
backupfolder).
Verification
- On the Cluster storage tab for the project, the new cluster storage instance is displayed with the needed storage class in the Storage class column and the relevant workbench in the Connected workbenches column.
-
On the Workbenches tab for the project, the new cluster storage instance is displayed for the workbench in the Cluster storage section and has the mount path:
/opt/app-root/src.
4.5. Deleting cluster storage from a project Copy linkLink copied to clipboard!
You can delete cluster storage from your projects to help you free up resources and delete unwanted storage space.
Prerequisites
- You have logged in to Red Hat OpenShift AI.
- You have created a project with cluster storage.
Procedure
From the OpenShift AI dashboard, click Projects.
The Projects page opens.
Click the name of the project that you want to delete the storage from.
A project details page opens.
- Click the Cluster storage tab.
Click the action menu (⋮) beside the storage that you want to delete and then click Delete storage.
The Delete storage dialog opens.
- Enter the name of the storage in the text field to confirm that you intend to delete it.
- Click Delete storage.
Verification
- The storage that you deleted is no longer displayed on the Cluster storage tab for the project.
- The persistent volume (PV) and persistent volume claim (PVC) associated with the cluster storage are both permanently deleted. This data is not recoverable.