Search
SupportConsoleDevelopersStart a trialContact

Updating OpenShift Container Storage

download PDF
Red Hat OpenShift Container Storage 4.6

Instructions for cluster and storage administrators

Red Hat Storage Documentation Team

Abstract

This document explains how to update Red Hat OpenShift Container Storage.

Chapter 1. Overview of the OpenShift Container Storage update process

You can upgrade Red Hat OpenShift Container Storage and its components, either between minor releases like 4.5 and 4.6, or between batch updates like 4.6.0 and 4.6.1.

You need to upgrade the different parts of OpenShift Container Storage in a specific order.

  1. Update OpenShift Container Platform according to the Updating clusters documentation for OpenShift Container Platform.

  2. Update OpenShift Container Storage.

    1. Update the OpenShift Container Storage operator, using the appropriate process for your setup:

    2. If you use local storage:

      1. Update the Local Storage operator.

        See Checking for Local Storage Operator deployments if you are unsure.

      2. Perform post-update configuration changes for clusters backed by local storage.

        See Post-update configuration for clusters backed by local storage for details.

Update considerations

Review the following important considerations before you begin.

  • Red Hat recommends using the same version of Red Hat OpenShift Container Platform with Red Hat OpenShift Container Storage.

    See the Interoperability Matrix for more information about supported combinations of OpenShift Container Platform and OpenShift Container Storage.

  • The Local Storage Operator is fully supported only when the Local Storage Operator version matches the Red Hat OpenShift Container Platform version.

Chapter 2. OpenShift Container Storage upgrade channels and releases

In OpenShift Container Platform 4.1, Red Hat introduced the concept of channels for recommending the appropriate release versions for cluster upgrades. By controlling the pace of upgrades, these upgrade channels allow you to choose an upgrade strategy. As OpenShift Container Storage gets deployed as an Operator in OpenShift Container Platform, it follows the same strategy to control the pace of upgrades by shipping the fixes in multiple channels. Upgrade channels are tied to a minor version of OpenShift Container Storage. For example, OpenShift Container Storage 4.6 upgrade channels recommend upgrades from 4.5 to 4.6 and upgrades within 4.6. They do not recommend upgrades to 4.7 or later releases. This strategy ensures that administrators can explicitly decide to upgrade to the next minor version of OpenShift Container Storage.

Upgrade channels control only release selection and do not impact the version of the cluster that you install; the ocs-operator decides the version of OpenShift Container Storage to be installed. Out of the box, it always installs the latest OpenShift Container Storage release maintaining the compatibility with OpenShift Container Platform. So on OpenShift Container Platform 4.6, OpenShift Container Storage 4.6 will be the latest version which can be installed.

OpenShift Container Storage upgrades are tied to the OpenShift Container Platform upgrade to ensure that compatibility and interoperability are maintained with the OpenShift Container Platform. For OpenShift Container Storage 4.6, OpenShift Container Platform 4.5, 4.6 and 4.7 are supported. OpenShift Container Platform 4.5 and 4.7 is supported to maintain backward and forward compatibility with OpenShift Container Platform. Keep the OpenShift Container Storage version the same as OpenShift Container Platform in order to get the benefit of all the features and enhancements in that release.

OpenShift Container Platform 4.6 offers the following upgrade channels:

  • stable-4.6

  • eus-4.6 (only available when running 4.6)

    stable-4.6 channel
    Once a new version is Generally Available, the stable channel corresponding to the minor version gets updated with the new image which can be used to upgrade. You can use the stable-4.6 channel to upgrade from OpenShift Container storage 4.5 and upgrades within 4.6.
    eus-4.6 channel

    In addition to the stable channel, certain minor versions of OpenShift Container Platform offer Extended Update Support (EUS). These EUS versions extend the maintenance phase for customers with Premium Subscriptions to 14 months. OpenShift Container Storage 4.6 is currently the only minor version with EUS.

    Although there is no difference between stable-4.6 and eus-4.6 channels until OpenShift Container Platform 4.6 transitions to the EUS phase, you can switch to the EUS channel as soon as it becomes available. When OpenShift Container Platform 4.6 transitions to the EUS phase of its lifecycle, the stable-4.6 channel will no longer receive subsequent z-stream updates. After you upgrade to a version that is exclusive to the EUS channel, that cluster will no longer be eligible for minor version upgrades until upgrades to the next EUS version become available. The next planned EUS version is to be 4.10 and the upgrade to that version will require a serial set of version upgrades, such as from 4.6 to 4.7 to 4.8 to 4.9 to 4.10.

    Additionally, you may only switch to the EUS channel when your cluster is running a supported version of OpenShift Container Platform 4.6.

    Finally, if you install a 4.6 version that is exclusive to EUS, you will similarly not be able to upgrade to a later minor version until upgrades are provided to 4.10.

Chapter 3. Updating OpenShift Container Storage in internal mode

Use the following procedures to update your OpenShift Container Storage cluster deployed in internal mode.

3.1. Enabling automatic updates for OpenShift Container Storage operator in internal mode

Use this procedure to enable automatic update approval for updating OpenShift Container Storage operator in OpenShift Container Platform.

Prerequisites

  • Under Persistent Storage in the Status card, confirm that the OCS Cluster and Data Resiliency has a green tick mark.
  • Under Object Service in the Status card, confirm that both the Object Service and Data Resiliency are in Ready state (green tick).
  • Update the OpenShift Container Platform cluster to the latest stable release of version 4.5.X or 4.6.Y, see Updating Clusters.

  • Switch the Red Hat OpenShift Container Storage channel from stable-4.5 to stable-4.6. For details about channels, see OpenShift Container Storage upgrade channels and releases.

    Note

    You are required to switch channels only when you are updating minor versions (for example, updating from 4.5 to 4.6) and not when updating between batch updates of 4.6 (for example, updating from 4.6.0 to 4.6.1).

  • Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop down list.

  • Ensure that you have sufficient time to complete the Openshift Container Storage update process, as the update time varies depending on the number of OSDs that run in the cluster.

Procedure

  1. Log in to OpenShift Web Console.
  2. Click Operators → Installed Operators
  3. Select the openshift-storage project.
  4. Click the OpenShift Container Storage operator name.
  5. Click the Subscription tab and click the link under Approval.
  6. Select Automatic (default) and click Save.

  7. Perform one of the following depending on the Upgrade Status:

    • Upgrade Status shows requires approval.

      Note

      Upgrade status shows requires approval if the new OpenShift Container Storage version is already detected in the channel, and approval strategy was changed from Manual to Automatic at the time of update.

      1. Click on the Install Plan link.
      2. On the InstallPlan Details page, click Preview Install Plan.
      3. Review the install plan and click Approve.
      4. Wait for the Status to change from Unknown to Created.
      5. Click Operators → Installed Operators
      6. Select the openshift-storage project.
      7. Wait for the Status to change to Up to date

    • Upgrade Status does not show requires approval:

      1. Wait for the update to initiate. This may take up to 20 minutes.
      2. Click Operators → Installed Operators
      3. Select the openshift-storage project.
      4. Wait for the Status to change to Up to date

Verification steps

  1. Click Overview → Persistent Storage tab and in the Status card confirm that the OCS Cluster and Data Resiliency has a green tick mark indicating it is healthy.
  2. Click Overview → Object Service tab and in the Status card confirm that both the Object Service and Data Resiliency are in Ready state (green tick) indicating it is healthy.

  3. Click Operators → Installed Operators → OpenShift Container Storage Operator. Under Storage Cluster, verify that the cluster service status is Ready.

    Note

    Once updated from OpenShift Container Storage version 4.5 to 4.6, the Version field here will still display 4.5. This is because the ocs-operator does not update the string represented in this field.

  4. Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods. Select openshift-storage from the Project drop down list.

  5. If verification steps fail, contact Red Hat Support.

Additional Resources

If you face any issues while updating OpenShift Container Storage, see the Commonly required logs for troubleshooting section in the Troubleshooting guide.

3.2. Manually updating OpenShift Container Storage operator in internal mode

Use this procedure to update OpenShift Container Storage operator by providing manual approval to the install plan.

Prerequisites

  • Under Persistent Storage in the Status card, confirm that the OCS Cluster and Data Resiliency has a green tick mark.
  • Under Object Service in the Status card, confirm that both the Object Service and Data Resiliency are in Ready state (green tick).
  • Update the OpenShift Container Platform cluster to the latest stable release of version 4.5.X or 4.6.Y, see Updating Clusters.

  • Switch the Red Hat OpenShift Container Storage channel from stable-4.5 to stable-4.6. For details about channels, see OpenShift Container Storage upgrade channels and releases.

    Note

    You are required to switch channels only when you are updating minor versions (for example, updating from 4.5 to 4.6) and not when updating between batch updates of 4.6 (for example, updating from 4.6.0 to 4.6.1).

  • Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop down list.

  • Ensure that you have sufficient time to complete the Openshift Container Storage update process, as the update time varies depending on the number of OSDs that run in the cluster.

Procedure

  1. Log in to OpenShift Web Console.
  2. Click Operators → Installed Operators
  3. Select the openshift-storage project.
  4. Click the OpenShift Container Storage operator name.
  5. Click the Subscription tab and click the link under Approval.
  6. Select Manual and click Save.
  7. Wait for the Upgrade Status to change to Upgrading.
  8. If the Upgrade Status shows requires approval, click on requires approval.
  9. On the InstallPlan Details page, click Preview Install Plan.
  10. Review the install plan and click Approve.
  11. Wait for the Status to change from Unknown to Created.
  12. Click Operators → Installed Operators
  13. Select the openshift-storage project.
  14. Wait for the Status to change to Up to date

Verification steps

  1. Click Overview → Persistent Storage tab and in the Status card confirm that the OCS Cluster and Data Resiliency has a green tick mark indicating it is healthy.
  2. Click Overview → Object Service tab and in the Status card confirm that both the Object Service and Data Resiliency are in Ready state (green tick) indicating it is healthy.

  3. Click Operators → Installed Operators → OpenShift Container Storage Operator. Under Storage Cluster, verify that the cluster service status is Ready.

    Note

    Once updated from OpenShift Container Storage version 4.5 to 4.6, the Version field here will still display 4.5. This is because the ocs-operator does not update the string represented in this field.

  4. Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop down list.

  5. If verification steps fail, contact Red Hat Support.

Additional Resources

If you face any issues while updating OpenShift Container Storage, see the Commonly required logs for troubleshooting section in the Troubleshooting guide.

Chapter 4. Updating OpenShift Container Storage in external mode

Use the following procedures to update your OpenShift Container Storage cluster deployed in external mode.

Important

Upgrading Red Hat OpenShift Container Storage Operator does not upgrade the external Red Hat Ceph Storage cluster. It only upgrades the Red Hat OpenShift Container Storage Services running on the OpenShift Container Platform.

To upgrade the external Red Hat Ceph Storage cluster contact your Red Hat Ceph Storage administrator.

4.1. Enabling automatic updates for OpenShift Container Storage operator in external mode

Use this procedure to enable automatic update approval for updating OpenShift Container Storage operator in OpenShift Container Platform.

Note

Updating OpenShift Container Storage will not update the external Red Hat Ceph Storage cluster.

Prerequisites

  • Red Hat Ceph Storage version 4.2z1 or later is required for the external cluster. For more information, see this knowledge base article on Red Hat Ceph Storage releases and corresponding Ceph package versions.
  • Update the OpenShift Container Platform cluster to the latest stable release of version 4.5.X or 4.6.Y, see Updating Clusters.

  • Switch the Red Hat OpenShift Container Storage channel from stable-4.5 to stable-4.6. For details about channels, see OpenShift Container Storage upgrade channels and releases.

    Note

    You are required to switch channels only when you are updating minor versions (for example, updating from 4.5 to 4.6) and not when updating between batch updates of 4.6 (for example, updating from 4.6.0 to 4.6.1).

  • Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop down list.

  • Under Persistent Storage in the Status card, confirm that the OCS Cluster has a green tick mark.
  • Under Object Service in the Status card, confirm that both the Object Service and Data Resiliency are in Ready state (green tick).
  • Ensure that you have sufficient time to complete the Openshift Container Storage update process.

Procedure

  1. Log in to OpenShift Web Console.
  2. Click Operators → Installed Operators
  3. Select the openshift-storage project.
  4. Click on the OpenShift Container Storage operator name.
  5. Click the Subscription tab and click the link under Approval.
  6. Select Automatic (default) and click Save.

  7. Perform one of the following depending on the Upgrade Status:

    • Upgrade Status shows requires approval.

      Note

      Upgrade status shows requires approval if the new OpenShift Container Storage version is already detected in the channel, and approval strategy was changed from Manual to Automatic at the time of update.

      1. Click the Install Plan link.
      2. On the InstallPlan Details page, click Preview Install Plan.
      3. Review the install plan and click Approve.
      4. Wait for the Status to change from Unknown to Created.
      5. Click Operators → Installed Operators
      6. Select the openshift-storage project.
      7. Wait for the Status to change to Up to date

    • Upgrade Status does not show requires approval:

      1. Wait for the update to initiate. This may take up to 20 minutes.
      2. Click Operators → Installed Operators
      3. Select the openshift-storage project.
      4. Wait for the Status to change to Up to date

Verification steps

  1. Click Overview → Persistent Storage tab and in the Status card confirm that the OCS Cluster has a green tick mark indicating it is healthy.
  2. Click Overview → Object Service tab and in the Status card confirm that both the Object Service and Data Resiliency are in Ready state (green tick) indicating it is healthy.

  3. Click Operators → Installed Operators → OpenShift Container Storage Operator. Under Storage Cluster, verify that the cluster service status in Ready.

    Note

    Once updated from OpenShift Container Storage version 4.5 to 4.6, the Version field here will still display 4.5. This is because the ocs-operator does not update the string represented in this field.

  4. Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop down list.

  5. If verification steps fail, contact Red Hat Support.

Next steps

  1. Enable monitoring to view the Object Gateway (RGW) metrics. See Enabling monitoring for the Object Service dashboard.

4.2. Manually updating OpenShift Container Storage operator in external mode

Use this procedure to update OpenShift Container Storage operator by providing manual approval to the install plan.

Note

Updating OpenShift Container Storage will not update the external Red Hat Ceph Storage cluster.

Prerequisites

  • Red Hat Ceph Storage version 4.2z1 or later is required for the external cluster. For more information, see this knowledge base article on Red Hat Ceph Storage releases and corresponding Ceph package versions.
  • Update the OpenShift Container Platform cluster to the latest stable release of version 4.5.X or 4.6.Y, see Updating Clusters.

  • Switch the Red Hat OpenShift Container Storage channel from stable-4.5 to stable-4.6. For details about channels, see OpenShift Container Storage upgrade channels and releases.

    Note

    You are required to switch channels only when you are updating minor versions (for example, updating from 4.5 to 4.6) and not when updating between batch updates of 4.6 (for example, updating from 4.6.0 to 4.6.1).

  • Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop down list.

  • Under Persistent Storage in the Status card, confirm that the OCS Cluster has a green tick mark.
  • Under Object Service in the Status card, confirm that both the Object Service and Data Resiliency are in Ready state (green tick).
  • Ensure that you have sufficient time to complete the Openshift Container Storage update process.

Procedure

  1. Log in to OpenShift Web Console.
  2. Click Operators → Installed Operators
  3. Select the openshift-storage project.
  4. Click the OpenShift Container Storage operator name.
  5. Click Subscription tab and click the link under Approval.
  6. Select Manual and click Save.
  7. Wait for the Upgrade Status to change to Upgrading.
  8. If the Upgrade Status shows requires approval, click on requires approval.
  9. On the InstallPlan Details page, click Preview Install Plan.
  10. Review the install plan and click Approve.
  11. Wait for the Status to change from Unknown to Created.
  12. Click Operators → Installed Operators
  13. Select the openshift-storage project.
  14. Wait for the Status to change to Up to date

Verification steps

  1. Click Overview → Persistent Storage tab and in the Status card confirm that the OCS Cluster has a green tick mark indicating it is healthy.
  2. Click Overview → Object Service tab and in the Status card confirm that both the Object Service and Data Resiliency are in Ready state (green tick) indicating it is healthy.
  3. Click Operators → Installed Operators → OpenShift Container Storage Operator. Under Storage Cluster, verify that the cluster service status in Ready.

  4. Ensure that all OpenShift Container Storage Pods, including the operator pods, are in Running state in the openshift-storage namespace.

    To view the state of the pods, click Workloads → Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop down list.

    Note

    Once updated from OpenShift Container Storage version 4.5 to 4.6, the Version field here will still display 4.5. This is because the ocs-operator does not update the string represented in this field.

  5. If verification steps fail, contact Red Hat Support.

Next steps

  1. Enable monitoring to view the Object Gateway (RGW) metrics. See Enabling monitoring for the Object Service dashboard.

4.3. Enabling monitoring for the Object Service dashboard

In order to view the Object Gateway (RGW) metrics in the Object Service dashboard in OpenShift Container Storage 4.6, you must enable monitoring. For more information about the Object Service Dashboard, see Object Service dashboard indicators and Metrics in the Object Service dashboard.

Prerequisites

Procedure

  1. Download the python script for updating and extracting the Ceph cluster details.

    1. Click Operators → Installed Operators on the OpenShift Web Console to view the installed operators.
    2. On the Installed Operators page, click the Storage Cluster link in the Provided APIs column for the installed OpenShift Container Storage operator.
    3. Click Create Storage Cluster.
    4. Select Mode as External.
    5. In the Connect to external cluster section, click the Download Script link to download the python script for extracting the Ceph cluster details.
    6. Click Cancel as the storage cluster has already been created.

  2. Update the permission caps for the OpenShift Container Storage client on the Red Hat Ceph Storage cluster.

    1. Contact your Red Hat Ceph Storage administrator to run the downloaded python script on a Red Hat Ceph Storage client node.

    2. To run the script: 

      # python3 ceph-external-cluster-details-exporter.py --upgrade --run-as-user=<client_name_used_for_OCS_4.5_install>

      --run-as-user provides the client name used during the OpenShift Container Storage 4.5 deployment. If this option was not used during the deployment of OpenShift Container Storage 4.5, the default client name client.healthchecker is set.

      The updated permissions for the user are set as: 

      caps: [mgr] allow command config
caps: [mon] allow r, allow command quorum_status, allow command version
caps: [osd] allow rwx pool=default.rgw.meta, allow r pool=.rgw.root, allow rw pool=default.rgw.control, allow rx pool=default.rgw.log, allow x pool=default.rgw.buckets.index

  3. To enable monitoring for the Object Service Dashboard, the monitoring information must be added to the external secret rook-ceph-external-cluster-details in the openshift-storage namespace. There are two ways to update the secret:

    • To update the secret using the CLI, refer to this Red Hat Knowledgebase article.

    • To update the secret from the OpenShift Web Console, follow the steps below.

      1. Run the script downloaded in step 1v: 

        # python3 ceph-external-cluster-details-exporter.py --rbd-data-pool-name <rbd block pool name> --monitoring-endpoint <ceph mgr prometheus exporter endpoint> --monitoring-endpoint-port <ceph mgr prometheus exporter port> --run-as-user <client_name_used_for_OCS_4.5_install>  [optional arguments]

        monitoring-endpoint is the IP address of the active ceph-mgr reachable from the OpenShift Container Platform cluster.

        monitoring-endpoint-port is the port associated with the ceph-mgr Prometheus exporter specified by --monitoring-endpoint. Only port 9283 is supported in OpenShift Container Storage 4.6.

        Note

        Ensure that all the parameters, including the optional arguments, except for monitoring-endpoint and monitoring-endpoint-port, are the same parameters that were used during the deployment of OpenShift Container Storage 4.5 in external mode.

      2. Save the JSON output generated after running the script in the previous step. Example output: 

        [{"name": "rook-ceph-mon-endpoints", "kind": "ConfigMap", "data": {"data": "xxx.xxx.xxx.xxx:xxxx", "maxMonId": "0", "mapping": "{}"}}, {"name": "rook-ceph-mon", "kind": "Secret", "data": {"admin-secret": "admin-secret", "fsid": "<fs-id>", "mon-secret": "mon-secret"}}, {"name": "rook-ceph-operator-creds", "kind": "Secret", "data": {"userID": "client.healthchecker", "userKey": "<user-key>"}}, {"name": "rook-csi-rbd-node", "kind": "Secret", "data": {"userID": "csi-rbd-node", "userKey": "<user-key>"}}, {"name": "ceph-rbd", "kind": "StorageClass", "data": {"pool": "ceph-rbd"}}, {"name": "monitoring-endpoint", "kind": "CephCluster", "data": {"MonitoringEndpoint": "xxx.xxx.xxx.xxx", "MonitoringPort": "xxxx"}}, {"name": "rook-csi-rbd-provisioner", "kind": "Secret", "data": {"userID": "csi-rbd-provisioner", "userKey": "<user-key>"}}, {"name": "rook-csi-cephfs-provisioner", "kind": "Secret", "data": {"adminID": "csi-cephfs-provisioner", "adminKey": "<admin-key>"}}, {"name": "rook-csi-cephfs-node", "kind": "Secret", "data": {"adminID": "csi-cephfs-node", "adminKey": "<admin-key>"}}, {"name": "cephfs", "kind": "StorageClass", "data": {"fsName": "cephfs", "pool": "cephfs_data"}}, {"name": "ceph-rgw", "kind": "StorageClass", "data": {"endpoint": "xxx.xxx.xxx.xxx:xxxx", "poolPrefix": "default"}}]

      3. On the OpenShift Container Platform Web Console, under the openshift-storage namespace, navigate to Workloads → Secrets and click rook-ceph-external-cluster-details:

        Example rook-ceph-external-cluster-details page
      4. From the Actions dropdown menu, click Edit Secret.

      5. On the Edit Key/Value Secret page, click Browse next to the Value text box and upload the JSON file saved previously.

        Example Edit Key/Value Secret page
      6. Click Save.

Verification Steps

  1. Click Home → Overview from the left pane of the OpenShift Web Console and click the Object Service tab.

  2. In the Status card, verify that the Object Service displays a green tick icon as shown in following image:

    Example of Object Service status verified

  3. In the Performance card, select Object Gateway (RGW) from the dropdown to view the metrics:

    Example Object Gateway (RGW) metrics
  4. Click Overview → Persistent Storage tab and in Status card confirm that the OCS Cluster has a green tick mark.

Chapter 5. Post-update configuration changes

In some cases, additional configuration steps are required after an update to ensure that all features work as expected.

5.1. Post-update configuration for clusters backed by local storage

In Red Hat OpenShift Container Platform 4.6 and onward, the Local Storage operator provides new custom resource types for managing local storage:

  • LocalVolumeDiscovery
  • LocalVolumeSet

These resource types are not automatically handled as part of an update from earlier versions, and must be created manually.

5.1.1. Creating a LocalVolumeDiscovery custom resource using the command line

Create a LocalVolumeDiscovery custom resource to ensure that the device management user interface can discover the state of local devices and provide information about devices that are available on cluster nodes.

Prerequisites

  • Administrative access to the OpenShift Container Platform cluster.

Procedure

  1. Change into the project that has Local Storage operator installed. 

    $ oc project local-storage-project

    Replace local-storage-project with the name of your Local Storage project.

    In version 4.5 and earlier the name of the default local storage project is local-storage. In version 4.6 and later, the name of the default local storage project is openshift-local-storage.

  2. Define the LocalVolumeDiscovery custom resource.

    For example, define the following in a local-volume-discovery.yaml file. 

    apiVersion: local.storage.openshift.io/v1alpha1
kind: LocalVolumeDiscovery
metadata:
  name: auto-discover-devices
spec:
  nodeSelector:
    nodeSelectorTerms:
      - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
              - worker1.example.com
              - worker2.example.com
              - worker3.example.com

  3. Create the LocalVolumeDiscovery custom resource. 

    $ oc create -f local-volume-discovery.yaml

Verification steps

  1. Log in to the OpenShift web console.
  2. Click Compute → Node and click on the name of the node.
  3. Click the Disks tab and check that you can see the devices available on that node.

5.1.2. Creating a LocalVolumeSet custom resource using the command line

Create a LocalVolumeSet custom resource to automatically provision certain storage devices as persistent volumes based on criteria that you specify. Persistent volumes are created for any devices that match the deviceInclusionSpec criteria on any node that matches the nodeSelector criteria.

Prerequisites

  • Administrative access to the OpenShift Container Platform cluster.

Procedure

  1. Define a LocalVolumeSet custom resource in a local-volume-set.yaml file. 

    apiVersion: local.storage.openshift.io/v1alpha1
kind: LocalVolumeSet
metadata:
  name: localblock
spec:
  nodeSelector:
    nodeSelectorTerms:
      - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
              - worker1.example.com
              - worker2.example.com
              - worker3.example.com
  storageClassName: localblock
  volumeMode: Block
  maxDeviceCount: 10 # optional, limit devices provisioned per node
  deviceInclusionSpec:
    deviceTypes: # list of types to allow
      - disk
      - part # omit this to use only whole devices
    deviceMechanicalProperty:
      - NonRotational
    minSize: 100Gi # optional, minimum size of device to allow
    maxSize: 100Ti # optional, maximum size of device to allow
    models: # (optional) list of models to allow
      - SAMSUNG
      - Crucial_CT525MX3
    vendors: # (optional) list of device vendors to allow
      - ATA
      - ST2000LM

    The above definition selects whole disks or partitions on specific models of non-rotational devices that are between 100 GB and 100 TB in size, provided by specific vendors, from the worker1, worker2 and worker3 nodes. The localblock storage class is created and persistent volumes are provisioned from discovered devices.

    Important

    Select an appropriate value for minSize to ensure system partitions are not selected.

  2. Create the LocalVolumeSet. 

    $ oc create -f local-volume-set.yaml

Verification steps

  1. Use the following command to track provisioning of persistent volumes for devices that match the deviceInclusionSpec. It can take a few minutes to provision persistent volumes. 

    $ oc describe localvolumeset localblock
[...]
Status:
  Conditions:
    Last Transition Time:          2020-11-17T05:03:32Z
    Message:                       DiskMaker: Available, LocalProvisioner: Available
    Status:                        True
    Type:                          DaemonSetsAvailable
    Last Transition Time:          2020-11-17T05:03:34Z
    Message:                       Operator reconciled successfully.
    Status:                        True
    Type:                          Available
  Observed Generation:             1
  Total Provisioned Device Count: 4
Events:
Type    Reason      Age          From                Message
----    ------      ----         ----                -------
Normal  Discovered  2m30s (x4    localvolumeset-     ip-10-0-147-124.us-east-
        NewDevice   over 2m30s)  symlink-controller  2.compute.internal -
                                                     found possible matching
                                                     disk, waiting 1m to claim
Normal  FoundMatch  89s (x4      localvolumeset-     ip-10-0-147-124.us-east-
        ingDisk     over 89s)    symlink-controller  2.compute.internal -
                                                     symlinking matching disk

  2. Verify the state of the provisioned persistent volumes. 

    $ oc get pv
                     ACCESS   RECLAIM             STORAGE
NAME       CAPACITY  MODES    POLICY   STATUS     CLASS       AGE
local-pv-  500Gi     RWO      Delete   Available  localblock  7m48s
3584969f
local-pv-  500Gi     RWO      Delete   Available  localblock  7m48s
3aee84fa
local-pv-  500Gi     RWO      Delete   Available  localblock  7m48s
644d09ac
local-pv-  500Gi     RWO      Delete   Available  localblock  7m48s
c73cee1

5.1.3. Adding annotations

Use this procedure to add annotations to storage cluster to enable replacing of failed storage devices through the user interface when you upgraded to OpenShift Container Storage 4.6 from a previous version.

Procedure

  1. Log in to OpenShift Container Platform Web Console.
  2. Click Home → Search.
  3. Search for StorageCluster in Resources and click on it.
  4. Beside ocs-storagecluster, click Action menu (⋮) → Edit annotations.
  5. Add cluster.ocs.openshift.io/local-devices and true for KEY and VALUE respectively.
  6. Click Save.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.