Chapter 4. Scaling storage nodes

Procedure

1. Navigate to the OpenShift Web Console.
2. Click on Operators on the left navigation bar.
3. Select Installed Operators.

4. In the window, click OpenShift Container Storage Operator:

   

5. In the top navigation bar, scroll right and click Storage Cluster tab.

   
6. The visible list should have only one item. Click (⋮) on the far right to extend the options menu.

7. Select Add Capacity from the options menu.

   

   From this dialog box, you can set the requested additional capacity and the storage class. Add capacity will show the capacity selected at the time of installation and will allow to add the capacity only in this increment. On AWS, the storage class should be set to gp2. On VMWare, the storage class should be set to thin.

   Note

   The effectively provisioned capacity will be three times as much as you put into the Requested Capacity field because OpenShift Container Storage uses a replica count of 3.

8. Once you are done with your setting, click Add. You will not see the status of the storage cluster until it reaches Ready. You might need to wait a couple of minutes after you see the Ready state.

Verification steps

1. Navigate to Dashboards OCS PV tab, then check the Capacity breakdown card.

   
2. Note that the capacity increases based on your selections.

Procedure

1. To add storage capacity to OpenShift Container Platform nodes with OpenShift Container Storage installed, you need to find the unique by-id identifier for available devices that you want to add, that is, a minimum of one device per worker node. For more details, see Finding available storage devices.

   Note

   Make sure you perform this process for all the existing nodes (minimum of 3) for which you want to add storage.

   Example of LocalVolume CR local-storage-block-expand.yaml using OpenShift Storage Container label as node selector and by-id device identifier:

   apiVersion: local.storage.openshift.io/v1
   kind: LocalVolume
   metadata:
     name: local-block
     namespace: local-storage
   spec:
     nodeSelector:
       nodeSelectorTerms:
       - matchExpressions:
           - key: cluster.ocs.openshift.io/openshift-storage
             operator: In
             values:
             - ""
     storageClassDevices:
       - storageClassName: localblock
         volumeMode: Block
         devicePaths:
           - /dev/disk/by-id/nvme-Amazon_EC2_NVMe_Instance_Storage_AWS10382E5D7441494EC   # <-- modify this line
           - /dev/disk/by-id/nvme-Amazon_EC2_NVMe_Instance_Storage_AWS60382E5D7441494EC   # <-- modify this line
           - /dev/disk/by-id/nvme-Amazon_EC2_NVMe_Instance_Storage_AWS1F45C01D7E84FE3E9   # <-- modify this line
           - /dev/disk/by-id/nvme-Amazon_EC2_NVMe_Instance_Storage_AWS6F45C01D7E84FE3E9   # <-- modify this line
           - /dev/disk/by-id/nvme-Amazon_EC2_NVMe_Instance_Storage_AWS136BC945B4ECB9AE4   # <-- modify this line
           - /dev/disk/by-id/nvme-Amazon_EC2_NVMe_Instance_Storage_AWS636BC945B4ECB9AE4   # <-- modify this line

   You can see that in this CR new devices using by-id have been added. Each device maps to nvme1n1 on one of the three worker nodes.

   • nvme-Amazon_EC2_NVMe_Instance_Storage_AWS60382E5D7441494EC
   • nvme-Amazon_EC2_NVMe_Instance_Storage_AWS6F45C01D7E84FE3E9
   • nvme-Amazon_EC2_NVMe_Instance_Storage_AWS636BC945B4ECB9AE4

2. Create the LocalVolume CR.

   $ oc apply -f local-storage-block-expand.yaml

   Example output:

   localvolume.local.storage.openshift.io/local-block configured

3. Display PVs with localblock.

   $ oc get pv

   Example output:

   local-pv-1d63db9e   1769Gi     RWO            Delete           Available           localblock              33s
   local-pv-1eb9da0a   1769Gi     RWO            Delete           Available           localblock              25s
   local-pv-31021a83   1769Gi     RWO            Delete           Available           localblock              48s
   ...

   There are three more available PVs that you need to add to the StorageCluster.

4. To expand storage capacity, modify the count for StorageDeviceSets from 1 to 2.

   Example of StorageCluster CR cluster-service-AWS-expand.yaml using monDataDirHostPath and localblock StorageClass:

   apiVersion: ocs.openshift.io/v1
   kind: StorageCluster
   metadata:
     name: ocs-storagecluster
     namespace: openshift-storage
   spec:
     manageNodes: false
     monDataDirHostPath: /var/lib/rook
     storageDeviceSets:
     - count: 2   # <-- modify count to 2
       dataPVCTemplate:
         spec:
           accessModes:
           - ReadWriteOnce
           resources:
             requests:
               storage: 1
           storageClassName: localblock
           volumeMode: Block
       name: ocs-deviceset
       placement: {}
       portable: false
       replica: 3
       resources: {}

   Important

   To ensure that the OSDs have a guaranteed size across the nodes, the storage size for storageDeviceSets must be specified as less than or equal to the size of the desired PVs created on the nodes.

5. Create the StorageCluster CR.

   $ oc apply -f cluster-service-AWS-expand.yaml

   Example output:

   storagecluster.ocs.openshift.io/ocs-storagecluster configured

6. Display pods. You must now have three more OSD pods (osd-3, osd-4, and osd-5) and three more osd-prepare pods.

   $ oc get pods -n openshift-storage | grep 'ceph-osd'

   Example output:

   rook-ceph-osd-3-568d8797b6-j5xqx                                  1/1     Running     0          14m
   rook-ceph-osd-4-cc4747fdf-5glgl                                   1/1     Running     0          14m
   rook-ceph-osd-5-94c46bbcc-tb7pw                                   1/1     Running     0          14m
   ...
   rook-ceph-osd-prepare-ocs-deviceset-0-1-mcmlv-qmn4r               0/1     Completed   0          14m
   rook-ceph-osd-prepare-ocs-deviceset-1-1-tjh2d-fl5zc               0/1     Completed   0          14m
   rook-ceph-osd-prepare-ocs-deviceset-2-1-nqlkg-x9wdn               0/1     Completed   0          14m

1. Execute the following command and verify that the new node is present in the output:

   $ oc get nodes --show-labels | grep cluster.ocs.openshift.io/openshift-storage= |cut -d' ' -f1

2. Click Workloads Pods, confirm that at least the following pods on the new node are in Running state:

   • csi-cephfsplugin-*
   • csi-rbdplugin-*

3. Verify the health of OpenShift Container Storage cluster using the persistent storage dashboard:

   1. Click Home Dashboards from the left pane of the OpenShift Web Console and click the OCS PV tab.
   2. In the Status card, verify that the cluster health displays a green tick icon.