Chapter 4. Configuring persistent storage

Procedure

1. Edit the previously created local volume to remove any unwanted disks.

   1. Edit the cluster resource:

      $ oc edit localvolume <name> -n openshift-local-storage

   2. Navigate to the lines under devicePaths, and delete any representing unwanted disks.

2. Delete any persistent volumes created.

   $ oc delete pv <pv-name>

3. Delete directory and included symlinks on the node.

   Warning

   The following step involves accessing a node as the root user. Modifying the state of the node beyond the steps in this procedure could result in cluster instability.

   $ oc debug node/<node-name> -- chroot /host rm -rf /mnt/local-storage/<sc-name> 1

   1

   The name of the storage class used to create the local volumes.

Procedure

1. Delete any local volume resources installed in the project, such as localvolume, localvolumeset, and localvolumediscovery:

   $ oc delete localvolume --all --all-namespaces
   $ oc delete localvolumeset --all --all-namespaces
   $ oc delete localvolumediscovery --all --all-namespaces

2. Uninstall the Local Storage Operator from the web console.

   1. Log in to the OpenShift Container Platform web console.
   2. Navigate to Operators Installed Operators.
   3. Type Local Storage into the filter box to locate the Local Storage Operator.
   4. Click the Options menu at the end of the Local Storage Operator.
   5. Click Uninstall Operator.
   6. Click Remove in the window that appears.

3. The PVs created by the Local Storage Operator will remain in the cluster until deleted. After these volumes are no longer in use, delete them by running the following command:

   $ oc delete pv <pv-name>

4. Delete the openshift-local-storage project:

   $ oc delete project openshift-local-storage

Procedure

1. Create a namespace for the LVM Storage Operator.

   1. Save the following YAML in the lvms-namespace.yaml file:

      apiVersion: v1
      kind: Namespace
      metadata:
        labels:
          openshift.io/cluster-monitoring: "true"
          pod-security.kubernetes.io/enforce: privileged
          pod-security.kubernetes.io/audit: privileged
          pod-security.kubernetes.io/warn: privileged
        name: openshift-storage

   2. Create the Namespace CR:

      $ oc create -f lvms-namespace.yaml

2. Create an Operator group for the LVM Storage Operator.

   1. Save the following YAML in the lvms-operatorgroup.yaml file:

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: openshift-storage-operatorgroup
        namespace: openshift-storage
      spec:
        targetNamespaces:
        - openshift-storage

   2. Create the OperatorGroup CR:

      $ oc create -f lvms-operatorgroup.yaml

3. Subscribe to the LVM Storage Operator.

   1. Save the following YAML in the lvms-sub.yaml file:

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: lvms
        namespace: openshift-storage
      spec:
        installPlanApproval: Automatic
        name: lvms-operator
        source: redhat-operators
        sourceNamespace: openshift-marketplace

   2. Create the Subscription CR:

      $ oc create -f lvms-sub.yaml

4. To verify that the Operator is installed, enter the following command:

   $ oc get csv -n openshift-storage -o custom-columns=Name:.metadata.name,Phase:.status.phase

   Example output

   Name                         Phase
   4.13.0-202301261535          Succeeded

Procedure

1. Log in to the OpenShift Container Platform Web Console.
2. Click Operators OperatorHub.
3. Scroll or type LVM Storage into the Filter by keyword box to find LVM Storage.
4. Click Install.

5. Set the following options on the Install Operator page:

   1. Update Channel as stable-4.13.
   2. Installation Mode as A specific namespace on the cluster.
   3. Installed Namespace as Operator recommended namespace openshift-storage. If the openshift-storage namespace does not exist, it is created during the operator installation.

   4. Approval Strategy as Automatic or Manual.

      If you select Automatic updates, then the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention.

      If you select Manual updates, then the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator to a newer version.

6. Click Install.

Procedure

1. Get the currentCSV value for the LVM Storage Operator by running the following command:

   $ oc get subscription.operators.coreos.com lvms-operator -n <namespace> -o yaml | grep currentCSV

   Example output

   currentCSV: lvms-operator.v4.15.3

2. Delete the subscription by running the following command:

   $ oc delete subscription.operators.coreos.com lvms-operator -n <namespace>

   Example output

   subscription.operators.coreos.com "lvms-operator" deleted

3. Delete the CSV for the LVM Storage Operator in the target namespace by running the following command:

   $ oc delete clusterserviceversion <currentCSV> -n <namespace> 1

   1

   Replace <currentCSV> with the currentCSV value for the LVM Storage Operator.

   Example output

   clusterserviceversion.operators.coreos.com "lvms-operator.v4.15.3" deleted

Procedure

1. From the Operators Installed Operators page, scroll to LVM Storage or type LVM Storage into the Filter by name to find and click on it.
2. Click on the LVMCluster tab.
3. On the right-hand side of the LVMCluster page, select Delete LVMCluster from the Actions drop-down menu.
4. Click on the Details tab.
5. On the right-hand side of the Operator Details page, select Uninstall Operator from the Actions drop-down menu.
6. Select Remove. LVM Storage stops running and is completely removed.

Procedure

1. Follow the steps in the Creating the image set configuration procedure. To create an ImageSetConfiguration resource for LVM Storage, you can use the following example YAML file:

   Example ImageSetConfiguration file for LVM Storage

   kind: ImageSetConfiguration
   apiVersion: mirror.openshift.io/v1alpha2
   archiveSize: 4 1
   storageConfig: 2
     registry:
       imageURL: example.com/mirror/oc-mirror-metadata 3
       skipTLS: false
   mirror:
     platform:
       channels:
       - name: stable-4.13 4
         type: ocp
       graph: true 5
     operators:
     - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.13 6
       packages:
       - name: lvms-operator 7
         channels:
         - name: stable 8
     additionalImages:
     - name: registry.redhat.io/ubi9/ubi:latest 9
     helm: {}

   1

   Add archiveSize to set the maximum size, in GiB, of each file within the image set.

   2

   Set the back-end location to save the image set metadata to. This location can be a registry or local directory. It is required to specify storageConfig values, unless you are using the Technology Preview OCI feature.

   3

   Set the registry URL for the storage backend.

   4

   Set the channel to retrieve the OpenShift Container Platform images from.

   5

   Add graph: true to generate the OpenShift Update Service (OSUS) graph image to allow for an improved cluster update experience when using the web console. For more information, see About the OpenShift Update Service.

   6

   Set the Operator catalog to retrieve the OpenShift Container Platform images from.

   7

   Specify only certain Operator packages to include in the image set. Remove this field to retrieve all packages in the catalog.

   8

   Specify only certain channels of the Operator packages to include in the image set. You must always include the default channel for the Operator package even if you do not use the bundles in that channel. You can find the default channel by running the following command: oc mirror list operators --catalog=<catalog_name> --package=<package_name>.

   9

   Specify any additional images to include in image set.

2. Follow the procedure in the Mirroring an image set to a mirror registry section.
3. Follow the procedure in the Configuring image registry repository mirroring section.

Procedure

1. Log in to the RHACM CLI using your OpenShift Container Platform credentials.

2. Create a namespace in which you will create policies.

   # oc create ns lvms-policy-ns

3. To create a policy, save the following YAML to a file with a name such as policy-lvms-operator.yaml:

   apiVersion: apps.open-cluster-management.io/v1
   kind: PlacementRule
   metadata:
     name: placement-install-lvms
   spec:
     clusterConditions:
     - status: "True"
       type: ManagedClusterConditionAvailable
     clusterSelector: 1
       matchExpressions:
       - key: mykey
         operator: In
         values:
         - myvalue
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: PlacementBinding
   metadata:
     name: binding-install-lvms
   placementRef:
     apiGroup: apps.open-cluster-management.io
     kind: PlacementRule
     name: placement-install-lvms
   subjects:
   - apiGroup: policy.open-cluster-management.io
     kind: Policy
     name: install-lvms
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: Policy
   metadata:
     annotations:
       policy.open-cluster-management.io/categories: CM Configuration Management
       policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
       policy.open-cluster-management.io/standards: NIST SP 800-53
     name: install-lvms
   spec:
     disabled: false
     remediationAction: enforce
     policy-templates:
     - objectDefinition:
         apiVersion: policy.open-cluster-management.io/v1
         kind: ConfigurationPolicy
         metadata:
           name: install-lvms
         spec:
           object-templates:
           - complianceType: musthave
             objectDefinition:
               apiVersion: v1
               kind: Namespace
               metadata:
                 labels:
                   openshift.io/cluster-monitoring: "true"
                   pod-security.kubernetes.io/enforce: privileged
                   pod-security.kubernetes.io/audit: privileged
                   pod-security.kubernetes.io/warn: privileged
                 name: openshift-storage
           - complianceType: musthave
             objectDefinition:
               apiVersion: operators.coreos.com/v1
               kind: OperatorGroup
               metadata:
                 name: openshift-storage-operatorgroup
                 namespace: openshift-storage
               spec:
                 targetNamespaces:
                 - openshift-storage
           - complianceType: musthave
             objectDefinition:
               apiVersion: operators.coreos.com/v1alpha1
               kind: Subscription
               metadata:
                 name: lvms
                 namespace: openshift-storage
               spec:
                 installPlanApproval: Automatic
                 name: lvms-operator
                 source: redhat-operators
                 sourceNamespace: openshift-marketplace
           remediationAction: enforce
           severity: low
     - objectDefinition:
         apiVersion: policy.open-cluster-management.io/v1
         kind: ConfigurationPolicy
         metadata:
           name: lvms
         spec:
           object-templates:
              - complianceType: musthave
                objectDefinition:
                  apiVersion: lvm.topolvm.io/v1alpha1
                  kind: LVMCluster
                  metadata:
                    name: my-lvmcluster
                    namespace: openshift-storage
                  spec:
                    storage:
                      deviceClasses:
                      - name: vg1
                        default: true
                        deviceSelector: 2
                          paths:
                          - /dev/disk/by-path/pci-0000:87:00.0-nvme-1
                          - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
                        thinPoolConfig:
                          name: thin-pool-1
                          sizePercent: 90
                          overprovisionRatio: 10
                        nodeSelector: 3
                          nodeSelectorTerms:
                          - matchExpressions:
                              - key: app
                                operator: In
                                values:
                                - test1
           remediationAction: enforce
           severity: low

   1

   Replace the key and value in PlacementRule.spec.clusterSelector to match the labels set on the single-node OpenShift clusters on which you want to install LVM Storage.

   2

   To control or restrict the volume group to your preferred disks, you can manually specify the local paths of the disks in the deviceSelector section of the LVMCluster YAML.

   3

   To add a node filter, which is a subset of the additional worker nodes, specify the required filter in the nodeSelector section. LVM Storage detects and uses the additional worker nodes when the new nodes show up.

   Important

   This nodeSelector node filter matching is not the same as the pod label matching.

4. Create the policy in the namespace by running the following command:

   # oc create -f policy-lvms-operator.yaml -n lvms-policy-ns 1

   1

   The policy-lvms-operator.yaml is the name of the file to which the policy is saved.

   This creates a Policy, a PlacementRule, and a PlacementBinding object in the lvms-policy-ns namespace. The policy creates a Namespace, OperatorGroup, Subscription, and LVMCluster resource on the clusters that match the placement rule. This deploys the Operator on the single-node OpenShift clusters which match the selection criteria and configures it to set up the required resources to provision storage. The Operator uses all the disks specified in the LVMCluster CR. If no disks are specified, the Operator uses all the unused disks on the single-node OpenShift node.

   Important

   After a device is added to the LVMCluster, it cannot be removed.

1. Remove all the Persistent volume claims (PVCs) and volume snapshots provisioned by LVM Storage.
2. Remove the LVMCluster resources to clean up Logical Volume Manager resources created on the disks.
3. Create an additional policy to uninstall the Operator.

Procedure

1. In the OpenShift CLI (oc), delete the RHACM policy that you created for deploying and configuring LVM Storage on the hub cluster by using the following command:

   # oc delete -f policy-lvms-operator.yaml -n lvms-policy-ns 1

   1

   The policy-lvms-operator.yaml is the name of the file to which the policy was saved.

2. To create a policy for removing the LVMCluster resource, save the following YAML to a file with a name such as lvms-remove-policy.yaml. This enables the Operator to clean up all Logical Volume Manager resources that it created on the cluster.

   apiVersion: policy.open-cluster-management.io/v1
   kind: Policy
   metadata:
     name: policy-lvmcluster-delete
     annotations:
       policy.open-cluster-management.io/standards: NIST SP 800-53
       policy.open-cluster-management.io/categories: CM Configuration Management
       policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
   spec:
     remediationAction: enforce
     disabled: false
     policy-templates:
       - objectDefinition:
           apiVersion: policy.open-cluster-management.io/v1
           kind: ConfigurationPolicy
           metadata:
             name: policy-lvmcluster-removal
           spec:
             remediationAction: enforce 1
             severity: low
             object-templates:
               - complianceType: mustnothave
                 objectDefinition:
                   kind: LVMCluster
                   apiVersion: lvm.topolvm.io/v1alpha1
                   metadata:
                     name: my-lvmcluster
                     namespace: openshift-storage 2
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: PlacementBinding
   metadata:
     name: binding-policy-lvmcluster-delete
   placementRef:
     apiGroup: apps.open-cluster-management.io
     kind: PlacementRule
     name: placement-policy-lvmcluster-delete
   subjects:
     - apiGroup: policy.open-cluster-management.io
       kind: Policy
       name: policy-lvmcluster-delete
   ---
   apiVersion: apps.open-cluster-management.io/v1
   kind: PlacementRule
   metadata:
     name: placement-policy-lvmcluster-delete
   spec:
     clusterConditions:
       - status: "True"
         type: ManagedClusterConditionAvailable
     clusterSelector:
       matchExpressions:
         - key: mykey
           operator: In
           values:
             - myvalue

   1

   The policy-template spec.remediationAction is overridden by the preceding parameter value for spec.remediationAction.

   2

   This namespace field must have the openshift-storage value.

3. Set the value of the PlacementRule.spec.clusterSelector field to select the clusters from which to uninstall LVM Storage.

4. Create the policy by running the following command:

   # oc create -f lvms-remove-policy.yaml -n lvms-policy-ns

5. To create a policy to check if the LVMCluster CR has been removed, save the following YAML to a file with a name such as check-lvms-remove-policy.yaml:

   apiVersion: policy.open-cluster-management.io/v1
   kind: Policy
   metadata:
     name: policy-lvmcluster-inform
     annotations:
       policy.open-cluster-management.io/standards: NIST SP 800-53
       policy.open-cluster-management.io/categories: CM Configuration Management
       policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
   spec:
     remediationAction: inform
     disabled: false
     policy-templates:
       - objectDefinition:
           apiVersion: policy.open-cluster-management.io/v1
           kind: ConfigurationPolicy
           metadata:
             name: policy-lvmcluster-removal-inform
           spec:
             remediationAction: inform 1
             severity: low
             object-templates:
               - complianceType: mustnothave
                 objectDefinition:
                   kind: LVMCluster
                   apiVersion: lvm.topolvm.io/v1alpha1
                   metadata:
                     name: my-lvmcluster
                     namespace: openshift-storage 2
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: PlacementBinding
   metadata:
     name: binding-policy-lvmcluster-check
   placementRef:
     apiGroup: apps.open-cluster-management.io
     kind: PlacementRule
     name: placement-policy-lvmcluster-check
   subjects:
     - apiGroup: policy.open-cluster-management.io
       kind: Policy
       name: policy-lvmcluster-inform
   ---
   apiVersion: apps.open-cluster-management.io/v1
   kind: PlacementRule
   metadata:
     name: placement-policy-lvmcluster-check
   spec:
     clusterConditions:
       - status: "True"
         type: ManagedClusterConditionAvailable
     clusterSelector:
       matchExpressions:
         - key: mykey
           operator: In
           values:
             - myvalue

   1

   The policy-template spec.remediationAction is overridden by the preceding parameter value for spec.remediationAction.

   2

   The namespace field must have the openshift-storage value.

6. Create the policy by running the following command:

   # oc create -f check-lvms-remove-policy.yaml -n lvms-policy-ns

7. Check the policy status by running the following command:

   # oc get policy -n lvms-policy-ns

   Example output

   NAME                       REMEDIATION ACTION   COMPLIANCE STATE   AGE
   policy-lvmcluster-delete   enforce              Compliant          15m
   policy-lvmcluster-inform   inform               Compliant          15m

8. After both the policies are compliant, save the following YAML to a file with a name such as lvms-uninstall-policy.yaml to create a policy to uninstall LVM Storage.

   apiVersion: apps.open-cluster-management.io/v1
   kind: PlacementRule
   metadata:
     name: placement-uninstall-lvms
   spec:
     clusterConditions:
     - status: "True"
       type: ManagedClusterConditionAvailable
     clusterSelector:
       matchExpressions:
       - key: mykey
         operator: In
         values:
         - myvalue
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: PlacementBinding
   metadata:
     name: binding-uninstall-lvms
   placementRef:
     apiGroup: apps.open-cluster-management.io
     kind: PlacementRule
     name: placement-uninstall-lvms
   subjects:
   - apiGroup: policy.open-cluster-management.io
     kind: Policy
     name: uninstall-lvms
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: Policy
   metadata:
     annotations:
       policy.open-cluster-management.io/categories: CM Configuration Management
       policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
       policy.open-cluster-management.io/standards: NIST SP 800-53
     name: uninstall-lvms
   spec:
     disabled: false
     policy-templates:
     - objectDefinition:
         apiVersion: policy.open-cluster-management.io/v1
         kind: ConfigurationPolicy
         metadata:
           name: uninstall-lvms
         spec:
           object-templates:
           - complianceType: mustnothave
             objectDefinition:
               apiVersion: v1
               kind: Namespace
               metadata:
                 name: openshift-storage
           - complianceType: mustnothave
             objectDefinition:
               apiVersion: operators.coreos.com/v1
               kind: OperatorGroup
               metadata:
                 name: openshift-storage-operatorgroup
                 namespace: openshift-storage
               spec:
                 targetNamespaces:
                 - openshift-storage
           - complianceType: mustnothave
             objectDefinition:
               apiVersion: operators.coreos.com/v1alpha1
               kind: Subscription
               metadata:
                 name: lvms-operator
                 namespace: openshift-storage
           remediationAction: enforce
           severity: low
     - objectDefinition:
         apiVersion: policy.open-cluster-management.io/v1
         kind: ConfigurationPolicy
         metadata:
           name: policy-remove-lvms-crds
         spec:
           object-templates:
           - complianceType: mustnothave
             objectDefinition:
               apiVersion: apiextensions.k8s.io/v1
               kind: CustomResourceDefinition
               metadata:
                 name: logicalvolumes.topolvm.io
           - complianceType: mustnothave
             objectDefinition:
               apiVersion: apiextensions.k8s.io/v1
               kind: CustomResourceDefinition
               metadata:
                 name: lvmclusters.lvm.topolvm.io
           - complianceType: mustnothave
             objectDefinition:
               apiVersion: apiextensions.k8s.io/v1
               kind: CustomResourceDefinition
               metadata:
                 name: lvmvolumegroupnodestatuses.lvm.topolvm.io
           - complianceType: mustnothave
             objectDefinition:
               apiVersion: apiextensions.k8s.io/v1
               kind: CustomResourceDefinition
               metadata:
                 name: lvmvolumegroups.lvm.topolvm.io
           remediationAction: enforce
           severity: high

9. Create the policy by running the following command:

   # oc create -f lvms-uninstall-policy.yaml -ns lvms-policy-ns

Procedure

1. Log in to OpenShift Container Platform console of the single-node OpenShift cluster.
2. From the Operators Installed Operators page, click on the LVM Storage Operator in the openshift-storage namespace.
3. Click on the LVMCluster tab to list the LVMCluster CR created on the cluster.
4. Select Edit LVMCluster from the Actions drop-down menu.
5. Click on the YAML tab.

6. Edit the LVMCluster CR YAML to add the new device path in the deviceSelector section:

   Note

   In case the deviceSelector field is not included during the LVMCluster creation, it is not possible to add the deviceSelector section to the CR. You need to remove the LVMCluster and then create a new CR.

   apiVersion: lvm.topolvm.io/v1alpha1
   kind: LVMCluster
   metadata:
     name: my-lvmcluster
   spec:
     storage:
       deviceClasses:
       - name: vg1
         default: true
         deviceSelector:
           paths:
           - /dev/disk/by-path/pci-0000:87:00.0-nvme-1 1
           - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
           - /dev/disk/by-path/pci-0000:89:00.0-nvme-1 2
         thinPoolConfig:
           name: thin-pool-1
           sizePercent: 90
           overprovisionRatio: 10

   1

   The path can be added by name (/dev/sdb) or by path.

   2

   A new disk is added.

Procedure

1. Log in to the RHACM CLI using your OpenShift Container Platform credentials.
2. Find the disk that you want to add. The disk to be added needs to match with the device name and path of the existing disks.

3. To add capacity to the single-node OpenShift cluster, edit the deviceSelector section of the existing policy YAML, for example, policy-lvms-operator.yaml.

   Note

   In case the deviceSelector field is not included during the LVMCluster creation, it is not possible to add the deviceSelector section to the CR. You need to remove the LVMCluster and then recreate from the new CR.

   apiVersion: apps.open-cluster-management.io/v1
   kind: PlacementRule
   metadata:
     name: placement-install-lvms
   spec:
     clusterConditions:
     - status: "True"
       type: ManagedClusterConditionAvailable
     clusterSelector:
       matchExpressions:
       - key: mykey
         operator: In
         values:
         - myvalue
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: PlacementBinding
   metadata:
     name: binding-install-lvms
   placementRef:
     apiGroup: apps.open-cluster-management.io
     kind: PlacementRule
     name: placement-install-lvms
   subjects:
   - apiGroup: policy.open-cluster-management.io
     kind: Policy
     name: install-lvms
   ---
   apiVersion: policy.open-cluster-management.io/v1
   kind: Policy
   metadata:
     annotations:
       policy.open-cluster-management.io/categories: CM Configuration Management
       policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
       policy.open-cluster-management.io/standards: NIST SP 800-53
     name: install-lvms
   spec:
     disabled: false
     remediationAction: enforce
     policy-templates:
     - objectDefinition:
         apiVersion: policy.open-cluster-management.io/v1
         kind: ConfigurationPolicy
         metadata:
           name: install-lvms
         spec:
           object-templates:
           - complianceType: musthave
             objectDefinition:
               apiVersion: v1
               kind: Namespace
               metadata:
                 labels:
                   openshift.io/cluster-monitoring: "true"
                   pod-security.kubernetes.io/enforce: privileged
                   pod-security.kubernetes.io/audit: privileged
                   pod-security.kubernetes.io/warn: privileged
                 name: openshift-storage
           - complianceType: musthave
             objectDefinition:
               apiVersion: operators.coreos.com/v1
               kind: OperatorGroup
               metadata:
                 name: openshift-storage-operatorgroup
                 namespace: openshift-storage
               spec:
                 targetNamespaces:
                 - openshift-storage
           - complianceType: musthave
             objectDefinition:
               apiVersion: operators.coreos.com/v1alpha1
               kind: Subscription
               metadata:
                 name: lvms
                 namespace: openshift-storage
               spec:
                 installPlanApproval: Automatic
                 name: lvms-operator
                 source: redhat-operators
                 sourceNamespace: openshift-marketplace
           remediationAction: enforce
           severity: low
     - objectDefinition:
         apiVersion: policy.open-cluster-management.io/v1
         kind: ConfigurationPolicy
         metadata:
           name: lvms
         spec:
           object-templates:
              - complianceType: musthave
                objectDefinition:
                  apiVersion: lvm.topolvm.io/v1alpha1
                  kind: LVMCluster
                  metadata:
                    name: my-lvmcluster
                    namespace: openshift-storage
                  spec:
                    storage:
                      deviceClasses:
                      - name: vg1
                        default: true
                        deviceSelector:
                          paths:
                          - /dev/disk/by-path/pci-0000:87:00.0-nvme-1
                          - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
                          - /dev/disk/by-path/pci-0000:89:00.0-nvme-1 # new disk is added
                        thinPoolConfig:
                          name: thin-pool-1
                          sizePercent: 90
                          overprovisionRatio: 10
                        nodeSelector:
                          nodeSelectorTerms:
                          - matchExpressions:
                              - key: app
                                operator: In
                                values:
                                - test1
           remediationAction: enforce
           severity: low

4. Edit the policy by running the following command:

   # oc edit -f policy-lvms-operator.yaml -ns lvms-policy-ns 1

   1

   The policy-lvms-operator.yaml is the name of the existing policy.

   This uses the new disk specified in the LVMCluster CR to provision storage.

Procedure

1. Modify the .spec.resources.requests.storage field in the desired PVC resource to the new size by running the following command:

   oc patch <pvc_name> -n <application_namespace> -p '{ "spec": { "resources": { "requests": { "storage": "<desired_size>" }}}}'

2. Watch the status.conditions field of the PVC to see if the resize has completed. OpenShift Container Platform adds the Resizing condition to the PVC during expansion, which is removed after the expansion completes.

Procedure

1. Log in to the single-node OpenShift for which you need to run the oc command.

2. Save the following YAML to a file with a name such as lvms-vol-snapshot.yaml.

   Example YAML to create a volume snapshot

   apiVersion: snapshot.storage.k8s.io/v1
   kind: VolumeSnapshot
   metadata:
       name: lvm-block-1-snap
   spec:
       volumeSnapshotClassName: lvms-vg1
       source:
           persistentVolumeClaimName: lvm-block-1

3. Create the snapshot by running the following command in the same namespace as the PVC:

   # oc create -f lvms-vol-snapshot.yaml

Procedure

1. Identify the storage class name of the source PVC and volume snapshot name.

2. Save the following YAML to a file with a name such as lvms-vol-restore.yaml to restore the snapshot.

   Example YAML to restore a PVC.

   kind: PersistentVolumeClaim
   apiVersion: v1
   metadata:
     name: lvm-block-1-restore
   spec:
     accessModes:
     - ReadWriteOnce
     volumeMode: Block
     Resources:
       Requests:
         storage: 2Gi
     storageClassName: lvms-vg1
     dataSource:
       name: lvm-block-1-snap
       kind: VolumeSnapshot
       apiGroup: snapshot.storage.k8s.io

3. Create the policy by running the following command in the same namespace as the snapshot:

   # oc create -f lvms-vol-restore.yaml

Procedure

1. Delete the volume snapshot resource by running the following command:

   # oc delete volumesnapshot <volume_snapshot_name> -n <namespace>

   Note

   When you delete a persistent volume claim (PVC), the snapshots of the PVC are not deleted.

2. To delete the restored volume snapshot, delete the PVC that was created to restore the volume snapshot by running the following command:

   # oc delete pvc <pvc_name> -n <namespace>

Procedure

1. Identify the storage class of the source PVC.

2. To create a volume clone, save the following YAML to a file with a name such as lvms-vol-clone.yaml:

   Example YAML to clone a volume

   apiVersion: v1
   kind: PersistentVolumeClaim
   Metadata:
     name: lvm-block-1-clone
   Spec:
     storageClassName: lvms-vg1
     dataSource:
       name: lvm-block-1
       kind: PersistentVolumeClaim
     accessModes:
      - ReadWriteOnce
     volumeMode: Block
     Resources:
       Requests:
         storage: 2Gi

3. Create the policy in the same namespace as the source PVC by running the following command:

   # oc create -f lvms-vol-clone.yaml

Procedure

1. Retrieve the list of PVCs by running the following command:

   $ oc get pvc

   Example output

   NAME        STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
   lvms-test   Pending                                      lvms-vg1       11s

2. Inspect the events associated with a PVC stuck in the Pending state by running the following command:

   $ oc describe pvc <pvc_name> 1

   1

   Replace <pvc_name> with the name of the PVC. For example, lvms-vg1.

   Example output

   Type     Reason              Age               From                         Message
   ----     ------              ----              ----                         -------
   Warning  ProvisioningFailed  4s (x2 over 17s)  persistentvolume-controller  storageclass.storage.k8s.io "lvms-vg1" not found

Procedure

1. Verify that the LVMCluster CR is present by running the following command:

   $ oc get lvmcluster -n openshift-storage

   Example output

   NAME            AGE
   my-lvmcluster   65m

2. If the LVMCluster CR is not present, create an LVMCluster CR. For more information, see "Creating a Logical Volume Manager cluster on a single-node OpenShift worker node".

3. In the openshift-storage namespace, check that all the LVM Storage pods are in the Running state by running the following command:

   $ oc get pods -n openshift-storage

   Example output

   NAME                                  READY   STATUS    RESTARTS      AGE
   lvms-operator-7b9fb858cb-6nsml        3/3     Running   0             70m
   topolvm-controller-5dd9cf78b5-7wwr2   5/5     Running   0             66m
   topolvm-node-dr26h                    4/4     Running   0             66m
   vg-manager-r6zdv                      1/1     Running   0             66m

   The output of this command must contain a running instance of the following pods:

   • lvms-operator
   • vg-manager
   • topolvm-controller

   • topolvm-node

     If the topolvm-node pod is stuck in the Init state, it is due to a failure to locate an available disk for LVM Storage to use. To retrieve the necessary information to troubleshoot this issue, review the logs of the vg-manager pod by running the following command:

     $ oc logs -l app.kubernetes.io/component=vg-manager -n openshift-storage

Procedure

1. Inspect the events associated with a PVC by running the following command:

   $ oc describe pvc <pvc_name> 1

   1

   Replace <pvc_name> with the name of the PVC.

2. Establish a direct connection to the host where the problem is occurring.
3. Resolve the disk issue.

Procedure

1. Switch to the openshift-storage namespace by running the following command:

   $ oc project openshift-storage

2. Check if the LogicalVolume custom resources (CRs) are present by running the following command:

   $ oc get logicalvolume

   1. If the LogicalVolume CRs are present, delete them by running the following command:

      $ oc delete logicalvolume <name> 1

      1

      Replace <name> with the name of the LogicalVolume CR.

   2. After deleting the LogicalVolume CRs, remove their finalizers by running the following command:

      $ oc patch logicalvolume <name> -p '{"metadata":{"finalizers":[]}}' --type=merge 1

      1

      Replace <name> with the name of the LogicalVolume CR.

3. Check if the LVMVolumeGroup CRs are present by running the following command:

   $ oc get lvmvolumegroup

   1. If the LVMVolumeGroup CRs are present, delete them by running the following command:

      $ oc delete lvmvolumegroup <name> 1

      1

      Replace <name> with the name of the LVMVolumeGroup CR.

   2. After deleting the LVMVolumeGroup CRs, remove their finalizers by running the following command:

      $ oc patch lvmvolumegroup <name> -p '{"metadata":{"finalizers":[]}}' --type=merge 1

      1

      Replace <name> with the name of the LVMVolumeGroup CR.

4. Delete any LVMVolumeGroupNodeStatus CRs by running the following command:

   $ oc delete lvmvolumegroupnodestatus --all

5. Delete the LVMCluster CR by running the following command:

   $ oc delete lvmcluster --all

   1. After deleting the LVMCluster CR, remove its finalizer by running the following command:

      $ oc patch lvmcluster <name> -p '{"metadata":{"finalizers":[]}}' --type=merge 1

      1

      Replace <name> with the name of the LVMCluster CR.