Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 1. Deploying OpenShift Data Foundation on single node OpenShift clusters using RHACM

1.1. Requirements for deploying using RHACM

Before you begin deploying OpenShift Data Foundation Logical Volume Manager Operator on single node Openshift clusters, ensure that the following requirements are met:

  1. You have installed Red Hat Advanced Cluster Management for Kubernetes (RHACM) on an OpenShift cluster. For information, see Red Hat Advanced Cluster Management for Kubernetes: Install.
  2. Every managed SNO cluster has dedicated disks that are used to provision storage.

1.2. Installing the OpenShift Data Foundation Logical Volume Manager Operator using RHACM

The OpenShift Data Foundation Logical Volume Manager Operator is deployed on single node OpenShift (SNO) clusters using Red Hat Advanced Cluster Management for Kubernetes (RHACM). You create a Policy on RHACM that deploys and configures the operator when it is applied to managed clusters which match the selector specified in the PlacementRule. The policy is also applied to clusters that are imported later and satisfy the PlacementRule.

Prerequisites

  • Access to the RHACM cluster using an account with cluster-admin and operator installation permissions.
  • Dedicated disks on each SNO cluster to be used by OpenShift Data Foundation Logical Volume Manager Operator.
Note

Ensure that no other storage provisioner is installed on the SNO cluster. OpenShift Data Foundation Logical Volume Manager Operator should be the only storage provisioner as it uses all available disks on the node.

Procedure

  1. Log in to the RHACM CLI using your OpenShift credentials.

    For more information, see Install Red Hat Advanced Cluster Management for Kubernetes.

  2. Create a namespace in which you will create policies. 

    # oc create ns lvm-policy-ns

  3. Save the following YAML to a file with a name such as policy-lvm-operator.yaml to create a policy. 

    # This policy verifies the installation of the official version of the {product-name-short}
# Logical Volume Manager Operator on the managed clusters.
# If set to "enforce" it installs the operator.
# Used APIs: OLM, ODF-LVMO #https://github.com/operator-framework/operator-lifecycle-manager
# https://github.com/red-hat-storage/lvm-operator

apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-install-odf-lvm-operator
spec:
  clusterConditions:
  - status: "True"
    type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
    - key: vendor
      operator: In
      values:
      - OpenShift
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-install-odf-lvm-operator
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-install-odf-lvm-operator
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: Policy
  name: install-odf-lvm-operator
---
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  annotations:
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
    policy.open-cluster-management.io/standards: NIST SP 800-53
  name: install-odf-lvm-operator
spec:
  disabled: false
  remediationAction: enforce
  policy-templates:
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: install-odf-lvm-operator
      spec:
        object-templates:
        - complianceType: musthave
          objectDefinition:
            apiVersion: v1
            kind: Namespace
            metadata:
              labels:
                openshift.io/cluster-monitoring: "true"
                pod-security.kubernetes.io/enforce: privileged
                pod-security.kubernetes.io/audit: privileged
                pod-security.kubernetes.io/warn: privileged
              name: openshift-storage
        - complianceType: musthave
          objectDefinition:
            apiVersion: operators.coreos.com/v1
            kind: OperatorGroup
            metadata:
              name: openshift-storage-operatorgroup
              namespace: openshift-storage
            spec:
              targetNamespaces:
              - openshift-storage
        - complianceType: musthave
          objectDefinition:
            apiVersion: operators.coreos.com/v1alpha1
            kind: Subscription
            metadata:
              name: odf-lvm-operator
              namespace: openshift-storage
            spec:
              installPlanApproval: Automatic
              name: odf-lvm-operator
              source: redhat-operators
              sourceNamespace: openshift-marketplace
        remediationAction: enforce
        severity: low
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: odf-lvmcluster
      spec:
        object-templates:
           - complianceType: musthave
             objectDefinition:
               apiVersion: lvm.topolvm.io/v1alpha1
               kind: LVMCluster
               metadata:
                 name: odf-lvmcluster
                 namespace: openshift-storage
               spec:
                 storage:
                   deviceClasses:
                   - name: vg1
                     thinPoolConfig:
                       name: thin-pool-1
                       sizePercent: 90
                       overprovisionRatio: 10
        remediationAction: enforce
        severity: low

  4. Create the policy in the namespace by running the following command: 

    # oc create -f policy-lvm-operator.yaml -n lvm-policy-ns

    where, policy-lvm-operator.yaml is the name of the file to which the policy is saved.

    This creates a Policy, a PlacementRule, and a PlacementBinding in the namespace, lvm-policy-ns. The Policy creates a Namespace, OperatorGroup, Subscription, and LVMCluster resource on the clusters matching the PlacementRule. This deploys the operator on the SNO clusters which match the selection criteria and configures it to set up the required resources to provision storage. The operator uses all the unused disks after installation.

1.3. Uninstalling OpenShift Data Foundation Logical Volume Manager Operator installed using RHACM

To uninstall OpenShift Data Foundation Logical Volume Manager Operator when you have installed the operator using RHACM, you need to delete the ACM policy that you created for deploying and configuring the operator. However, when you delete the ACM policy, the resources that the policy has created are not removed. You need to create additional policies to remove the resources.

As the resources that are created are not removed when you delete the policy, you need to perform the following steps:

  • Remove all the PVCs and volume snapshots provisioned by the Logical Volume Manager Operator.
  • Remove the LVMCluster resources to clean up the Logical Volume Manager resources created on the disks.
  • Create an additional policy to uninstall the operator.

Prerequisites

  • Ensure that the following are deleted before deleting the policy:

    • All the applications on the managed clusters that are using the storage provisioned by the OpenShift Data Foundation Logical Volume Manager Operator.
    • Persistent volume claims (PVCs) and persistent volumes (PVs) provisioned using the OpenShift Data Foundation Logical Volume Manager Operator.
    • All volume snapshots provisioned by the OpenShift Data Foundation Logical Volume Manager Operator.
  • Ensure that no logical volume resources exist by using the oc get logicalvolume command.
  • Access to the RHACM cluster using an account with cluster-admin role.

Procedure

  1. In the OpenShift command-line interface, delete the ACM policy that you created for deploying and configuring the OpenShift Data Foundation Logical Volume Manager Operator on the hub cluster by using the following command: 

    # oc delete -f policy-lvm-operator.yaml -n lvm-policy-ns

  2. Save the following YAML to a file with a name such as odf-lvmcluster-deletion.yaml to create a policy for removing the LVMCluster. This enables the operator to clean up all the Logical Volume Manager resources that it created on the cluster. 

    apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  name: policy-lvmcluster-delete
  annotations:
    policy.open-cluster-management.io/standards: NIST SP 800-53
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
  remediationAction: enforce
  disabled: false
  policy-templates:
    - objectDefinition:
        apiVersion: policy.open-cluster-management.io/v1
        kind: ConfigurationPolicy
        metadata:
          name: policy-lvmcluster-removal
        spec:
          remediationAction: enforce # the policy-template spec.remediationAction is overridden by the preceding parameter value for spec.remediationAction.
          severity: low
          object-templates:
            - complianceType: mustnothave
              objectDefinition:
                kind: LVMCluster
                apiVersion: lvm.topolvm.io/v1alpha1
                metadata:
                  name: odf-lvmcluster
                  namespace: openshift-storage # must have namespace 'openshift-storage'
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
 name: binding-policy-lvmcluster-delete
placementRef:
 apiGroup: apps.open-cluster-management.io
 kind: PlacementRule
 name: placement-policy-lvmcluster-delete
subjects:
 - apiGroup: policy.open-cluster-management.io
   kind: Policy
   name: policy-lvmcluster-delete
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
 name: placement-policy-lvmcluster-delete
spec:
 clusterConditions:
   - status: 'True'
     type: ManagedClusterConditionAvailable
 clusterSelector:
   matchExpressions:
     - key: vendor
       operator: In
       values:
         - OpenShift

  3. Create the policy by running the following command: 

    # oc create -f odf-lvmcluster-deletion.yaml -n lvm-policy-ns

  4. Save the following YAML to a file with a name such as check-odf-lvmcluster-deletion.yaml to create a policy to check if the LVMCluster CR has been removed. 

    apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  name: policy-lvmcluster-inform
  annotations:
    policy.open-cluster-management.io/standards: NIST SP 800-53
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
  remediationAction: inform
  disabled: false
  policy-templates:
    - objectDefinition:
        apiVersion: policy.open-cluster-management.io/v1
        kind: ConfigurationPolicy
        metadata:
          name: policy-lvmcluster-removal-inform
        spec:
          remediationAction: inform # the policy-template spec.remediationAction is overridden by the preceding parameter value for spec.remediationAction.
          severity: low
          object-templates:
            - complianceType: mustnothave
              objectDefinition:
                kind: LVMCluster
                apiVersion: lvm.topolvm.io/v1alpha1
                metadata:
                  name: odf-lvmcluster
                  namespace: openshift-storage # must have namespace 'openshift-storage'
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
 name: binding-policy-lvmcluster-check
placementRef:
 apiGroup: apps.open-cluster-management.io
 kind: PlacementRule
 name: placement-policy-lvmcluster-check
subjects:
 - apiGroup: policy.open-cluster-management.io
   kind: Policy
   name: policy-lvmcluster-inform
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
 name: placement-policy-lvmcluster-check
spec:
 clusterConditions:
   - status: 'True'
     type: ManagedClusterConditionAvailable
 clusterSelector:
   matchExpressions:
     - key: vendor
       operator: In
       values:
         - OpenShift

  5. Create the policy by running the following command: 

    # oc create -f check-odf-lvmcluster-deletion.yaml -n lvm-policy-ns

  6. Check the policy status. 

    # oc get policy -n lvm-policy-ns
NAME                       REMEDIATION ACTION   COMPLIANCE STATE   AGE
policy-lvmcluster-delete   enforce              Compliant          15m
policy-lvmcluster-inform   inform               Compliant          15m

  7. After both the policies are compliant, save the following YAML to a file with a name such as odf-lvm-operator-remove-policy.yaml to create a policy to uninstall the OpenShift Data Foundation Logical Volume Manager Operator. 

    apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-uninstall-odf-lvm-operator
spec:
  clusterConditions:
  - status: "True"
    type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
    - key: vendor
      operator: In
      values:
      - OpenShift
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-uininstall-odf-lvm-operator
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-uninstall-odf-lvm-operator
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: Policy
  name: uninstall-odf-lvm-operator
---
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  annotations:
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
    policy.open-cluster-management.io/standards: NIST SP 800-53
  name: uninstall-odf-lvm-operator
spec:
  disabled: false
  policy-templates:
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: uninstall-odf-lvm-operator
      spec:
        object-templates:
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: v1
            kind: Namespace
            metadata:
              name: openshift-storage
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: operators.coreos.com/v1
            kind: OperatorGroup
            metadata:
              name: openshift-storage-operatorgroup
              namespace: openshift-storage
            spec:
              targetNamespaces:
              - openshift-storage
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: operators.coreos.com/v1alpha1
            kind: Subscription
            metadata:
              name: odf-lvm-operator
              namespace: openshift-storage
            spec:
              installPlanApproval: Automatic
              name: odf-lvm-operator
              source: redhat-operators
              sourceNamespace: openshift-marketplace
        remediationAction: enforce
        severity: low
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: policy-remove-lvm-operator-crds
      spec:
        object-templates:
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: logicalvolumes.topolvm.cybozu.com
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: lvmclusters.lvm.topolvm.io
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: lvmvolumegroupnodestatuses.lvm.topolvm.io
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: lvmvolumegroups.lvm.topolvm.io
        remediationAction: enforce
        severity: high

  8. Create the policy by running the following command: 

    # oc create -f odf-lvm-operator-remove-policy.yaml -ns lvm-policy-ns
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.