Red Hat Summit 红帽全球峰会支持控制台（Console）开发人员开始试用联系人

3.10. Persistent storage using local volumes

OpenShift Container Platform can be provisioned with persistent storage by using local volumes. Local persistent volumes allow you to access local storage devices, such as a disk or partition, by using the standard PVC interface.

Local volumes can be used without manually scheduling Pods to nodes, because the system is aware of the volume node’s constraints. However, local volumes are still subject to the availability of the underlying node and are not suitable for all applications.

注意

Local volumes can only be used as a statically created Persistent Volume.

3.10.1. Installing the Local Storage Operator

The Local Storage Operator is not installed in OpenShift Container Platform by default. Use the following procedure to install and configure this Operator to enable local volumes in your cluster.

Prerequisites

  • Access to the OpenShift Container Platform web console or command-line interface (CLI).

Procedure

  1. Create the local-storage project:

    Copy to Clipboard Toggle word wrap 
    $ oc new-project local-storage

  2. Optional: Allow local storage creation on infrastructure nodes.

    You might want to use the Local Storage Operator to create volumes on infrastructure nodes in support of components such as logging and monitoring.

    You must adjust the default node selector so that the Local Storage Operator includes the infrastructure nodes, and not just worker nodes.

    To block the Local Storage Operator from inheriting the cluster-wide default selector, enter the following command:

    Copy to Clipboard Toggle word wrap 
    $ oc annotate project local-storage openshift.io/node-selector=''

From the UI

To install the Local Storage Operator from the web console, follow these steps:

  1. Log in to the OpenShift Container Platform web console.
  2. Navigate to Operators OperatorHub.
  3. Type Local Storage into the filter box to locate the Local Storage Operator.
  4. Click Install.
  5. On the Install Operator page, select A specific namespace on the cluster. Select local-storage from the drop-down menu.
  6. Adjust the values for Update Channel and Approval Strategy to the values that you want.
  7. Click Install.

Once finished, the Local Storage Operator will be listed in the Installed Operators section of the web console.

From the CLI

  1. Install the Local Storage Operator from the CLI.

    1. Run the following command to get the OpenShift Container Platform major and minor version. It is required for the channel value in the next step.

      Copy to Clipboard Toggle word wrap 
      $ OC_VERSION=$(oc version -o yaml | grep openshiftVersion | \
    grep -o '[0-9]*[.][0-9]*' | head -1)

    2. Create an object YAML file to define an Operator group and subscription for the Local Storage Operator, such as local-storage.yaml:

      Example local-storage

      Copy to Clipboard Toggle word wrap 
      apiVersion: operators.coreos.com/v1alpha2
kind: OperatorGroup
metadata:
  name: local-operator-group
  namespace: local-storage
spec:
  targetNamespaces:
    - local-storage
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: local-storage-operator
  namespace: local-storage
spec:
  channel: "${OC_VERSION}"
  installPlanApproval: Automatic
      1 
      
  name: local-storage-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace

      1
      The user approval policy for an install plan.

  2. Create the Local Storage Operator object by entering the following command:

    Copy to Clipboard Toggle word wrap 
    $ oc apply -f local-storage.yaml

    At this point, the Operator Lifecycle Manager (OLM) is now aware of the Local Storage Operator. A ClusterServiceVersion (CSV) for the Operator should appear in the target namespace, and APIs provided by the Operator should be available for creation.

  3. Verify local storage installation by checking that all pods and the Local Storage Operator have been created:

    1. Check that all the required pods have been created:

      Copy to Clipboard Toggle word wrap 
      $ oc -n local-storage get pods

      Example output

      Copy to Clipboard Toggle word wrap 
      NAME                                      READY   STATUS    RESTARTS   AGE
local-storage-operator-746bf599c9-vlt5t   1/1     Running   0          19m

    2. Check the ClusterServiceVersion (CSV) YAML manifest to see that the Local Storage Operator is available in the local-storage project:

      Copy to Clipboard Toggle word wrap 
      $ oc get csvs -n local-storage

      Example output

      Copy to Clipboard Toggle word wrap 
      NAME                                         DISPLAY         VERSION               REPLACES   PHASE
local-storage-operator.4.2.26-202003230335   Local Storage   4.2.26-202003230335              Succeeded

After all checks have passed, the Local Storage Operator is installed successfully.

3.10.2. Provisioning local volumes by using the Local Storage Operator

Local volumes cannot be created by dynamic provisioning. Instead, persistent volumes can be created by the Local Storage Operator. The local volume provisioner looks for any file system or block volume devices at the paths specified in the defined resource.

Prerequisites

  • The Local Storage Operator is installed.
  • Local disks are attached to the OpenShift Container Platform nodes.

Procedure

  1. Create the local volume resource. This must define the nodes and paths to the local volumes.

    注意

    Do not use different storage class names for the same device. Doing so will create multiple persistent volumes (PV)s.

    Example: Filesystem

    Copy to Clipboard Toggle word wrap 
    apiVersion: "local.storage.openshift.io/v1"
kind: "LocalVolume"
metadata:
  name: "local-disks"
  namespace: "local-storage"
    1 
    
spec:
  nodeSelector:
    2 
    
    nodeSelectorTerms:
    - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - ip-10-0-140-183
          - ip-10-0-158-139
          - ip-10-0-164-33
  storageClassDevices:
    - storageClassName: "local-sc"
      volumeMode: Filesystem
    3 
    
      fsType: xfs
    4 
    
      devicePaths:
    5 
    
        - /path/to/device
    6

    1
    The namespace where the Local Storage Operator is installed.
    2
    Optional: A node selector containing a list of nodes where the local storage volumes are attached. This example uses the node host names, obtained from oc get node. If a value is not defined, then the Local Storage Operator will attempt to find matching disks on all available nodes.
    3
    The volume mode, either Filesystem or Block, defining the type of the local volumes.
    4
    The file system that is created when the local volume is mounted for the first time.
    5
    The path containing a list of local storage devices to choose from.
    6
    Replace this value with your actual local disks filepath to the LocalVolume resource, such as /dev/xvdg. PVs are created for these local disks when the provisioner is deployed successfully.
    注意

    A raw block volume (volumeMode: block) is not formatted with a file system. You should use this mode only if any application running on the pod can use raw block devices.

    Example: Block

    Copy to Clipboard Toggle word wrap 
    apiVersion: "local.storage.openshift.io/v1"
kind: "LocalVolume"
metadata:
  name: "local-disks"
  namespace: "local-storage"
    1 
    
spec:
  nodeSelector:
    2 
    
    nodeSelectorTerms:
    - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - ip-10-0-136-143
          - ip-10-0-140-255
          - ip-10-0-144-180
  storageClassDevices:
    - storageClassName: "localblock-sc"
      volumeMode: Block
    3 
    
      devicePaths:
    4 
    
        - /path/to/device
    5

    1
    The namespace where the Local Storage Operator is installed.
    2
    Optional: A node selector containing a list of nodes where the local storage volumes are attached. This example uses the node host names, obtained from oc get node. If a value is not defined, then the Local Storage Operator will attempt to find matching disks on all available nodes.
    3
    The volume mode, either Filesystem or Block, defining the type of the local volumes.
    4
    The path containing a list of local storage devices to choose from.
    5
    Replace this value with your actual local disks filepath to the LocalVolume resource, such as /dev/xvdg. PVs are created for these local disks when the provisioner is deployed successfully.

  2. Create the local volume resource in your OpenShift Container Platform cluster, specifying the file you just created:

    Copy to Clipboard Toggle word wrap 
    $ oc create -f <local-volume>.yaml

  3. Verify that the provisioner was created, and that the corresponding daemon sets were created:

    Copy to Clipboard Toggle word wrap 
    $ oc get all -n local-storage

    Example output

    Copy to Clipboard Toggle word wrap 
    NAME                                          READY   STATUS    RESTARTS   AGE
pod/local-disks-local-provisioner-h97hj       1/1     Running   0          46m
pod/local-disks-local-provisioner-j4mnn       1/1     Running   0          46m
pod/local-disks-local-provisioner-kbdnx       1/1     Running   0          46m
pod/local-disks-local-diskmaker-ldldw         1/1     Running   0          46m
pod/local-disks-local-diskmaker-lvrv4         1/1     Running   0          46m
pod/local-disks-local-diskmaker-phxdq         1/1     Running   0          46m
pod/local-storage-operator-54564d9988-vxvhx   1/1     Running   0          47m

NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
service/local-storage-operator    ClusterIP   172.30.49.90     <none>        60000/TCP   47m

NAME                                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/local-disks-local-provisioner   3         3         3       3            3           <none>          46m
daemonset.apps/local-disks-local-diskmaker     3         3         3       3            3           <none>          46m

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/local-storage-operator   1/1     1            1           47m

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/local-storage-operator-54564d9988   1         1         1       47m

    Note the desired and current number of daemon set processes. If the desired count is 0, it indicates that the label selectors were invalid.

  4. Verify that the persistent volumes were created:

    Copy to Clipboard Toggle word wrap 
    $ oc get pv

    Example output

    Copy to Clipboard Toggle word wrap 
    NAME                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
local-pv-1cec77cf   100Gi      RWO            Delete           Available           local-sc                88m
local-pv-2ef7cd2a   100Gi      RWO            Delete           Available           local-sc                82m
local-pv-3fa1c73    100Gi      RWO            Delete           Available           local-sc                48m

重要

Editing the LocalVolume object does not change the fsType or volumeMode of existing persistent volumes because doing so might result in a destructive operation.

3.10.3. Provisioning local volumes without the Local Storage Operator

Local volumes cannot be created by dynamic provisioning. Instead, persistent volumes can be created by defining the persistent volume (PV) in an object definition. The local volume provisioner looks for any file system or block volume devices at the paths specified in the defined resource.

重要

Manual provisioning of PVs includes the risk of potential data leaks across PV reuse when PVCs are deleted. The Local Storage Operator is recommended for automating the life cycle of devices when provisioning local PVs.

Prerequisites

  • Local disks are attached to the OpenShift Container Platform nodes.

Procedure

  1. Define the PV. Create a file, such as example-pv-filesystem.yaml or example-pv-block.yaml, with the PersistentVolume object definition. This resource must define the nodes and paths to the local volumes.

    注意

    Do not use different storage class names for the same device. Doing so will create multiple PVs.

    example-pv-filesystem.yaml

    Copy to Clipboard Toggle word wrap 
    apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-pv-filesystem
spec:
  capacity:
    storage: 100Gi
  volumeMode: Filesystem
    1 
    
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
    2 
    
  local:
    path: /dev/xvdf
    3 
    
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - example-node

    1
    The volume mode, either Filesystem or Block, that defines the type of PVs.
    2
    The name of the storage class to use when creating PV resources. Use a storage class that uniquely identifies this set of PVs.
    3
    The path containing a list of local storage devices to choose from.
    注意

    A raw block volume (volumeMode: block) is not formatted with a file system. Use this mode only if any application running on the pod can use raw block devices.

    example-pv-block.yaml

    Copy to Clipboard Toggle word wrap 
    apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-pv-block
spec:
  capacity:
    storage: 100Gi
  volumeMode: Block
    1 
    
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
    2 
    
  local:
    path: /dev/xvdf
    3 
    
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - example-node

    1
    The volume mode, either Filesystem or Block, that defines the type of PVs.
    2
    The name of the storage class to use when creating PV resources. Be sure to use a storage class that uniquely identifies this set of PVs.
    3
    The path containing a list of local storage devices to choose from.

  2. Create the PV resource in your OpenShift Container Platform cluster. Specify the file you just created:

    Copy to Clipboard Toggle word wrap 
    $ oc create -f <example-pv>.yaml

  3. Verify that the local PV was created:

    Copy to Clipboard Toggle word wrap 
    $ oc get pv

    Example output

    Copy to Clipboard Toggle word wrap 
    NAME                    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                STORAGECLASS    REASON   AGE
example-pv-filesystem   100Gi      RWO            Delete           Available                        local-storage            3m47s
example-pv1             1Gi        RWO            Delete           Bound       local-storage/pvc1   local-storage            12h
example-pv2             1Gi        RWO            Delete           Bound       local-storage/pvc2   local-storage            12h
example-pv3             1Gi        RWO            Delete           Bound       local-storage/pvc3   local-storage            12h

3.10.4. Creating the local volume persistent volume claim

Local volumes must be statically created as a persistent volume claim (PVC) to be accessed by the pod.

Prerequisites

  • Persistent volumes have been created using the local volume provisioner.

Procedure

  1. Create the PVC using the corresponding storage class:

    Copy to Clipboard Toggle word wrap 
    kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: local-pvc-name
    1 
    
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Filesystem
    2 
    
  resources:
    requests:
      storage: 100Gi
    3 
    
  storageClassName: local-sc
    4
    1
    Name of the PVC.
    2
    The type of the PVC. Defaults to Filesystem.
    3
    The amount of storage available to the PVC.
    4
    Name of the storage class required by the claim.

  2. Create the PVC in the OpenShift Container Platform cluster, specifying the file you just created:

    Copy to Clipboard Toggle word wrap 
    $ oc create -f <local-pvc>.yaml

3.10.5. Attach the local claim

After a local volume has been mapped to a PersistentVolumeClaim (PVC) it can be specified inside of a resource.

Prerequisites

  • A PVC exists in the same namespace.

Procedure

  1. Include the defined claim in the resource’s Spec. The following example declares the PVC inside a Pod:

    Copy to Clipboard Toggle word wrap 
    apiVersion: v1
kind: Pod
spec:
  ...
  containers:
    volumeMounts:
    - name: localpvc
    1 
    
      mountPath: "/data"
    2 
    
  volumes:
  - name: localpvc
    persistentVolumeClaim:
      claimName: localpvc
    3
    1
    Name of the volume to mount.
    2
    Path inside the Pod where the volume is mounted.
    3
    Name of the existing PVC to use.

  2. Create the resource in the OpenShift Container Platform cluster, specifying the file you just created:

    Copy to Clipboard Toggle word wrap 
    $ oc create -f <local-pod>.yaml

3.10.6. Using tolerations with Local Storage Operator pods

Taints can be applied to nodes to prevent them from running general workloads. To allow the Local Storage Operator to use tainted nodes, you must add tolerations to the Pod or DaemonSet definition. This allows the created resources to run on these tainted nodes.

You apply tolerations to the Local Storage Operator pod through the LocalVolume resource and apply taints to a node through the node specification. A taint on a node instructs the node to repel all pods that do not tolerate the taint. Using a specific taint that is not on other pods ensures that the Local Storage Operator pod can also run on that node.

重要

Taints and tolerations consist of a key, value, and effect. As an argument, it is expressed as key=value:effect. An operator allows you to leave one of these parameters empty.

Prerequisites

  • The Local Storage Operator is installed.
  • Local disks are attached to OpenShift Container Platform nodes with a taint.
  • Tainted nodes are expected to provision local storage.

Procedure

To configure local volumes for scheduling on tainted nodes:

  1. Modify the YAML file that defines the Pod and add the LocalVolume spec, as shown in the following example:

    Copy to Clipboard Toggle word wrap 
      apiVersion: "local.storage.openshift.io/v1"
  kind: "LocalVolume"
  metadata:
    name: "local-disks"
    namespace: "local-storage"
  spec:
    tolerations:
      - key: localstorage
    1 
    
        operator: Equal
    2 
    
        value: "localstorage"
    3 
    
    storageClassDevices:
        - storageClassName: "localblock-sc"
          volumeMode: Block
    4 
    
          devicePaths:
    5 
    
            - /dev/xvdg
    1
    Specify the key that you added to the node.
    2
    Specify the Equal operator to require the key/value parameters to match. If operator is Exists, the system checks that the key exists and ignores the value. If operator is Equal, then the key and value must match.
    3
    Specify the value local of the tainted node.
    4
    The volume mode, either Filesystem or Block, defining the type of the local volumes.
    5
    The path containing a list of local storage devices to choose from.

The defined tolerations will be passed to the resulting daemon sets, allowing the diskmaker and provisioner pods to be created for nodes that contain the specified taints.

3.10.7. Deleting the Local Storage Operator’s resources

3.10.7.1. Removing a local volume

Occasionally, local volumes must be deleted. While removing the entry in the LocalVolume resource and deleting the PersistentVolume is typically enough, if you want to re-use the same device path or have it managed by a different storage class, then additional steps are needed.

警告

The following procedure involves accessing a node as the root user. Modifying the state of the node beyond the steps in this procedure could result in cluster instability.

Prerequisites

  • The persistent volume must be in a Released or Available state.

    警告

    Deleting a persistent volume that is still in use can result in data loss or corruption.

Procedure

  1. Edit the previously created local volume to remove any unwanted disks.

    1. Edit the cluster resource:

      Copy to Clipboard Toggle word wrap 
      $ oc edit localvolume <name> -n local-storage
    2. Navigate to the lines under devicePaths, and delete any representing unwanted disks.

  2. Delete any persistent volumes created.

    Copy to Clipboard Toggle word wrap 
    $ oc delete pv <pv-name>

  3. Delete any symlinks on the node.

    1. Create a debug pod on the node:

      Copy to Clipboard Toggle word wrap 
      $ oc debug node/<node-name>

    2. Change your root directory to the host:

      Copy to Clipboard Toggle word wrap 
      $ chroot /host

    3. Navigate to the directory containing the local volume symlinks.

      Copy to Clipboard Toggle word wrap 
      $ cd /mnt/local-storage/<sc-name>
      1
      1
      The name of the storage class used to create the local volumes.

    4. Delete the symlink belonging to the removed device.

      Copy to Clipboard Toggle word wrap 
      $ rm <symlink>

3.10.7.2. Uninstalling the Local Storage Operator

To uninstall the Local Storage Operator, you must remove the Operator and all created resources in the local-storage project.

警告

Uninstalling the Local Storage Operator while local storage PVs are still in use is not recommended. While the PVs will remain after the Operator’s removal, there might be indeterminate behavior if the Operator is uninstalled and reinstalled without removing the PVs and local storage resources.

Prerequisites

  • Access to the OpenShift Container Platform web console.

Procedure

  1. Delete any local volume resources in the project:

    Copy to Clipboard Toggle word wrap 
    $ oc delete localvolume --all --all-namespaces

  2. Uninstall the Local Storage Operator from the web console.

    1. Log in to the OpenShift Container Platform web console.
    2. Navigate to Operators Installed Operators.
    3. Type Local Storage into the filter box to locate the Local Storage Operator.
    4. Click the Options menu kebab at the end of the Local Storage Operator.
    5. Click Uninstall Operator.
    6. Click Remove in the window that appears.

  3. The PVs created by the Local Storage Operator will remain in the cluster until deleted. Once these volumes are no longer in use, delete them by running the following command:

    Copy to Clipboard Toggle word wrap 
    $ oc delete pv <pv-name>

  4. Delete the local-storage project:

    Copy to Clipboard Toggle word wrap 
    $ oc delete project local-storage
返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

Theme

© 2025 Red Hat, Inc.