Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 4. Managing namespace buckets

Namespace buckets let you connect data repositories on different providers together, so that you can interact with all of your data through a single unified view. Add the object bucket associated with each provider to the namespace bucket, and access your data through the namespace bucket to see all of your object buckets at once. This lets you write to your preferred storage provider while reading from multiple other storage providers, greatly reducing the cost of migrating to a new storage provider.

Note

A namespace bucket can only be used if its write target is available and functional.

You can interact with objects in the namespace buckets using the Amazon Simple Storage Service (S3) API.

Ensure that the credentials provided for the Multicloud Object Gateway (MCG) enables you to perform the AWS S3 namespace bucket operations. You can use the AWS tool, aws-cli to verify that all the operations can be performed on the target bucket. Also, the list bucket which is using this MCG account shows the target bucket.

Red Hat OpenShift Data Foundation supports the following namespace bucket operations:

See the Amazon S3 API reference documentation for the most up-to-date information about these operations and how to use them.

Additional resources

For more information about namespace buckets, see Managing namespace buckets.

Depending on the type of your deployment and whether you want to use YAML or the Multicloud Object Gateway (MCG) CLI, choose one of the following procedures to add a namespace bucket:

Prerequisites

Procedure

  1. Create a secret with the credentials: 

    apiVersion: v1
kind: Secret
metadata:
  name: <namespacestore-secret-name>
  type: Opaque
data:
  AWS_ACCESS_KEY_ID: <AWS ACCESS KEY ID ENCODED IN BASE64>
  AWS_SECRET_ACCESS_KEY: <AWS SECRET ACCESS KEY ENCODED IN BASE64>
    Copy to Clipboard Toggle word wrap

    where <namespacestore-secret-name> is a unique NamespaceStore name.

    You must provide and encode your own AWS access key ID and secret access key using Base64, and use the results in place of <AWS ACCESS KEY ID ENCODED IN BASE64> and <AWS SECRET ACCESS KEY ENCODED IN BASE64>.

  2. Create a NamespaceStore resource using OpenShift custom resource definitions (CRDs).

    A NamespaceStore represents underlying storage to be used as a read or write target for the data in the MCG namespace buckets.

    To create a NamespaceStore resource, apply the following YAML: 

    apiVersion: noobaa.io/v1alpha1
kind: NamespaceStore
metadata:
  finalizers:
  - noobaa.io/finalizer
  labels:
    app: noobaa
  name: <resource-name>
  namespace: openshift-storage
spec:
  awsS3:
    secret:
      name: <namespacestore-secret-name>
      namespace: <namespace-secret>
    targetBucket: <target-bucket>
  type: aws-s3
    Copy to Clipboard Toggle word wrap
    <resource-name>
    The name you want to give to the resource.
    <namespacestore-secret-name>
    The secret created in the previous step.
    <namespace-secret>
    The namespace where the secret can be found.
    <target-bucket>
    The target bucket you created for the NamespaceStore.

  3. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • A namespace policy of type single requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type: Single
    single:
      resource: <resource>
      Copy to Clipboard Toggle word wrap
      <my-bucket-class>
      The unique namespace bucket class name.
      <resource>
      The name of a single NamespaceStore that defines the read and write target of the namespace bucket.

    • A namespace policy of type multi requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1

kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type: Multi
    multi:
      writeResource: <write-resource>
      readResources:
      - <read-resources>
      - <read-resources>
      Copy to Clipboard Toggle word wrap
      <my-bucket-class>
      A unique bucket class name.
      <write-resource>
      The name of a single NamespaceStore that defines the write target of the namespace bucket.
      <read-resources>
      A list of the names of the NamespaceStores that defines the read targets of the namespace bucket.

  4. Create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in the earlier step using the following YAML: 

    apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: <resource-name>
  namespace: openshift-storage
spec:
  generateBucketName: <my-bucket>
  storageClassName: openshift-storage.noobaa.io
  additionalConfig:
    bucketclass: <my-bucket-class>
    Copy to Clipboard Toggle word wrap
    <resource-name>
    The name you want to give to the resource.
    <my-bucket>
    The name you want to give to the bucket.
    <my-bucket-class>
    The bucket class created in the previous step.

After the OBC is provisioned by the operator, a bucket is created in the MCG, and the operator creates a Secret and ConfigMap with the same name and in the same namespace as that of the OBC.

Prerequisites

Procedure

  1. Create a secret with the credentials: 

    apiVersion: v1
kind: Secret
metadata:
  name: <namespacestore-secret-name>
  type: Opaque
data:
  IBM_COS_ACCESS_KEY_ID: <IBM COS ACCESS KEY ID ENCODED IN BASE64>
  IBM_COS_SECRET_ACCESS_KEY: <IBM COS SECRET ACCESS KEY ENCODED IN BASE64>
    Copy to Clipboard Toggle word wrap
    <namespacestore-secret-name>

    A unique NamespaceStore name.

    You must provide and encode your own IBM COS access key ID and secret access key using Base64, and use the results in place of <IBM COS ACCESS KEY ID ENCODED IN BASE64> and <IBM COS SECRET ACCESS KEY ENCODED IN BASE64>.

  2. Create a NamespaceStore resource using OpenShift custom resource definitions (CRDs).

    A NamespaceStore represents underlying storage to be used as a read or write target for the data in the MCG namespace buckets.

    To create a NamespaceStore resource, apply the following YAML: 

    apiVersion: noobaa.io/v1alpha1
kind: NamespaceStore
metadata:
  finalizers:
  - noobaa.io/finalizer
  labels:
    app: noobaa
  name: bs
  namespace: openshift-storage
spec:
  s3Compatible:
    endpoint: <IBM COS ENDPOINT>
    secret:
      name: <namespacestore-secret-name>
      namespace: <namespace-secret>
    signatureVersion: v2
    targetBucket: <target-bucket>
  type: ibm-cos
    Copy to Clipboard Toggle word wrap
    <IBM COS ENDPOINT>
    The appropriate IBM COS endpoint.
    <namespacestore-secret-name>
    The secret created in the previous step.
    <namespace-secret>
    The namespace where the secret can be found.
    <target-bucket>
    The target bucket you created for the NamespaceStore.

  3. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • The namespace policy of type single requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type: Single
    single:
      resource: <resource>
      Copy to Clipboard Toggle word wrap
      <my-bucket-class>
      The unique namespace bucket class name.
      <resource>
      The name of a single NamespaceStore that defines the read and write target of the namespace bucket.

    • The namespace policy of type multi requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type: Multi
    multi:
      writeResource: <write-resource>
      readResources:
      - <read-resources>
      - <read-resources>
      Copy to Clipboard Toggle word wrap
      <my-bucket-class>
      The unique bucket class name.
      <write-resource>
      The name of a single NamespaceStore that defines the write target of the namespace bucket.
      <read-resources>
      A list of the NamespaceStores names that defines the read targets of the namespace bucket.

  4. To create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in the previous step, apply the following YAML: 

    apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: <resource-name>
  namespace: openshift-storage
spec:
  generateBucketName: <my-bucket>
  storageClassName: openshift-storage.noobaa.io
  additionalConfig:
    bucketclass: <my-bucket-class>
    Copy to Clipboard Toggle word wrap
    <resource-name>
    The name you want to give to the resource.
    <my-bucket>
    The name you want to give to the bucket.
    <my-bucket-class>

    The bucket class created in the previous step.

    After the OBC is provisioned by the operator, a bucket is created in the MCG, and the operator creates a Secret and ConfigMap with the same name and in the same namespace as that of the OBC.

Prerequisites

# subscription-manager repos --enable=rh-odf-4-for-rhel-8-x86_64-rpms
# yum install mcg
Copy to Clipboard Toggle word wrap
Note

Specify the appropriate architecture for enabling the repositories using subscription manager. For instance, in case of IBM Z use the following command: 

# subscription-manager repos --enable=rh-odf-4-for-rhel-8-s390x-rpms
Copy to Clipboard Toggle word wrap

Alternatively, you can install the MCG package from the OpenShift Data Foundation RPMs found here https://access.redhat.com/downloads/content/547/ver=4/rhel---8/4/x86_64/package.

Note

Choose the correct Product Variant according to your architecture.

Procedure

  1. In the MCG command-line interface, create a NamespaceStore resource.

    A NamespaceStore represents an underlying storage to be used as a read or write target for the data in MCG namespace buckets. 

    $ noobaa namespacestore create aws-s3 <namespacestore> --access-key <AWS ACCESS KEY> --secret-key <AWS SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
    Copy to Clipboard Toggle word wrap
    <namespacestore>
    The name of the NamespaceStore.
    <AWS ACCESS KEY> and <AWS SECRET ACCESS KEY>
    The AWS access key ID and secret access key you created for this purpose.
    <bucket-name>
    The existing AWS bucket name. This argument tells the MCG which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.

  2. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy can be either single or multi.

    • To create a namespace bucket class with a namespace policy of type single: 

      $ noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage
      Copy to Clipboard Toggle word wrap
      <resource-name>
      The name you want to give the resource.
      <my-bucket-class>
      A unique bucket class name.
      <resource>
      A single namespace-store that defines the read and write target of the namespace bucket.

    • To create a namespace bucket class with a namespace policy of type multi: 

      $ noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage
      Copy to Clipboard Toggle word wrap
      <resource-name>
      The name you want to give the resource.
      <my-bucket-class>
      A unique bucket class name.
      <write-resource>
      A single namespace-store that defines the write target of the namespace bucket.
      <read-resources>s
      A list of namespace-stores separated by commas that defines the read targets of the namespace bucket.

  3. Create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in the previous step. 

    $ noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
    Copy to Clipboard Toggle word wrap
    <bucket-name>
    A bucket name of your choice.
    <custom-bucket-class>
    The name of the bucket class created in the previous step.

    After the OBC is provisioned by the operator, a bucket is created in the MCG, and the operator creates a Secret and a ConfigMap with the same name and in the same namespace as that of the OBC.

Prerequisites

  • Openshift Container Platform with OpenShift Data Foundation operator installed.
  • Access to the Multicloud Object Gateway (MCG), see Chapter 2, Accessing the Multicloud Object Gateway with your applications.

  • Download the MCG command-line interface: 

    # subscription-manager repos --enable=rh-odf-4-for-rhel-8-x86_64-rpms
# yum install mcg
    Copy to Clipboard Toggle word wrap
    Note

    Specify the appropriate architecture for enabling the repositories using subscription manager.

    • For IBM Power, use the following command: 
    # subscription-manager repos --enable=rh-odf-4-for-rhel-8-ppc64le-rpms
    Copy to Clipboard Toggle word wrap
    • For IBM Z, use the following command: 
    # subscription-manager repos --enable=rh-odf-4-for-rhel-8-s390x-rpms
    Copy to Clipboard Toggle word wrap

    Alternatively, you can install the MCG package from the OpenShift Data Foundation RPMs found here https://access.redhat.com/downloads/content/547/ver=4/rhel---8/4/x86_64/package.

    Note

    Choose the correct Product Variant according to your architecture.

Procedure

  1. In the MCG command-line interface, create a NamespaceStore resource.

    A NamespaceStore represents an underlying storage to be used as a read or write target for the data in the MCG namespace buckets. 

    $ noobaa namespacestore create ibm-cos <namespacestore> --endpoint <IBM COS ENDPOINT> --access-key <IBM ACCESS KEY> --secret-key <IBM SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
    Copy to Clipboard Toggle word wrap
    <namespacestore>
    The name of the NamespaceStore.
    <IBM ACCESS KEY>, <IBM SECRET ACCESS KEY>, <IBM COS ENDPOINT>
    An IBM access key ID, secret access key, and the appropriate regional endpoint that corresponds to the location of the existing IBM bucket.
    <bucket-name>
    An existing IBM bucket name. This argument tells the MCG which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.

  2. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • To create a namespace bucket class with a namespace policy of type single: 

      $ noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage
      Copy to Clipboard Toggle word wrap
      <resource-name>
      The name you want to give the resource.
      <my-bucket-class>
      A unique bucket class name.
      <resource>
      A single NamespaceStore that defines the read and write target of the namespace bucket.

    • To create a namespace bucket class with a namespace policy of type multi: 

      $ noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage
      Copy to Clipboard Toggle word wrap
      <resource-name>
      The name you want to give the resource.
      <my-bucket-class>
      A unique bucket class name.
      <write-resource>
      A single NamespaceStore that defines the write target of the namespace bucket.
      <read-resources>
      A comma-separated list of NamespaceStores that defines the read targets of the namespace bucket.

  3. Create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in the earlier step. 

    $ noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
    Copy to Clipboard Toggle word wrap
    <bucket-name>
    A bucket name of your choice.
    <custom-bucket-class>
    The name of the bucket class created in the previous step.

After the OBC is provisioned by the operator, a bucket is created in the MCG, and the operator creates a Secret and ConfigMap with the same name and in the same namespace as that of the OBC.

You can add namespace buckets using the OpenShift Container Platform user interface. For information about namespace buckets, see Managing namespace buckets.

Prerequisites

  • Ensure that Openshift Container Platform with OpenShift Data Foundation operator is already installed.
  • Access to the Multicloud Object Gateway (MCG).

Procedure

  1. On the OpenShift Web Console, navigate to Storage Object Storage Namespace Store tab.

  2. Click Create namespace store to create a namespacestore resources to be used in the namespace bucket.

    1. Enter a namespacestore name.
    2. Choose a provider and region.
    3. Either select an existing secret, or click Switch to credentials to create a secret by entering a secret key and secret access key.
    4. Enter a target bucket.
    5. Click Create.
  3. On the Namespace Store tab, verify that the newly created namespacestore is in the Ready state.
  4. Repeat steps 2 and 3 until you have created all the desired amount of resources.

  5. Navigate to Bucket Class tab and click Create Bucket Class.

    1. Choose Namespace BucketClass type radio button.
    2. Enter a BucketClass name and click Next.

    3. Choose a Namespace Policy Type for your namespace bucket, and then click Next.

      • If your namespace policy type is Single, you need to choose a read resource.
      • If your namespace policy type is Multi, you need to choose read resources and a write resource.
      • If your namespace policy type is Cache, you need to choose a Hub namespace store that defines the read and write target of the namespace bucket.
    4. Select one Read and Write NamespaceStore which defines the read and write targets of the namespace bucket and click Next.
    5. Review your new bucket class details, and then click Create Bucket Class.
  6. Navigate to Bucket Class tab and verify that your newly created resource is in the Ready phase.

  7. Navigate to Object Bucket Claims tab and click Create Object Bucket Claim.

    1. Enter ObjectBucketClaim Name for the namespace bucket.
    2. Select StorageClass as openshift-storage.noobaa.io.
    3. Select the BucketClass that you created earlier for your namespacestore from the list. By default, noobaa-default-bucket-class gets selected.
    4. Click Create. The namespace bucket is created along with Object Bucket Claim for your namespace.
  8. Navigate to Object Bucket Claims tab and verify that the Object Bucket Claim created is in Bound state.
  9. Navigate to Object Buckets tab and verify that the your namespace bucket is present in the list and is in Bound state.

Many legacy applications use file systems to share data sets. You can access and share the legacy data in the file system by using the S3 operations. To share data you need to do the following:

  • Export the pre-existing file system datasets, that is, RWX volume such as Ceph FileSystem (CephFS) or create a new file system datasets using the S3 protocol.
  • Access file system datasets from both file system and S3 protocol.
  • Configure S3 accounts and map them to the existing or a new file system unique identifiers (UIDs) and group identifiers (GIDs).

Prerequisites

  • Openshift Container Platform with OpenShift Data Foundation operator installed.
  • Access to the Multicloud Object Gateway (MCG).

Procedure

  1. Log into the OpenShift Web Console.
  2. Click Storage Object Storage.
  3. Click the NamespaceStore tab to create NamespaceStore resources to be used in the namespace bucket.
  4. Click Create namespacestore.
  5. Enter a name for the NamespaceStore.
  6. Choose Filesystem as the provider.
  7. Choose the Persistent volume claim.

  8. Enter a folder name.

    If the folder name exists, then that folder is used to create the NamespaceStore or else a folder with that name is created.

  9. Click Create.
  10. Verify the NamespaceStore is in the Ready state.

You can either create a new account with NamespaceStore filesystem configuration or convert an existing normal account into a NamespaceStore filesystem account by editing the YAML.

Note

You cannot remove a NamespaceStore filesystem configuration from an account.

Prerequisites

  • Download the Multicloud Object Gateway (MCG) command-line interface: 

    # subscription-manager repos --enable=rh-odf-4-for-rhel-8-x86_64-rpms
# yum install mcg
    Copy to Clipboard Toggle word wrap

Procedure

  • Create a new account with NamespaceStore filesystem configuration using the MCG command-line interface. 

    $ noobaa account create <noobaa-account-name> [flags]
    Copy to Clipboard Toggle word wrap

    For example: 

    $ noobaa account create testaccount --full_permission --nsfs_account_config --gid 10001 --uid 10001 –default_resource fs_namespacestore
    Copy to Clipboard Toggle word wrap

    allow_bucket_create

    Indicates whether the account is allowed to create new buckets. Supported values are true or false. Default value is true.

    allowed_buckets

    A comma separated list of bucket names to which the user is allowed to have access and management rights.

    default_resource

    The NamespaceStore resource on which the new buckets will be created when using the S3 CreateBucket operation. The NamespaceStore must be backed by an RWX (ReadWriteMany) persistent volume claim (PVC).

    full_permission

    Indicates whether the account should be allowed full permission or not. Supported values are true or false. Default value is false.

    new_buckets_path

    The filesystem path where directories corresponding to new buckets will be created. The path is inside the filesystem of NamespaceStore filesystem PVCs where new directories are created to act as the filesystem mapping of newly created object bucket classes.

    nsfs_account_config

    A mandatory field that indicates if the account is used for NamespaceStore filesystem.

    nsfs_only

    Indicates whether the account is used only for NamespaceStore filesystem or not. Supported values are true or false. Default value is false. If it is set to 'true', it limits you from accessing other types of buckets.

    uid

    The user ID of the filesystem to which the MCG account will be mapped and it is used to access and manage data on the filesystem

    gid

    The group ID of the filesystem to which the MCG account will be mapped and it is used to access and manage data on the filesystem

    The MCG system sends a response with the account configuration and its S3 credentials: 

    # NooBaaAccount spec:
allow_bucket_creation: true
Allowed_buckets:
  full_permission: true
  permission_list: []
default_resource: noobaa-default-namespace-store
Nsfs_account_config:
  gid: 10001
  new_buckets_path: /
  nsfs_only: true
  uid: 10001
INFO[0006] ✅ Exists: Secret "noobaa-account-testaccount"
Connection info:
  AWS_ACCESS_KEY_ID      : <aws-access-key-id>
  AWS_SECRET_ACCESS_KEY  : <aws-secret-access-key>
    Copy to Clipboard Toggle word wrap

    You can list all the custom resource definition (CRD) based accounts by using the following command: 

    $ noobaa account list
NAME          ALLOWED_BUCKETS   DEFAULT_RESOURCE               PHASE   AGE
testaccount   [*]               noobaa-default-backing-store   Ready   1m17s
    Copy to Clipboard Toggle word wrap

    If you are interested in a particular account, you can read its custom resource definition (CRD) directly by the account name: 

    $ oc get noobaaaccount/testaccount -o yaml
spec:
  allow_bucket_creation: true
  allowed_buckets:
    full_permission: true
    permission_list: []
  default_resource: noobaa-default-namespace-store
  nsfs_account_config:
    gid: 10001
    new_buckets_path: /
    nsfs_only: true
    uid: 10001
    Copy to Clipboard Toggle word wrap

When using the Multicloud Object Gateway (MCG) NamespaceStore filesystem (NSFS) feature, you need to have the Persistent Volume Claim (PVC) where the data resides in the openshift-storage namespace. In almost all cases, the data you need to access is not in the openshift-storage namespace, but in the namespace that the legacy application uses.

In order to access data stored in another namespace, you need to create a PVC in the openshift-storage namespace that points to the same CephFS volume that the legacy application uses.

Procedure

  1. Display the application namespace with scc: 

    $ oc get ns <application_namespace> -o yaml | grep scc
    Copy to Clipboard Toggle word wrap
    <application_namespace>

    Specify the name of the application namespace.

    For example: 

    $ oc get ns testnamespace -o yaml | grep scc

openshift.io/sa.scc.mcs: s0:c26,c5
openshift.io/sa.scc.supplemental-groups: 1000660000/10000
openshift.io/sa.scc.uid-range: 1000660000/10000
    Copy to Clipboard Toggle word wrap

  2. Navigate into the application namespace: 

    $ oc project <application_namespace>
    Copy to Clipboard Toggle word wrap

    For example: 

    $ oc project testnamespace
    Copy to Clipboard Toggle word wrap

  3. Ensure that a ReadWriteMany (RWX) PVC is mounted on the pod that you want to consume from the noobaa S3 endpoint using the MCG NSFS feature: 

    $ oc get pvc

NAME                                               STATUS VOLUME
CAPACITY ACCESS MODES STORAGECLASS              AGE
cephfs-write-workload-generator-no-cache-pv-claim  Bound  pvc-aa58fb91-c3d2-475b-bbee-68452a613e1a
10Gi     RWX          ocs-storagecluster-cephfs 12s
    Copy to Clipboard Toggle word wrap 
    $ oc get pod

NAME                                                READY   STATUS              RESTARTS   AGE
cephfs-write-workload-generator-no-cache-1-cv892    1/1     Running             0          11s
    Copy to Clipboard Toggle word wrap

  4. Check the mount point of the Persistent Volume (PV) inside your pod.

    1. Get the volume name of the PV from the pod: 

      $ oc get pods <pod_name> -o jsonpath='{.spec.volumes[]}'
      Copy to Clipboard Toggle word wrap
      <pod_name>

      Specify the name of the pod.

      For example: 

      $ oc get pods cephfs-write-workload-generator-no-cache-1-cv892 -o jsonpath='{.spec.volumes[]}'

{"name":"app-persistent-storage","persistentVolumeClaim":{"claimName":"cephfs-write-workload-generator-no-cache-pv-claim"}}
      Copy to Clipboard Toggle word wrap

      In this example, the name of the volume for the PVC is cephfs-write-workload-generator-no-cache-pv-claim.

    2. List all the mounts in the pod, and check for the mount point of the volume that you identified in the previous step: 

      $ oc get pods <pod_name> -o jsonpath='{.spec.containers[].volumeMounts}'
      Copy to Clipboard Toggle word wrap

      For example: 

      $ oc get pods cephfs-write-workload-generator-no-cache-1-cv892 -o jsonpath='{.spec.containers[].volumeMounts}'

[{"mountPath":"/mnt/pv","name":"app-persistent-storage"},{"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount","name":"kube-api-access-8tnc5","readOnly":true}]
      Copy to Clipboard Toggle word wrap

  5. Confirm the mount point of the RWX PV in your pod: 

    $ oc exec -it <pod_name> -- df <mount_path>
    Copy to Clipboard Toggle word wrap
    <mount_path>

    Specify the path to the mount point that you identified in the previous step.

    For example: 

    $ oc exec -it cephfs-write-workload-generator-no-cache-1-cv892 -- df /mnt/pv

main
Filesystem
1K-blocks Used Available  Use%  Mounted on
172.30.202.87:6789,172.30.120.254:6789,172.30.77.247:6789:/volumes/csi/csi-vol-cc416d9e-dbf3-11ec-b286-0a580a810213/edcfe4d5-bdcb-4b8e-8824-8a03ad94d67c
10485760  0    10485760   0%    /mnt/pv
    Copy to Clipboard Toggle word wrap

  6. Ensure that the UID and SELinux labels are the same as the ones that the legacy namespace uses: 

    $ oc exec -it <pod_name> -- ls -latrZ <mount_path>
    Copy to Clipboard Toggle word wrap

    For example: 

    $ oc exec -it cephfs-write-workload-generator-no-cache-1-cv892 -- ls -latrZ /mnt/pv/

total 567
drwxrwxrwx. 3 root       root system_u:object_r:container_file_t:s0:c26,c5      2 May 25 06:35 .
-rw-r--r--. 1 1000660000 root system_u:object_r:container_file_t:s0:c26,c5 580138 May 25 06:35 fs_write_cephfs-write-workload-generator-no-cache-1-cv892-data.log
drwxrwxrwx. 3 root       root system_u:object_r:container_file_t:s0:c26,c5     30 May 25 06:35 ..
    Copy to Clipboard Toggle word wrap

  7. Get the information of the legacy application RWX PV that you want to make accessible from the openshift-storage namespace: 

    $ oc get pv | grep <pv_name>
    Copy to Clipboard Toggle word wrap
    <pv_name>

    Specify the name of the PV.

    For example: 

    $ oc get pv | grep pvc-aa58fb91-c3d2-475b-bbee-68452a613e1a

pvc-aa58fb91-c3d2-475b-bbee-68452a613e1a   10Gi       RWX            Delete           Bound    testnamespace/cephfs-write-workload-generator-no-cache-pv-claim   ocs-storagecluster-cephfs              47s
    Copy to Clipboard Toggle word wrap

  8. Ensure that the PVC from the legacy application is accessible from the openshift-storage namespace so that one or more noobaa-endpoint pods can access the PVC.

    1. Find the values of the subvolumePath and volumeHandle from the volumeAttributes. You can get these values from the YAML description of the legacy application PV: 

      $ oc get pv <pv_name> -o yaml
      Copy to Clipboard Toggle word wrap

      For example: 

      $ oc get pv pvc-aa58fb91-c3d2-475b-bbee-68452a613e1a -o yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/provisioned-by: openshift-storage.cephfs.csi.ceph.com
  creationTimestamp: "2022-05-25T06:27:49Z"
  finalizers:
  - kubernetes.io/pv-protection
  name: pvc-aa58fb91-c3d2-475b-bbee-68452a613e1a
  resourceVersion: "177458"
  uid: 683fa87b-5192-4ccf-af2f-68c6bcf8f500
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: cephfs-write-workload-generator-no-cache-pv-claim
    namespace: testnamespace
    resourceVersion: "177453"
    uid: aa58fb91-c3d2-475b-bbee-68452a613e1a
  csi:
    controllerExpandSecretRef:
      name: rook-csi-cephfs-provisioner
      namespace: openshift-storage
    driver: openshift-storage.cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: rook-csi-cephfs-node
      namespace: openshift-storage
    volumeAttributes:
      clusterID: openshift-storage
      fsName: ocs-storagecluster-cephfilesystem
      storage.kubernetes.io/csiProvisionerIdentity: 1653458225664-8081-openshift-storage.cephfs.csi.ceph.com
      subvolumeName: csi-vol-cc416d9e-dbf3-11ec-b286-0a580a810213
      subvolumePath: /volumes/csi/csi-vol-cc416d9e-dbf3-11ec-b286-0a580a810213/edcfe4d5-bdcb-4b8e-8824-8a03ad94d67c
    volumeHandle: 0001-0011-openshift-storage-0000000000000001-cc416d9e-dbf3-11ec-b286-0a580a810213
  persistentVolumeReclaimPolicy: Delete
  storageClassName: ocs-storagecluster-cephfs
  volumeMode: Filesystem
status:
  phase: Bound
      Copy to Clipboard Toggle word wrap

    2. Use the subvolumePath and volumeHandle values that you identified in the previous step to create a new PV and PVC object in the openshift-storage namespace that points to the same CephFS volume as the legacy application PV:

      Example YAML file: 

      $ cat << EOF >> pv-openshift-storage.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: cephfs-pv-legacy-openshift-storage
spec:
  storageClassName: ""
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
      1 
      
  csi:
    driver: openshift-storage.cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: rook-csi-cephfs-node
      namespace: openshift-storage
    volumeAttributes:
    # Volume Attributes can be copied from the Source testnamespace PV
      "clusterID": "openshift-storage"
      "fsName": "ocs-storagecluster-cephfilesystem"
      "staticVolume": "true"
    # rootpath is the subvolumePath: you copied from the Source testnamespace PV
      "rootPath": /volumes/csi/csi-vol-cc416d9e-dbf3-11ec-b286-0a580a810213/edcfe4d5-bdcb-4b8e-8824-8a03ad94d67c
    volumeHandle: 0001-0011-openshift-storage-0000000000000001-cc416d9e-dbf3-11ec-b286-0a580a810213-clone
      2 
      
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: cephfs-pvc-legacy
  namespace: openshift-storage
spec:
  storageClassName: ""
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
      3 
      
  volumeMode: Filesystem
  # volumeName should be same as PV name
  volumeName: cephfs-pv-legacy-openshift-storage
EOF
      Copy to Clipboard Toggle word wrap
      1
      The storage capacity of the PV that you are creating in the openshift-storage namespace must be the same as the original PV.
      2
      The volume handle for the target PV that you create in openshift-storage needs to have a different handle than the original application PV, for example, add -clone at the end of the volume handle.
      3
      The storage capacity of the PVC that you are creating in the openshift-storage namespace must be the same as the original PVC.

    3. Create the PV and PVC in the openshift-storage namespace using the YAML file specified in the previous step: 

      $ oc create -f <YAML_file>
      Copy to Clipboard Toggle word wrap
      <YAML_file>

      Specify the name of the YAML file.

      For example: 

      $ oc create -f pv-openshift-storage.yaml

persistentvolume/cephfs-pv-legacy-openshift-storage created
persistentvolumeclaim/cephfs-pvc-legacy created
      Copy to Clipboard Toggle word wrap

    4. Ensure that the PVC is available in the openshift-storage namespace: 

      $ oc get pvc -n openshift-storage

NAME                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                  AGE
cephfs-pvc-legacy                     Bound    cephfs-pv-legacy-openshift-storage         10Gi       RWX                                          14s
      Copy to Clipboard Toggle word wrap

    5. Navigate into the openshift-storage project: 

      $ oc project openshift-storage

Now using project "openshift-storage" on server "https://api.cluster-5f6ng.5f6ng.sandbox65.opentlc.com:6443".
      Copy to Clipboard Toggle word wrap

    6. Create the NSFS namespacestore: 

      $ noobaa namespacestore create nsfs <nsfs_namespacestore> --pvc-name='<cephfs_pvc_name>' --fs-backend='CEPH_FS'
      Copy to Clipboard Toggle word wrap
      <nsfs_namespacestore>
      Specify the name of the NSFS namespacestore.
      <cephfs_pvc_name>

      Specify the name of the CephFS PVC in the openshift-storage namespace.

      For example: 

      $ noobaa namespacestore create nsfs legacy-namespace --pvc-name='cephfs-pvc-legacy' --fs-backend='CEPH_FS'
      Copy to Clipboard Toggle word wrap

    7. Ensure that the noobaa-endpoint pod restarts and that it successfully mounts the PVC at the NSFS namespacestore, for example, /nsfs/legacy-namespace mountpoint: 

      $ oc exec -it <noobaa_endpoint_pod_name> -- df -h /nsfs/<nsfs_namespacestore>
      Copy to Clipboard Toggle word wrap
      <noobaa_endpoint_pod_name>

      Specify the name of the noobaa-endpoint pod.

      For example: 

      $ oc exec -it noobaa-endpoint-5875f467f5-546c6 -- df -h /nsfs/legacy-namespace

Filesystem                                                                                                                                                Size  Used Avail Use% Mounted on
172.30.202.87:6789,172.30.120.254:6789,172.30.77.247:6789:/volumes/csi/csi-vol-cc416d9e-dbf3-11ec-b286-0a580a810213/edcfe4d5-bdcb-4b8e-8824-8a03ad94d67c   10G     0   10G   0% /nsfs/legacy-namespace
      Copy to Clipboard Toggle word wrap

    8. Create a MCG user account: 

      $ noobaa account create <user_account> --full_permission --allow_bucket_create=true --new_buckets_path='/' --nsfs_only=true --nsfs_account_config=true --gid <gid_number> --uid <uid_number> --default_resource='legacy-namespace'
      Copy to Clipboard Toggle word wrap
      <user_account>
      Specify the name of the MCG user account.
      <gid_number>
      Specify the GID number.
      <uid_number>

      Specify the UID number.

      Important

      Use the same UID and GID as that of the legacy application. You can find it from the previous output.

      For example: 

      $ noobaa account create leguser --full_permission --allow_bucket_create=true --new_buckets_path='/' --nsfs_only=true --nsfs_account_config=true --gid 0 --uid 1000660000 --default_resource='legacy-namespace'
      Copy to Clipboard Toggle word wrap

    9. Create a MCG bucket.

      1. Create a dedicated folder for S3 inside the NSFS share on the CephFS PV and PVC of the legacy application pod: 

        $ oc exec -it <pod_name> -- mkdir <mount_path>/nsfs
        Copy to Clipboard Toggle word wrap

        For example: 

        $ oc exec -it cephfs-write-workload-generator-no-cache-1-cv892 -- mkdir /mnt/pv/nsfs
        Copy to Clipboard Toggle word wrap

      2. Create the MCG bucket using the nsfs/ path: 

        $ noobaa api bucket_api create_bucket '{
  "name": "<bucket_name>",
  "namespace":{
    "write_resource": { "resource": "<nsfs_namespacestore>", "path": "nsfs/" },
    "read_resources": [ { "resource": "<nsfs_namespacestore>", "path": "nsfs/" }]
  }
}'
        Copy to Clipboard Toggle word wrap

        For example: 

        $ noobaa api bucket_api create_bucket '{
  "name": "legacy-bucket",
  "namespace":{
    "write_resource": { "resource": "legacy-namespace", "path": "nsfs/" },
    "read_resources": [ { "resource": "legacy-namespace", "path": "nsfs/" }]
  }
}'
        Copy to Clipboard Toggle word wrap

    10. Check the SELinux labels of the folders residing in the PVCs in the legacy application and openshift-storage namespaces: 

      $ oc exec -it <noobaa_endpoint_pod_name> -n openshift-storage -- ls -ltraZ /nsfs/<nsfs_namespacstore>
      Copy to Clipboard Toggle word wrap

      For example: 

      $ oc exec -it noobaa-endpoint-5875f467f5-546c6 -n openshift-storage -- ls -ltraZ /nsfs/legacy-namespace

total 567
drwxrwxrwx. 3 root       root system_u:object_r:container_file_t:s0:c0,c26      2 May 25 06:35 .
-rw-r--r--. 1 1000660000 root system_u:object_r:container_file_t:s0:c0,c26 580138 May 25 06:35 fs_write_cephfs-write-workload-generator-no-cache-1-cv892-data.log
drwxrwxrwx. 3 root       root system_u:object_r:container_file_t:s0:c0,c26     30 May 25 06:35 ..
      Copy to Clipboard Toggle word wrap 
      $ oc exec -it <pod_name> -- ls -latrZ <mount_path>
      Copy to Clipboard Toggle word wrap

      For example: 

      $ oc exec -it cephfs-write-workload-generator-no-cache-1-cv892 -- ls -latrZ /mnt/pv/

total 567
drwxrwxrwx. 3 root       root system_u:object_r:container_file_t:s0:c26,c5      2 May 25 06:35 .
-rw-r--r--. 1 1000660000 root system_u:object_r:container_file_t:s0:c26,c5 580138 May 25 06:35 fs_write_cephfs-write-workload-generator-no-cache-1-cv892-data.log
drwxrwxrwx. 3 root       root system_u:object_r:container_file_t:s0:c26,c5     30 May 25 06:35 ..
      Copy to Clipboard Toggle word wrap

      In these examples, you can see that the SELinux labels are not the same which results in permission denied or access issues.

  9. Ensure that the legacy application and openshift-storage pods use the same SELinux labels on the files.

    You can do this in one of the following ways:

  10. Delete the NSFS namespacestore:

    1. Delete the MCG bucket: 

      $ noobaa bucket delete <bucket_name>
      Copy to Clipboard Toggle word wrap

      For example: 

      $ noobaa bucket delete legacy-bucket
      Copy to Clipboard Toggle word wrap

    2. Delete the MCG user account: 

      $ noobaa account delete <user_account>
      Copy to Clipboard Toggle word wrap

      For example: 

      $ noobaa account delete leguser
      Copy to Clipboard Toggle word wrap

    3. Delete the NSFS namespacestore: 

      $ noobaa namespacestore delete <nsfs_namespacestore>
      Copy to Clipboard Toggle word wrap

      For example: 

      $ noobaa namespacestore delete legacy-namespace
      Copy to Clipboard Toggle word wrap

  11. Delete the PV and PVC:

    Important

    Before you delete the PV and PVC, ensure that the PV has a retain policy configured. 

    $ oc delete pv <cephfs_pv_name>
    Copy to Clipboard Toggle word wrap 
    $ oc delete pvc <cephfs_pvc_name>
    Copy to Clipboard Toggle word wrap
    <cephfs_pv_name>
    Specify the CephFS PV name of the legacy application.
    <cephfs_pvc_name>

    Specify the CephFS PVC name of the legacy application.

    For example: 

    $ oc delete pv cephfs-pv-legacy-openshift-storage
    Copy to Clipboard Toggle word wrap 
    $ oc delete pvc cephfs-pvc-legacy
    Copy to Clipboard Toggle word wrap

  1. Display the current openshift-storage namespace with sa.scc.mcs: 

    $ oc get ns openshift-storage -o yaml | grep sa.scc.mcs

openshift.io/sa.scc.mcs: s0:c26,c0
    Copy to Clipboard Toggle word wrap

  2. Edit the legacy application namespace, and modify the sa.scc.mcs with the value from the sa.scc.mcs of the openshift-storage namespace: 

    $ oc edit ns <appplication_namespace>
    Copy to Clipboard Toggle word wrap

    For example: 

    $ oc edit ns testnamespace
    Copy to Clipboard Toggle word wrap 
    $ oc get ns <application_namespace> -o yaml | grep sa.scc.mcs
    Copy to Clipboard Toggle word wrap

    For example: 

    $ oc get ns testnamespace -o yaml | grep sa.scc.mcs

openshift.io/sa.scc.mcs: s0:c26,c0
    Copy to Clipboard Toggle word wrap
  3. Restart the legacy application pod. A relabel of all the files take place and now the SELinux labels match with the openshift-storage deployment.

  1. Create a new scc with the MustRunAs and seLinuxOptions options, with the Multi Category Security (MCS) that the openshift-storage project uses.

    Example YAML file: 

    $ cat << EOF >> scc.yaml
allowHostDirVolumePlugin: false
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegeEscalation: true
allowPrivilegedContainer: false
allowedCapabilities: null
apiVersion: security.openshift.io/v1
defaultAddCapabilities: null
fsGroup:
  type: MustRunAs
groups:
- system:authenticated
kind: SecurityContextConstraints
metadata:
  annotations:
  name: restricted-pvselinux
priority: null
readOnlyRootFilesystem: false
requiredDropCapabilities:
- KILL
- MKNOD
- SETUID
- SETGID
runAsUser:
  type: MustRunAsRange
seLinuxContext:
  seLinuxOptions:
    level: s0:c26,c0
  type: MustRunAs
supplementalGroups:
  type: RunAsAny
users: []
volumes:
- configMap
- downwardAPI
- emptyDir
- persistentVolumeClaim
- projected
- secret
EOF
    Copy to Clipboard Toggle word wrap 
    $ oc create -f scc.yaml
    Copy to Clipboard Toggle word wrap

  2. Create a service account for the deployment and add it to the newly created scc.

    1. Create a service account: 

      $ oc create serviceaccount <service_account_name>
      Copy to Clipboard Toggle word wrap
      <service_account_name>`

      Specify the name of the service account.

      For example: 

      $ oc create serviceaccount testnamespacesa
      Copy to Clipboard Toggle word wrap

    2. Add the service account to the newly created scc: 

      $ oc adm policy add-scc-to-user restricted-pvselinux -z <service_account_name>
      Copy to Clipboard Toggle word wrap

      For example: 

      $ oc adm policy add-scc-to-user restricted-pvselinux -z testnamespacesa
      Copy to Clipboard Toggle word wrap

  3. Patch the legacy application deployment so that it uses the newly created service account. This allows you to specify the SELinux label in the deployment: 

    $ oc patch dc/<pod_name> '{"spec":{"template":{"spec":{"serviceAccountName": "<service_account_name>"}}}}'
    Copy to Clipboard Toggle word wrap

    For example: 

    $ oc patch dc/cephfs-write-workload-generator-no-cache --patch '{"spec":{"template":{"spec":{"serviceAccountName": "testnamespacesa"}}}}'
    Copy to Clipboard Toggle word wrap

  4. Edit the deployment to specify the security context to use at the SELinux label in the deployment configuration: 

    $ oc edit dc <pod_name> -n <application_namespace>
    Copy to Clipboard Toggle word wrap

    Add the following lines: 

    spec:
 template:
    metadata:
      securityContext:
        seLinuxOptions:
          Level: <security_context_value>
    Copy to Clipboard Toggle word wrap
    <security_context_value>

    You can find this value when you execute the command to create a dedicated folder for S3 inside the NSFS share, on the CephFS PV and PVC of the legacy application pod.

    For example: 

    $ oc edit dc cephfs-write-workload-generator-no-cache -n testnamespace
    Copy to Clipboard Toggle word wrap 
    spec:
 template:
    metadata:
      securityContext:
        seLinuxOptions:
          level: s0:c26,c0
    Copy to Clipboard Toggle word wrap

  5. Ensure that the security context to be used at the SELinux label in the deployment configuration is specified correctly: 

    $ oc get dc <pod_name> -n <application_namespace> -o yaml | grep -A 2 securityContext
    Copy to Clipboard Toggle word wrap

    For example" 

    $ oc get dc cephfs-write-workload-generator-no-cache -n testnamespace -o yaml | grep -A 2 securityContext

      securityContext:
        seLinuxOptions:
          level: s0:c26,c0
    Copy to Clipboard Toggle word wrap

    The legacy application is restarted and begins using the same SELinux labels as the openshift-storage namespace.

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat