Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 6. NFS cluster and export management

As a storage administrator, you can create an NFS cluster, customize it, and export Ceph File System namespace over the NFS protocol.

6.1. Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • Installation and configuration of the Ceph Metadata Server daemons (ceph-mds).
  • Create and mount a Ceph File System.

6.2. Creating an NFS cluster

Create an NFS cluster with the nfs cluster create command. This creates a common recovery pool for all NFS Ganesa daemons, new user based on the cluster name, and a common NFS Ganesha configuration RADOS object.

Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • An existing Ceph File System.
  • Root-level access to Ceph Monitor.
  • Installation of the nfs-ganesha, nfs-ganesha-ceph, nfs-ganesha-rados-grace, and nfs-ganesha-rados-urls packages on the Ceph Manager hosts.
  • Root-level access to the client.

Procedure

  1. Log into the Cephadm shell:

    Example

    [root@mds ~]# cephadm shell
    Copy to Clipboard

  2. Enable the Ceph Manager NFS module:

    Example

    [ceph: root@host01 /]# ceph mgr module enable nfs
    Copy to Clipboard

  3. Create an NFS Ganesha cluster:

    Syntax

    ceph nfs cluster create CLUSTER_NAME [PLACEMENT] [--ingress] [--virtual_ip IP_ADDRESS] [--ingress-mode {default|keepalive-only|haproxy-standard|haproxy-protocol}] [--port PORT]
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs cluster create nfs-cephfs "host01 host02"
NFS Cluster Created Successfully
    Copy to Clipboard

    In this example, the NFS Ganesha cluster name is nfs-cephfs and the daemon containers are deployed to host01, and host02.

    Important

    Red Hat only supports one NFS Ganesha daemon running per host.

  4. Verify the NFS Ganesha cluster information:

    Syntax

    ceph nfs cluster info [CLUSTER_NAME]
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs cluster info nfs-cephfs
{
    "nfs-cephfs": [
        {
            "hostname": "host01",
            "ip": "10.74.179.124",
            "port": 2049
        },
        {
            "hostname": "host02",
            "ip": "10.74.180.160",
            "port": 2049
        }
    ]
}
    Copy to Clipboard

    Note

    Specifying the CLUSTER_NAME is optional.

6.3. Customizing an NFS configuration

Customize an NFS cluster with the configuration file. With this, the NFS cluster uses the specified configuration and has precedence over the default configuration blocks.

Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • Root-level access to a Ceph Metadata Server (MDS) node.
  • An NFS cluster created using the ceph nfs cluster create command.

Procedure

  1. Create a configuration file:

    Example

    [ceph: root@host01 /]# touch nfs-cephfs.conf
    Copy to Clipboard

  2. Enable logging in the configuration file with the following block:

    Example

    [ceph: root@host01 /]# vi nfs-cephfs.conf

LOG {
 COMPONENTS {
     ALL = FULL_DEBUG;
 }
}
    Copy to Clipboard

  3. Set the new configuration:

    Syntax

    ceph nfs cluster config set CLUSTER_NAME -i PATH_TO_CONFIG_FILE
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs cluster config set nfs-cephfs -i nfs-cephfs.conf

NFS-Ganesha Config Set Successfully
    Copy to Clipboard

  4. View the customized NFS Ganesha configuration:

    Syntax

    ceph nfs cluster config get CLUSTER_NAME
    Copy to Clipboard

    Example

    [ceph: root@host01 /]#  ceph nfs cluster config get nfs-cephfs

LOG {
    COMPONENTS {
        ALL = FULL_DEBUG;
    }
}
    Copy to Clipboard

    This provides the output, if any, of the user-defined configuration.

  5. Optional: If you want to remove the user-defined configuration, run the following command:

    Syntax

    ceph nfs cluster config reset CLUSTER_NAME
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs cluster config reset nfs-cephfs

NFS-Ganesha Config Reset Successfully
    Copy to Clipboard

6.4. Exporting Ceph File System namespaces over the NFS protocol (Limited Availability)

Ceph File Systems (CephFS) namespaces can be exported over the NFS protocol using a NFS Ganesha file server. To export a CephFS namespace, you must first have a running NFS Ganesha cluster.

Note

This technology is Limited Availability. See the Deprecated functionality chapter for additional information.

Important

Red Hat supports only NFS version 4.0 or higher.

Important

NFS clients are unable to create CephFS snapshots through their native NFS mount. They must use server-side operator tooling for their snapshot needs.

Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • An NFS cluster created using the ceph nfs cluster create command.

Procedure

  1. Create a CephFS export:

    • For Red Hat Ceph Storage 5.0, create the export as follows:

      Syntax

      ceph nfs export create cephfs FILE_SYSTEM_NAME CLUSTER_NAME BINDING [--readonly] [--path=PATH_WITHIN_CEPHFS]
      Copy to Clipboard

      Example

      [ceph: root@host01 /]# ceph nfs export create cephfs cephfs01 nfs-cephfs /ceph  --path=/
      Copy to Clipboard

    • For Red Hat Ceph Storage 5.1 and later, create the export as follows:

      Syntax

      ceph nfs export create cephfs CLUSTER_NAME BINDING FILE_SYSTEM_NAME [--readonly] [--path=PATH_WITHIN_CEPHFS]
      Copy to Clipboard

      Example

      [ceph: root@host01 /]# ceph nfs export create cephfs nfs-cephfs /ceph cephfs01 --path=/
      Copy to Clipboard

      In this example, the BINDING (/ceph) is the pseudo root path, which must be unique, and an absolute path.

      Note

      The --readonly option exports a path with the read-only permission, the default being read, and write permissions.

      Note

      The PATH_WITHIN_CEPHFS can be a subvolume. You can get the absolute subvolume path by using the following command:

      Syntax

      ceph fs subvolume getpath VOLUME_NAME SUBVOLUME_NAME [--group_name SUBVOLUME_GROUP_NAME]
      Copy to Clipboard

      Example

      [ceph: root@host01 /]# ceph fs subvolume getpath cephfs sub0
      Copy to Clipboard

  2. View the export block based on the pseudo root name:

    Syntax

    ceph nfs export get CLUSTER_NAME BINDING
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export get nfs-cephfs /ceph
{
  "export_id": 1,
  "path": "/",
  "cluster_id": "nfs-cephfs",
  "pseudo": "/ceph",
  "access_type": "RW",
  "squash": "no_root_squash",
  "security_label": true,
  "protocols": [
    4
  ],
  "transports": [
    "TCP"
  ],
  "fsal": {
    "name": "CEPH",
    "user_id": "cephnfs11",
    "fs_name": "cephfs",
    "sec_label_xattr": ""
  },
  "clients": []
}
    Copy to Clipboard

  3. List the NFS exports:

    Syntax

    ceph nfs export ls CLUSTER_NAME [--detailed]
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export ls nfs-cephfs
[
  "/ceph/"
]
[ceph: root@host01 /]# ceph nfs export ls nfs-cephfs --detailed
[
  {
    "export_id": 100,
    "path": "/",
    "cluster_id": "nfs-cephfs",
    "pseudo": "/ceph/",
    "access_type": "RW",
    "squash": "no_root_squash",
    "security_label": true,
    "protocols": [
      4
    ],
    "transports": [
      "TCP"
    ],
    "fsal": {
      "name": "CEPH",
      "user_id": "nfstest01",
      "fs_name": "cephfs",
      "sec_label_xattr": ""
    },
    "clients": []
  }
]
    Copy to Clipboard

  4. Get the information of the NFS export:

    Syntax

    ceph nfs export info CLUSTER_NAME [PSEUDO_PATH]
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export info nfs-cephfs /ceph

{
  "export_id": 1,
  "path": "/",
  "cluster_id": "nfs-cephfs",
  "pseudo": "/ceph",
  "access_type": "RW",
  "squash": "none",
  "security_label": true,
  "protocols": [
    4
  ],
  "transports": [
    "TCP"
  ],
  "fsal": {
    "name": "CEPH",
    "user_id": "nfs.nfs-cephfs.1",
    "fs_name": "cephfs"
  },
  "clients": []
}
    Copy to Clipboard

  5. On a client host, mount the exported Ceph File System:

    Syntax

    mount -t nfs -o port=GANESHA_PORT HOST_NAME:BINDING LOCAL_MOUNT_POINT
    Copy to Clipboard

    Example

    [root@client01 ~]# mount -t nfs -o port=2049 host01:/ceph/ /mnt/nfs-cephfs
    Copy to Clipboard

    1. To automatically mount on boot, open and edit the /etc/fstab file by adding a new line:

      Syntax

      HOST_NAME:BINDING   LOCAL_MOUNT_POINT   nfs4   defaults,seclabel,vers=4.2,proto=tcp,port=2049  0  0
      Copy to Clipboard

      Example

      host01:/ceph/   /mnt/nfs-cephfs   nfs4   defaults,seclabel,vers=4.2,proto=tcp,port=2049  0  0
      Copy to Clipboard

  6. On a client host, to mount a exported NFS Ceph File System created with an ingress service:

    Syntax

    mount -t nfs VIRTUAL_IP_ADDRESS:BINDING LOCAL_MOUNT_POINT
    Copy to Clipboard

    • Replace VIRTUAL_IP_ADDRESS with --ingress --virtual-ip IP address used to create the NFS cluster.
    • Replace BINDING with the pseudo root path.

    • Replace LOCAL_MOUNT_POINT with the mount point to mount the export on.

      Example

      [root@client01 ~]# mount -t nfs 10.10.128.75:/nfs-cephfs /mnt
      Copy to Clipboard

      This example mounts the export nfs-cephfs that exists on a NFS cluster created with --ingress --virtual-ip 10.10.128.75 on the mount point /mnt.

6.5. Modifying the Ceph File System exports

You can modify the following parameters in an export with a configuration file:

  • access_type - This can be RW, RO, or NONE.
  • squash - This can be No_Root_Squash, None, or Root_Squash.
  • security_label - This can be true or false.

Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • An NFS export created.

Procedure

  1. View the export block based on the pseudo root name:

    Syntax

    ceph nfs export get CLUSTER_NAME BINDING
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export get nfs-cephfs /ceph
{
  "export_id": 1,
  "path": "/",
  "cluster_id": "nfs-cephfs",
  "pseudo": "/ceph",
  "access_type": "RO",
  "squash": "none",
  "security_label": true,
  "protocols": [
    4
  ],
  "transports": [
    "TCP"
  ],
  "fsal": {
    "name": "CEPH",
    "user_id": "cephnfs11",
    "fs_name": "cephfs",
    "sec_label_xattr": ""
  },
  "clients": []
}
    Copy to Clipboard

  2. Export the configuration file:

    Example

    [ceph: root@host01 /]# ceph nfs export get nfs-cephfs /ceph > export.conf
    Copy to Clipboard

  3. Edit the export information:

    Syntax

    {
  "export_id": EXPORT_ID,
  "path": "/",
  "cluster_id": "CLUSTER_NAME",
  "pseudo": "CLUSTER_PSEUDO_PATH",
  "access_type": "RW/RO",
  "squash": "SQUASH",
  "security_label": SECURITY_LABEL,
  "protocols": [
    PROTOCOL_ID_
  ],
  "transports": [
    "TCP"
  ],
  "fsal": {
    "name": "NAME",
    "user_id": "USER_ID",
    "fs_name": "FILE_SYSTEM_NAME",
    "sec_label_xattr": ""
  },
  "clients": []
}
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# vi export.conf

{
  "export_id": 1,
  "path": "/",
  "cluster_id": "nfs-cephfs",
  "pseudo": "/ceph",
  "access_type": "RW",
  "squash": "none",
  "security_label": true,
  "protocols": [
    4
  ],
  "transports": [
    "TCP"
  ],
  "fsal": {
    "name": "CEPH",
    "user_id": "cephnfs11",
    "fs_name": "cephfs",
    "sec_label_xattr": ""
  },
  "clients": []
}
    Copy to Clipboard

    In the above example, the access_type is modified from RO to RW.

  4. Apply the specification:

    Syntax

    ceph nfs export apply CLUSTER_NAME PATH_TO_EXPORT_FILE
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export apply nfs-cephfs -i export.conf
Added export /ceph
    Copy to Clipboard

  5. Get the updated export information:

    Syntax

    ceph nfs export get CLUSTER_NAME BINDING
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export get nfs-cephfs /ceph
{
  "export_id": 1,
  "path": "/",
  "cluster_id": "nfs-cephfs",
  "pseudo": "/ceph",
  "access_type": "RW",
  "squash": "none",
  "security_label": true,
  "protocols": [
    4
  ],
  "transports": [
    "TCP"
  ],
  "fsal": {
    "name": "CEPH",
    "user_id": "cephnfs11",
    "fs_name": "cephfs",
    "sec_label_xattr": ""
  },
  "clients": []
}
    Copy to Clipboard

6.6. Creating custom Ceph File System exports

You can customize the Ceph File System (CepFS) exports and apply the configuration.

Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • An NFS cluster created using the ceph nfs cluster create command.
  • A CephFS created.

Procedure

  1. Create a custom file:

    Example

    [ceph: root@host01 /]# touch export_new.conf
    Copy to Clipboard

  2. Create an export using the custom file:

    Syntax

    EXPORT {
  Export_Id = EXPORT_ID;
  Transports = TCP/UDP;
  Path = PATH;
  Pseudo = PSEUDO_PATH;
  Protocols = NFS_PROTOCOLS;
  Access_Type = ACCESS_TYPE;
  Attr_Expiration_Time = EXPIRATION_TIME;
  Squash = SQUASH;
  FSAL {
    Name = NAME;
    Filesystem = "CEPH_FILE_SYSTEM_NAME";
    User_Id = "USER_ID";
  }
}
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# cat export_new.conf
EXPORT {
  Export_Id = 2;
  Transports = TCP;
  Path = /;
  Pseudo = /ceph1/;
  Protocols = 4;
  Access_Type = RW;
  Attr_Expiration_Time = 0;
  Squash = None;
  FSAL {
    Name = CEPH;
    Filesystem = "cephfs";
    User_Id = "nfs.nfs-cephfs.2";
  }
}
    Copy to Clipboard

  3. Apply the specification:

    Syntax

    ceph nfs export apply CLUSTER_NAME -i PATH_TO_EXPORT_FILE
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export apply nfs-cephfs -i new_export.conf
Added export /ceph1
    Copy to Clipboard

  4. Get the updated export information:

    Syntax

    ceph nfs export get CLUSTER_NAME BINDING
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export get nfs-cephfs /ceph1
{
  "export_id": 1,
  "path": "/",
  "cluster_id": "nfs-cephfs",
  "pseudo": "/ceph1",
  "access_type": "RW",
  "squash": "None",
  "security_label": true,
  "protocols": [
    4
  ],
  "transports": [
    "TCP"
  ],
  "fsal": {
    "name": "CEPH",
    "user_id": "nfs.nfs-cephfs.2",
    "fs_name": "cephfs",
    "sec_label_xattr": ""
  },
  "clients": []
}
    Copy to Clipboard

6.7. Deleting Ceph File System exports

You can delete the Ceph File System (CephFS) NFS exports with the ceph export rm command.

Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • A CephFS created.

Procedure

  • Delete a CephFS export:

    Syntax

    ceph nfs export rm CLUSTER_NAME BINDING
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs export rm nfs-cephfs /ceph
    Copy to Clipboard

6.8. Deleting an NFS cluster

Delete an NFS cluster with the nfs cluster rm command. This deletes the deployed cluster. The removal of NFS daemons and the ingress service is asynchronous. Check the status of removal with ceph orch ls command.

Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • Root-level access to a Ceph Metadata Server (MDS) node.
  • NFS daemons deployed with ceph nfs cluster create command.

Procedure

  1. Log into the Cephadm shell:

    Example

    [root@mds ~]# cephadm shell
    Copy to Clipboard

  2. Remove an NFS Ganesha cluster:

    Syntax

    ceph nfs cluster rm CLUSTER_NAME
    Copy to Clipboard

    Example

    [ceph: root@host01 /]# ceph nfs cluster rm nfs-cephfs
    Copy to Clipboard

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat