Chapter 6. Enabling NFS over RDMA on an NFS server

Procedure

1. Install the rdma-core package:

   # dnf install rdma-core

   Copy to Clipboard Toggle word wrap

2. If the package was already installed, verify that the xprtrdma and svcrdma modules in the /etc/rdma/modules/rdma.conf file are uncommented:

   # NFS over RDMA client support
   xprtrdma
   # NFS over RDMA server support
   svcrdma

   Copy to Clipboard Toggle word wrap

3. Optional: By default, NFS over RDMA uses port 20049. If you want to use a different port, set the rdma-port setting in the [nfsd] section of the /etc/nfs.conf file:

   rdma-port=<port>

   Copy to Clipboard Toggle word wrap

4. Open the NFSoRDMA port in firewalld:

   # firewall-cmd --permanent --add-port={20049/tcp,20049/udp}
   # firewall-cmd --reload

   Copy to Clipboard Toggle word wrap

   Adjust the port numbers if you set a different port than 20049.

5. Restart the nfs-server service:

   # systemctl restart nfs-server

   Copy to Clipboard Toggle word wrap

Verification

1. On a client with InfiniBand hardware, perform the following steps:

   1. Install the following packages:

      # dnf install nfs-utils rdma-core

      Copy to Clipboard Toggle word wrap

   2. Mount an exported NFS share over RDMA:

      # mount -o rdma server.example.com:/nfs/projects/ /mnt/

      Copy to Clipboard Toggle word wrap

      If you set a port number other than the default (20049), pass port=<port_number> to the command:

      # mount -o rdma,port=<port_number> server.example.com:/nfs/projects/ /mnt/

      Copy to Clipboard Toggle word wrap

   3. Verify that the share was mounted with the rdma option:

      # mount | grep "/mnt"
      server.example.com:/nfs/projects/ on /mnt type nfs (...,proto=rdma,...)

      Copy to Clipboard Toggle word wrap