10.3. Configuring an NVMe/RDMA host

Procedure

1. Install the nvme-cli tool:

   # dnf install nvme-cli

2. Load the nvme-rdma module if it is not loaded:

   # modprobe nvme-rdma

3. Discover available subsystems on the NVMe controller:

   # nvme discover -t rdma -a 172.31.0.202 -s 4420
   
   Discovery Log Number of Records 2, Generation counter 2
   =====Discovery Log Entry 0======
   trtype: rdma
   adrfam: ipv4
   subtype: current discovery subsystem
   treq: not specified, sq flow control disable supported
   portid: 2
   trsvcid: 4420
   subnqn: nqn.2014-08.org.nvmexpress.discovery
   traddr: 172.31.0.202
   eflags: none
   rdma_prtype: not specified
   rdma_qptype: connected
   rdma_cms: rdma-cm
   rdma_pkey: 0000
   =====Discovery Log Entry 1======
   trtype: rdma
   adrfam: ipv4
   subtype: nvme subsystem
   treq: not specified, sq flow control disable supported
   portid: 2
   trsvcid: 4420
   subnqn: testnqn
   traddr: 172.31.0.202
   eflags: none
   rdma_prtype: not specified
   rdma_qptype: connected
   rdma_cms: rdma-cm
   rdma_pkey: 0000

4. Connect to the discovered subsystems:

   # nvme connect -t rdma -a 172.31.0.202 -s 4420 -n testnqn
   connecting to device: nvme0
   
   # lsblk
   NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
   sda                            8:0    0 465.8G  0 disk
   ├─sda1                         8:1    0     1G  0 part /boot
   └─sda2                         8:2    0 464.8G  0 part
     ├─rhel_rdma--virt--03-root 253:0    0    50G  0 lvm  /
     ├─rhel_rdma--virt--03-swap 253:1    0     4G  0 lvm  [SWAP]
     └─rhel_rdma--virt--03-home 253:2    0 410.8G  0 lvm  /home
   nvme0n1
   
   # cat /sys/class/nvme/nvme0/transport
   rdma

   Replace testnqn with the NVMe subsystem name.

   Replace 172.31.0.202 with the controller IP address.

   Replace 4420 with the port number.