Chapter 2. Configuring the core RDMA subsystem
The rdma
service configuration manages the network protocols and communication standards such as InfiniBand, iWARP, and RoCE.
Procedure
Install the
rdma-core
package:# dnf install rdma-core
Verification
Install the
libibverbs-utils
andinfiniband-diags
packages:# dnf install libibverbs-utils infiniband-diags
List the available InfiniBand devices:
# ibv_devices device node GUID ------ ---------------- mlx4_0 0002c903003178f0 mlx4_1 f4521403007bcba0
Display the information of the
mlx4_1
device:# ibv_devinfo -d mlx4_1 hca_id: mlx4_1 transport: InfiniBand (0) fw_ver: 2.30.8000 node_guid: f452:1403:007b:cba0 sys_image_guid: f452:1403:007b:cba3 vendor_id: 0x02c9 vendor_part_id: 4099 hw_ver: 0x0 board_id: MT_1090120019 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 2048 (4) sm_lid: 2 port_lid: 2 port_lmc: 0x01 link_layer: InfiniBand port: 2 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet
Display the status of the
mlx4_1
device:# ibstat mlx4_1 CA 'mlx4_1' CA type: MT4099 Number of ports: 2 Firmware version: 2.30.8000 Hardware version: 0 Node GUID: 0xf4521403007bcba0 System image GUID: 0xf4521403007bcba3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 2 LMC: 1 SM lid: 2 Capability mask: 0x0251486a Port GUID: 0xf4521403007bcba1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 0x04010000 Port GUID: 0xf65214fffe7bcba2 Link layer: Ethernet
The
ibping
utility pings an InfiniBand address and runs as a client/server by configuring the parameters.Start server mode
-S
on port number-P
with-C
InfiniBand certificate authority (CA) name on the host:# ibping -S -C mlx4_1 -P 1
Start client mode, send some packets
-c
on port number-P
by using-C
InfiniBand certificate authority (CA) name with-L
Local Identifier (LID) on the host:# ibping -c 50 -C mlx4_0 -P 1 -L 2