Search

Chapter 2. Configuring the core RDMA subsystem

download PDF

The rdma service configuration manages the network protocols and communication standards such as InfiniBand, iWARP, and RoCE.

Procedure

  • Install the rdma-core package:

    # dnf install rdma-core

Verification

  1. Install the libibverbs-utils and infiniband-diags packages:

    # dnf install libibverbs-utils infiniband-diags
  2. List the available InfiniBand devices:

    # ibv_devices
    
        device                 node GUID
        ------              ----------------
        mlx4_0              0002c903003178f0
        mlx4_1              f4521403007bcba0
  3. Display the information of the mlx4_1 device:

    # ibv_devinfo -d mlx4_1
    
    hca_id: mlx4_1
         transport:                  InfiniBand (0)
         fw_ver:                     2.30.8000
         node_guid:                  f452:1403:007b:cba0
         sys_image_guid:             f452:1403:007b:cba3
         vendor_id:                  0x02c9
         vendor_part_id:             4099
         hw_ver:                     0x0
         board_id:                   MT_1090120019
         phys_port_cnt:              2
              port:   1
                    state:              PORT_ACTIVE (4)
                    max_mtu:            4096 (5)
                    active_mtu:         2048 (4)
                    sm_lid:             2
                    port_lid:           2
                    port_lmc:           0x01
                    link_layer:         InfiniBand
    
              port:   2
                    state:              PORT_ACTIVE (4)
                    max_mtu:            4096 (5)
                    active_mtu:         4096 (5)
                    sm_lid:             0
                    port_lid:           0
                    port_lmc:           0x00
                    link_layer:         Ethernet
  4. Display the status of the mlx4_1 device:

    # ibstat mlx4_1
    
    CA 'mlx4_1'
         CA type: MT4099
         Number of ports: 2
         Firmware version: 2.30.8000
         Hardware version: 0
         Node GUID: 0xf4521403007bcba0
         System image GUID: 0xf4521403007bcba3
         Port 1:
               State: Active
               Physical state: LinkUp
               Rate: 56
               Base lid: 2
               LMC: 1
               SM lid: 2
               Capability mask: 0x0251486a
               Port GUID: 0xf4521403007bcba1
               Link layer: InfiniBand
         Port 2:
               State: Active
               Physical state: LinkUp
               Rate: 40
               Base lid: 0
               LMC: 0
               SM lid: 0
               Capability mask: 0x04010000
               Port GUID: 0xf65214fffe7bcba2
               Link layer: Ethernet
  5. The ibping utility pings an InfiniBand address and runs as a client/server by configuring the parameters.

    1. Start server mode -S on port number -P with -C InfiniBand certificate authority (CA) name on the host:

      # ibping -S -C mlx4_1 -P 1
    2. Start client mode, send some packets -c on port number -P by using -C InfiniBand certificate authority (CA) name with -L Local Identifier (LID) on the host:

      # ibping -c 50 -C mlx4_0 -P 1 -L 2
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.