Configure a network bond

A network bond aggregates physical and virtual network interfaces into a single logical one. The kernel exclusively handles all operations. You can create bonds on different device types, such as Ethernet or VLANs, for higher throughput or redundancy.

Red Hat Enterprise Linux provides administrators different options to configure bond devices. For example:

  • Use nmcli to configure bond connections using the command line.
  • Use the RHEL web console to configure bond connections using a web browser.
  • Use nmtui to configure bond connections in a text-based user interface.
  • Use nmstatectl to configure bond connections through the Nmstate API.
  • Use RHEL system roles to automate the bond configuration on one or multiple hosts.

Default behavior of controller and port interfaces
Copy link

Understanding the default behavior of NetworkManager when managing bond port interfaces helps you to troubleshoot problems more effectively.

Default behavior:

  • Starting the controller interface does not automatically start the port interfaces.
  • Starting a port interface always starts the controller interface.
  • Stopping the controller interface also stops the port interface.
  • A controller without ports can start static IP connections.
  • A controller without ports waits for ports when starting DHCP connections.
  • A controller with a DHCP connection waiting for ports completes when you add a port with a carrier.
  • A controller with a DHCP connection waiting for ports continues waiting when you add a port without a carrier.

Upstream switch configuration depending on the bonding modes
Copy link

Depending on the bonding mode you want to use, you must configure the ports on the switch.

Bonding mode Configuration on the switch

0 - balance-rr

Requires static EtherChannel enabled, not Link Aggregation Control Protocol (LACP)-negotiated.

1 - active-backup

No configuration required on the switch.

2 - balance-xor

Requires static EtherChannel enabled, not LACP-negotiated.

3 - broadcast

Requires static EtherChannel enabled, not LACP-negotiated.

4 - 802.3ad

Requires LACP-negotiated EtherChannel enabled.

5 - balance-tlb

No configuration required on the switch.

6 - balance-alb

No configuration required on the switch.

balance-slb

No configuration required on the switch.

For details on how to configure your switch, see the documentation of the switch.

Important

Certain network bonding features, such as the fail-over mechanism, do not support direct cable connections without a network switch.

Configure a network bond by using nmcli
Copy link

To configure a network bond on the command line, use the nmcli utility.

Before you begin
Copy link

  • Two or more physical or virtual network devices are installed on the server.
  • To use Ethernet devices as ports of the bond, the physical or virtual Ethernet devices must be installed on the server.
  • To use bridge or Virtual Local Area Network (VLAN) devices as ports of the bond, you can either create these devices while you create the bond or you can create them in advance as described in:

Procedure
Copy link

  1. Create a bond interface: 
    # nmcli connection add type bond con-name bond0 ifname bond0 bond.options "mode=active-backup"

    This command creates a bond named bond0 that uses the active-backup mode.

    To additionally set a Media Independent Interface (MII) monitoring interval, add the miimon=interval option to the bond.options property, for example:

    # nmcli connection add type bond con-name bond0 ifname bond0 bond.options "mode=active-backup,miimon=1000"
  2. Display the network interfaces, and note names of interfaces you plan to add to the bond: 
    # nmcli device status
     
    DEVICE   TYPE      STATE         CONNECTION
enp7s0   ethernet  disconnected  --
enp8s0   ethernet  disconnected  --
bridge0  bridge    connected     bridge0
bridge1  bridge    connected     bridge1
...

    In this example:

    • enp7s0 and enp8s0 are not configured. To use these devices as ports, add connection profiles in the next step.
    • bridge0 and bridge1 have existing connection profiles. To use these devices as ports, modify their profiles in the next step.
  3. Assign interfaces to the bond:
    1. If the interfaces you want to assign to the bond are not configured, create new connection profiles for them: 
      # nmcli connection add type ethernet port-type bond con-name bond0-port1 ifname enp7s0 controller bond0
# nmcli connection add type ethernet port-type bond con-name bond0-port2 ifname enp8s0 controller bond0

      These commands create profiles for enp7s0 and enp8s0, and add them to the bond0 connection.

    2. To assign an existing connection profile to the bond:
      1. Set the controller parameter of these connections to bond0: 
        # nmcli connection modify bridge0 controller bond0
# nmcli connection modify bridge1 controller bond0

        These commands assign the existing connection profiles named bridge0 and bridge1 to the bond0 connection.

      2. Reactivate the connections: 
        # nmcli connection up bridge0
# nmcli connection up bridge1
  4. Configure the IPv4 settings:
    • To set a static IPv4 address, network mask, default gateway, and DNS server to the bond0 connection, enter: 
      # nmcli connection modify bond0 ipv4.addresses '192.0.2.1/24' ipv4.gateway '192.0.2.254' ipv4.dns '192.0.2.253' ipv4.dns-search 'example.com' ipv4.method manual
    • To use DHCP, no action is required.
    • If you plan to use this bond device as a port of other devices, no action is required.
  5. Configure the IPv6 settings:
    • To set a static IPv6 address, network mask, default gateway, and DNS server to the bond0 connection, enter: 
      # nmcli connection modify bond0 ipv6.addresses '2001:db8:1::1/64' ipv6.gateway '2001:db8:1::fffe' ipv6.dns '2001:db8:1::fffd' ipv6.dns-search 'example.com' ipv6.method manual
    • To use stateless address autoconfiguration (SLAAC), no action is required.
    • If you plan to use this bond device as a port of other devices, no action is required.
  6. Optional: If you want to set any parameters on the bond ports, use the following command: 
    # nmcli connection modify bond0-port1 bond-port.<parameter><value>
  7. Activate the connection: 
    # nmcli connection up bond0
  8. Verify that the ports are connected, and the CONNECTION column displays the port’s connection name: 
    # nmcli device
     
    DEVICE   TYPE      STATE      CONNECTION
...
enp7s0   ethernet  connected  bond0-port1
enp8s0   ethernet  connected  bond0-port2

    When you activate any port of the connection, NetworkManager also activates the bond, but not the other ports of it. You can configure that Red Hat Enterprise Linux enables all ports automatically when the bond is enabled:

    1. Enable the connection.autoconnect-ports parameter of the bond’s connection: 
      # nmcli connection modify bond0 connection.autoconnect-ports 1
    2. Reactivate the bridge: 
      # nmcli connection up bond0

Results
Copy link

  1. Temporarily remove the network cable from one of the network devices and check if the other device in the bond is handling the traffic.

    Note that there is no method to properly test link failure events using software utilities. Tools that deactivate connections, such as nmcli, show only the bonding driver’s ability to handle port configuration changes and not actual link failure events.

  2. Display the status of the bond: 
    # cat /proc/net/bonding/bond0

Configure a network bond by using the RHEL web console
Copy link

Use the RHEL web console to configure a network bond if you prefer to manage network settings using a web browser-based interface.

Before you begin
Copy link

Procedure
Copy link

  1. Select the Networking tab in the navigation on the left side of the screen.
  2. Click Add bond in the Interfaces section.
  3. Enter the name of the bond device you want to create.
  4. Select the interfaces that should be members of the bond.
  5. Select the mode of the bond.

    If you select Active backup, the web console shows the additional field Primary in which you can select the preferred active device.

  6. Set the link monitoring mode. For example, when you use the Adaptive load balancing mode, set it to ARP.
  7. Optional: Adjust the monitoring interval, link up delay, and link down delay settings. Typically, you only change the defaults for troubleshooting purposes.
  8. Click Apply.
  9. By default, the bond uses a dynamic IP address. If you want to set a static IP address:
    1. Click the name of the bond in the Interfaces section.
    2. Click Edit next to the protocol you want to configure.
    3. Select Manual next to Addresses, and enter the IP address, prefix, and default gateway.
    4. In the DNS section, click the + button, and enter the IP address of the DNS server. Repeat this step to set multiple DNS servers.
    5. In the DNS search domains section, click the + button, and enter the search domain.
    6. If the interface requires static routes, configure them in the Routes section.
    7. Click Apply

Results
Copy link

  1. Select the Networking tab in the navigation on the left side of the screen, and check if there is incoming and outgoing traffic on the interface.
  2. Temporarily remove the network cable from one of the network devices and check if the other device in the bond is handling the traffic.

    Note that there is no method to properly test link failure events using software utilities. Tools that deactivate connections, such as the web console, show only the bonding driver’s ability to handle member configuration changes and not actual link failure events.

  3. Display the status of the bond: 
    # cat /proc/net/bonding/bond0

Configure a network bond by using nmtui
Copy link

The nmtui application provides a text-based user interface for NetworkManager. You can use nmtui to configure a network bond on a host without a graphical interface.

Before you begin
Copy link

  • Two or more physical or virtual network devices are installed on the server.
  • To use Ethernet devices as ports of the bond, the physical or virtual Ethernet devices must be installed on the server.

About this task
Copy link

Note

In nmtui:

  • Navigate by using the cursor keys.
  • Press a button by selecting it and hitting Enter.
  • Select and clear checkboxes by using Space.
  • To return to the previous screen, use ESC.

Procedure
Copy link

  1. If you do not know the network device names on which you want configure a network bond, display the available devices: 
    # nmcli device status
     
    DEVICE     TYPE      STATE                   CONNECTION
enp7s0     ethernet  unavailable             --
enp8s0     ethernet  unavailable             --
...
  2. Start nmtui: 
    # nmtui
  3. Select Edit a connection, and press Enter.
  4. Press Add.
  5. Select Bond from the list of network types, and press Enter.
  6. Optional: Enter a name for the NetworkManager profile to be created.

    On hosts with multiple profiles, a meaningful name makes it easier to identify the purpose of a profile.

  7. Enter the bond device name to be created into the Device field.
  8. Add ports to the bond to be created:
    1. Press Add next to the Ports list.
    2. Select the type of the interface you want to add as a port to the bond, for example, Ethernet.
    3. Optional: Enter a name for the NetworkManager profile to be created for this bond port.
    4. Enter the port’s device name into the Device field.
    5. Press OK to return to the window with the bond settings.
    6. Repeat these steps to add more ports to the bond.
  9. Set the bond mode. Depending on the value you set, nmtui displays additional fields for settings that are related to the selected mode.
  10. Depending on your environment, configure the IP address settings in the IPv4 configuration and IPv6 configuration areas accordingly. For this, press the button next to these areas, and select:
    • Disabled, if the bond does not require an IP address.
    • Automatic, if a DHCP server or stateless address autoconfiguration (SLAAC) dynamically assigns an IP address to the bond.
    • Manual, if the network requires static IP address settings. In this case, you must fill further fields:
      1. Press Show next to the protocol you want to configure to display additional fields.
      2. Press Add next to Addresses, and enter the IP address and the subnet mask in Classless Inter-Domain Routing (CIDR) format.

        If you do not specify a subnet mask, NetworkManager sets a /32 subnet mask for IPv4 addresses and /64 for IPv6 addresses.

      3. Enter the address of the default gateway.
      4. Press Add next to DNS servers, and enter the DNS server address.
      5. Press Add next to Search domains, and enter the DNS search domain.
  11. Press OK to create and automatically activate the new connection.
  12. Press Back to return to the main menu.
  13. Select Quit, and press Enter to close the nmtui application.

Results
Copy link

  1. Temporarily remove the network cable from one of the network devices and check if the other device in the bond is handling the traffic.

    Note that there is no method to properly test link failure events using software utilities. Tools that deactivate connections, such as nmcli, show only the bonding driver’s ability to handle port configuration changes and not actual link failure events.

  2. Display the status of the bond: 
    # cat /proc/net/bonding/bond0

Configure a network bond by using nmstatectl
Copy link

You can use the declarative Nmstate API to configure a network bond. Nmstate ensures that the result matches the configuration file or rolls back the changes.

Before you begin
Copy link

  • Two or more physical or virtual network devices are installed on the server.
  • To use Ethernet devices as ports in the bond, the physical or virtual Ethernet devices must be installed on the server.
  • To use bridge or Virtual Local Area Network (VLAN) devices as ports in the bond, set the interface name in the port list, and define the corresponding interfaces.
  • The nmstate package is installed.

About this task
Copy link

Depending on your environment, adjust the YAML file accordingly. For example, to use different devices than Ethernet adapters in the bond, adapt the base-iface attribute and type attributes of the ports you use in the bond.

Procedure
Copy link

  1. Create a YAML file, for example ~/create-bond.yml, with the following content: 
    ---
interfaces:
- name: bond0
  type: bond
  state: up
  ipv4:
    enabled: true
    address:
    - ip: 192.0.2.1
      prefix-length: 24
    dhcp: false
  ipv6:
    enabled: true
    address:
    - ip: 2001:db8:1::1
      prefix-length: 64
    autoconf: false
    dhcp: false
  link-aggregation:
    mode: active-backup
    port:
    - enp1s0
    - enp7s0
- name: enp1s0
  type: ethernet
  state: up
- name: enp7s0
  type: ethernet
  state: up

routes:
  config:
  - destination: 0.0.0.0/0
    next-hop-address: 192.0.2.254
    next-hop-interface: bond0
    metric: 300
  - destination: ::/0
    next-hop-address: 2001:db8:1::fffe
    next-hop-interface: bond0
    metric: 300

dns-resolver:
  config:
    search:
    - example.com
    server:
    - 192.0.2.200
    - 2001:db8:1::ffbb

    These settings define a network bond with the following settings:

    • Network interfaces in the bond: enp1s0 and enp7s0
    • Mode: active-backup
    • Static IPv4 address: 192.0.2.1 with a /24 subnet mask
    • Static IPv6 address: 2001:db8:1::1 with a /64 subnet mask
    • IPv4 default gateway: 192.0.2.254
    • IPv6 default gateway: 2001:db8:1::fffe
    • IPv4 DNS server: 192.0.2.200
    • IPv6 DNS server: 2001:db8:1::ffbb
    • DNS search domain: example.com
  2. Apply the settings to the system: 
    # nmstatectl apply ~/create-bond.yml

Results
Copy link

  1. Display the status of the devices and connections: 
    # nmcli device status
     
    DEVICE      TYPE      STATE      CONNECTION
bond0       bond      connected  bond0
  2. Display all settings of the connection profile: 
    # nmcli connection show bond0
     
    connection.id:              bond0
connection.uuid:            79cbc3bd-302e-4b1f-ad89-f12533b818ee
connection.stable-id:       --
connection.type:            bond
connection.interface-name:  bond0
...
  3. Display the connection settings in YAML format: 
    # nmstatectl show bond0

Configure a network bond by using the network RHEL system role
Copy link

You can use the network RHEL system role to configure a network bond and, if a connection profile for the bond’s parent device does not exist, the role can create it as well.

Before you begin
Copy link

  • You have prepared the control node and the managed nodes.
  • You are logged in to the control node as a user who can run playbooks on the managed nodes.
  • The account you use to connect to the managed nodes has sudo permissions for these nodes.
  • Two or more physical or virtual network devices are installed on the server.

About this task
Copy link

You can combine network interfaces in a bond to provide a logical interface with higher throughput or redundancy. To configure a bond, create a NetworkManager connection profile. By using Ansible and the network RHEL system role, you can automate this process and remotely configure connection profiles on the hosts defined in a playbook.

Procedure
Copy link

  1. Create a playbook file, for example, ~/playbook.yml, with the following content: 
    ---
- name: Configure the network
  hosts: managed-node-01.example.com
  tasks:
    - name: Bond connection profile with two Ethernet ports
      ansible.builtin.include_role:
        name: redhat.rhel_system_roles.network
      vars:
        network_connections:
          # Bond profile
          - name: bond0
            type: bond
            interface_name: bond0
            ip:
              dhcp4: yes
              auto6: yes
            bond:
              mode: active-backup
            state: up

          # Port profile for the 1st Ethernet device
          - name: bond0-port1
            interface_name: enp7s0
            type: ethernet
            controller: bond0
            state: up

          # Port profile for the 2nd Ethernet device
          - name: bond0-port2
            interface_name: enp8s0
            type: ethernet
            controller: bond0
            state: up

    The settings specified in the example playbook include the following:

    type: <profile_type>
    Sets the type of the profile to create. The example playbook creates three connection profiles: One for the bond and two for the Ethernet devices.
    dhcp4: yes
    Enables automatic IPv4 address assignment from DHCP, PPP, or similar services.
    auto6: yes
    Enables IPv6 auto-configuration. By default, NetworkManager uses Router Advertisements. If the router announces the managed flag, NetworkManager requests an IPv6 address and prefix from a DHCPv6 server.
    mode: <bond_mode>

    Sets the bonding mode. Possible values are:

    • balance-rr (default)
    • active-backup
    • balance-xor
    • broadcast
    • 802.3ad
    • balance-tlb
    • balance-alb.

    Depending on the mode you set, you need to set additional variables in the playbook.

    For details about all variables used in the playbook, see the /usr/share/ansible/roles/rhel-system-roles.network/README.md file on the control node.

  2. Validate the playbook syntax: 
    $ ansible-playbook --syntax-check ~/playbook.yml

    Note that this command only validates the syntax and does not protect against a wrong but valid configuration.

  3. Run the playbook: 
    $ ansible-playbook ~/playbook.yml

Results
Copy link

  • Temporarily remove the network cable from one of the network devices and check if the other device in the bond is handling the traffic.

    Note that there is no method to properly test link failure events using software utilities. Tools that deactivate connections, such as nmcli, show only the bonding driver’s ability to handle port configuration changes and not actual link failure events.

The different network bonding modes
Copy link

Bonding aggregates multiple network interfaces into a single logical one. The behavior of a bonded interface depends on its bonding policy, also known as mode, which provides either load-balancing or hot standby services.

The Linux bonding driver supports the following modes:

Balance-rr (Mode 0)

Balance-rr uses the round-robin algorithm that sequentially transmits packets from the first available port to the last one. This mode provides load balancing and fault tolerance.

This mode requires switch configuration of a port aggregation group, also called EtherChannel or similar port grouping. An EtherChannel is a port link aggregation technology to group multiple physical Ethernet links to one logical Ethernet link.

The drawback of this mode is that it is not suitable for heavy workloads and if TCP throughput or ordered packet delivery is essential.

Active-backup (Mode 1)

Active-backup uses the policy that determines that only one port is active in the bond. This mode provides fault tolerance and does not require any switch configuration.

If the active port fails, an alternate port becomes active. The bond sends a gratuitous address resolution protocol (ARP) response to the network. The gratuitous ARP forces the receiver of the ARP frame to update their forwarding table. The Active-backup mode transmits a gratuitous ARP to announce the new path to maintain connectivity for the host.

The primary option defines the preferred port of the bonding interface.

Balance-xor (Mode 2)

Balance-xor uses the selected transmit hash policy to send the packets. This mode provides load balancing, fault tolerance, and requires switch configuration to configure an Etherchannel or similar port grouping.

To alter packet transmission and balance transmit, this mode uses the xmit_hash_policy option. Depending on the source or destination of traffic on the interface, the interface requires an additional load-balancing configuration. See description xmit_hash_policy bonding parameter.

Broadcast (Mode 3)

Broadcast uses a policy that transmits every packet on all interfaces. This mode provides fault tolerance and requires a switch configuration to configure an EtherChannel or similar port grouping.

The drawback of this mode is that it is not suitable for heavy workloads and if TCP throughput or ordered packet delivery is essential.

802.3ad (Mode 4)

802.3ad uses the same-named IEEE standard dynamic link aggregation policy. This mode provides fault tolerance. This mode requires switch configuration to configure a Link Aggregation Control Protocol (LACP) port grouping.

This mode creates aggregation groups that share the same speed and duplex settings and uses all ports in the active aggregator. Depending on the source or destination of traffic on the interface, this mode requires an additional load-balancing configuration.

By default, the port selection for outgoing traffic depends on the transmit hash policy. Use the xmit_hash_policy option of the transmit hash policy to change the port selection and balance transmit.

The difference between the 802.3ad and the Balance-xor is compliance. The 802.3ad policy negotiates LACP between the port aggregation groups. See description xmit_hash_policy bonding parameter

Balance-tlb (Mode 5)

Balance-tlb uses the transmit load balancing policy. This mode provides fault tolerance, load balancing, and establishes channel bonding that does not require any switch support.

The active port receives the incoming traffic. In case of failure of the active port, another one takes over the MAC address of the failed port. To decide which interface processes the outgoing traffic, use one of the following modes:

  • Value 0: Uses the hash distribution policy to distribute traffic without load balancing
  • Value 1: Distributes traffic to each port by using load balancing

    With the bonding option tlb_dynamic_lb=0, this bonding mode uses the xmit_hash_policy bonding option to balance transmit. The primary option defines the preferred port of the bonding interface.

See description xmit_hash_policy bonding parameter.

Balance-alb (Mode 6)

Balance-alb uses an adaptive load balancing policy. This mode provides fault tolerance, load balancing, and does not require any special switch support.

This mode Includes balance-transmit load balancing (balance-tlb) and receive-load balancing for IPv4 and IPv6 traffic. The bonding intercepts ARP replies sent by the local system and overwrites the source hardware address of one of the ports in the bond. ARP negotiation manages the receive-load balancing. Therefore, different ports use different hardware addresses for the server.

The primary option defines the preferred port of the bonding interface. With the bonding option tlb_dynamic_lb=0, this bonding mode uses the xmit_hash_policy bonding option to balance transmit. See description xmit_hash_policy bonding parameter.

Additionally, you can use NetworkManager to configure the following mode:

Balance-slb

The source load balancing (SLB) bonding mode distributes outgoing data streams across multiple network interfaces based on the source address of the traffic and a Virtual Local Area Network (VLAN) hash. This mode does not require any switch configuration.

NetworkManager uses the balance-xor mode in combination with nftables rules to provide SLB. For details about configuring this mode, see Red Hat Knowledgebase solution Configuring a network bond on RHEL with source load balancing.

The xmit_hash_policy bonding parameter
Copy link

The xmit_hash_policy load balancing parameter selects the transmit hash policy for a node selection in the balance-xor, 802.3ad, balance-alb, and balance-tlb modes. It is only applicable to mode 5 and 6 if the tlb_dynamic_lb parameter is 0.

The possible values of tlb_dynamic_lb are layer2, layer2+3, layer3+4, encap2+3, encap3+4, and vlan+srcmac.

Refer the table for details:

Policy or Network layers

Layer2

Layer2+3

Layer3+4

encap2+3

encap3+4

VLAN+srcmac

Uses

XOR of source and destination MAC addresses and Ethernet protocol type

XOR of source and destination MAC addresses and IP addresses

XOR of source and destination ports and IP addresses

XOR of source and destination MAC addresses and IP addresses inside a supported tunnel, for example, Virtual Extensible LAN (VXLAN). This mode relies on skb_flow_dissect() function to obtain the header fields

XOR of source and destination ports and IP addresses inside a supported tunnel, for example, VXLAN. This mode relies on skb_flow_dissect() function to obtain the header fields

XOR of VLAN ID and source MAC vendor and source MAC device

Placement of traffic

All traffic to a particular network peer on the same underlying network interface

All traffic to a particular IP address on the same underlying network interface

All traffic to a particular IP address and port on the same underlying network interface

Primary choice

If network traffic is between this system and multiple other systems in the same broadcast domain

If network traffic between this system and multiple other systems goes through a default gateway

If network traffic between this system and another system uses the same IP addresses but goes through multiple ports

The encapsulated traffic is between the source system and multiple other systems using multiple IP addresses

The encapsulated traffic is between the source system and other systems using multiple port numbers

If the bond carries network traffic, from multiple containers or virtual machines (VM), that expose their MAC address directly to the external network such as the bridge network, and you cannot configure a switch for Mode 2 or Mode 4

Secondary choice

If network traffic is mostly between this system and multiple other systems behind a default gateway

If network traffic is mostly between this system and another system

Compliant

802.3ad

802.3ad

Not 802.3ad

Default policy

This is the default policy if no configuration is provided

For non-IP traffic, the formula is the same as for the layer2 transmit policy

For non-IP traffic, the formula is the same as for the layer2 transmit policy