Networking Guide
Configuring and managing networks, network interfaces, and network services in RHEL 7
Abstract
Note
Part I. Before You Begin
Chapter 1. Overview of Networking Topics
1.1. Comparing IP to non-IP Networks
Categories of Network Communication
- IP Networks
- Networks that communicate through Internet Protocol addresses. An IP network is implemented in the Internet and most internal networks. Ethernet, Cable Modems, DSL Modems, dial up modems, wireless networks, and VPN connections are typical examples.
- non-IP Networks
- Networks that are used to communicate through a lower layer rather than the transport layer. Note that these networks are rarely used. InfiniBand is a non-IP network, described in Chapter 13, Configure InfiniBand and RDMA Networks.
1.2. Comparing Static to Dynamic IP Addressing
- Static IP addressing
- When a device is assigned a static IP address, the address does not change over time unless changed manually. It is recommended to use static
IP
addressing if you want:- To ensure network address consistency for servers such as
DNS
, and authentication servers. - To use out-of-band management devices that work independently of other network infrastructure.
All the configuration tools listed in Section 3.1, “Selecting Network Configuration Methods” allow assigning staticIP
addresses manually. The nmcli tool is also suitable, described in Section 3.3.8, “Adding and Configuring a Static Ethernet Connection with nmcli” .For more information on automated configuration and management, see the OpenLMI chapter in the Red Hat Enterprise Linux 7 System Administrators Guide. The Red Hat Enterprise Linux 7 Installation Guide documents the use of a Kickstart file which can also be used for automating the assignment of network settings. - Dynamic IP addressing
- When a device is assigned a dynamic IP address, the address changes over time. For this reason, it is recommended for devices that connect to the network occasionally because IP address might be changed after rebooting the machine.Dynamic IP addresses are more flexible, easier to set up and administer. The dynamic host control protocol (DHCP) is a traditional method of dynamically assigning network configurations to hosts. See Section 14.1, “Why Use DHCP?” for more information. You can also use the nmcli tool, described in Section 3.3.7, “Adding and Configuring a Dynamic Ethernet Connection with nmcli” .
Note
There is no strict rule defining when to use static or dynamic IP address. It depends on user's needs, preferences and the network environment.By default, NetworkManager calls theDHCP
client, dhclient.
1.3. Configuring the DHCP Client Behavior
DHCP
client, dhclient by default.
Requesting an IP Address
DHCP
connection is started, a dhcp client requests an IP address from a DHCP
server. The time that a dhcp client waits for this request to be completed is 60 seconds by default. You can configure the ipv4.dhcp-timeout
property using the nmcli tool or the IPV4_DHCP_TIMEOUT
option in the /etc/sysconfig/network-scripts/ifcfg-ifname
file. For example, using nmcli:
~]# nmcli connection modify enp1s0 ipv4.dhcp-timeout 10
If an address cannot be obtained during this interval, the IPv4 configuration fails. The whole connection may fail, too, and this depends on the ipv4.may-fail
property:
- If
ipv4.may-fail
is set toyes
(default), the state of the connection depends on IPv6 configuration:- If the IPv6 configuration is enabled and successful, the connection is activated, but the IPv4 configuration can never be retried again.
- If the IPv6 configuration is disabled or does not get configured, the connection fails.
- If
ipv4.may-fail
is set tono
the connection is deactivated. In this case:- If the
autoconnect
property of the connection is enabled, NetworkManager retries to activate the connection as many times as set in theautoconnect-retries
property. The default is 4. - If the connection still cannot acquire the dhcp address, auto-activation fails.Note that after 5 minutes, the auto-connection process starts again and the dhcp client retries to acquire an address from the dhcp server.
Requesting a Lease Renewal
ipv4.dhcp-timeout
property in seconds (default is 60) to get the lease. If you get a reply during your attempts, the process stops and you get your lease renewed.
- If
ipv4.may-fail
is set toyes
(default) and IPv6 is successfully configured, the connection is activated and the dhcp client is restarted again every 2 minutes. - If
ipv4.may-fail
is set tono
, the connection is deactivated. In this case, if the connection has theautoconnect
property enabled, the connection is activated from scratch.
1.3.1. Making DHCPv4 Persistent
ipv4.dhcp-timeout
property either to the maximum for a 32-bit integer (MAXINT32), which is 2147483647
, or to the infinity
value:
~]$ nmcli connection modify enps1s0 ipv4.dhcp-timeout infinity
As a result, NetworkManager never stops trying to get or renew a lease from a DHCP server until it is successful.
IPADDR
property in the /etc/sysconfig/network-scripts/ifcfg-enp1s0
configuration file or by using nmcli:
~]$ nmcli connection modify enp1s0 ipv4.address 192.168.122.88/24
1.4. Setting the Wireless Regulatory Domain
regulatory.bin
file to keep its regulatory database information.
setregdomain
utility sets the regulatory domain for your system. Setregdomain
takes no arguments and is usually called through system script such as udev rather than manually by the administrator. If a country code look-up fails, the system administrator can define the COUNTRY
environment variable in the /etc/sysconfig/regdomain
file.
setregdomain(1)
man page — Sets regulatory domain based on country code.crda(8)
man page — Sends to the kernel a wireless regulatory domain for a given ISO or IEC 3166 alpha2.regulatory.bin(5)
man page — Shows the Linux wireless regulatory database.iw(8)
man page — Shows or manipulates wireless devices and their configuration.
1.5. Configuring netconsole
netconsole
kernel module enables to log kernel messages to another computer over the network.
netconsole
, you need to have an rsyslog
server that is properly configured on your network.
Procedure 1.1. Configuring an rsyslog server for netconsole
- Configure the
rsyslogd
daemon to listen on the 514/udp port and receive messages from the network by uncommenting the following lines in theMODULES
section of the/etc/rsyslog.conf
file:$ModLoad imudp $UDPServerRun 514
- Restart the
rsyslogd
service for the changes to take effect:]#
systemctl restart rsyslog
- Verify that
rsyslogd
is listening on the 514/udp port:]#
netstat -l | grep syslog
udp 0 0 0.0.0.0:syslog 0.0.0.0:* udp6 0 0 [::]:syslog [::]:*The0.0.0.0:syslog
and[::]:syslog
values in thenetstat -l
output mean thatrsyslogd
is listening on defaultnetconsole
port defined in the/etc/services
file:]$
cat /etc/services | grep syslog
syslog 514/udp syslog-conn 601/tcp # Reliable Syslog Service syslog-conn 601/udp # Reliable Syslog Service syslog-tls 6514/tcp # Syslog over TLS syslog-tls 6514/udp # Syslog over TLS syslog-tls 6514/dccp # Syslog over TLS
Netconsole
is configured using the /etc/sysconfig/netconsole
file, which is a part of the initscripts package. This package is installed by default and it also provides the netconsole
service.
Procedure 1.2. Configuring a Sending Machine
- Set the value of the
SYSLOGADDR
variable in the/etc/sysconfig/netconsole
file to match the IP address of thesyslogd
server. For example:SYSLOGADDR=192.168.0.1
- Restart the
netconsole
service for the changes to take effect:]#
systemctl restart netconsole.service
- Enable
netconsole.service
to run after rebooting the system:]#
systemctl enable netconsole.service
- View the
netconsole
messages from the client in the/var/log/messages
file (default) or in the file specified inrsyslog.conf
.]#
cat /var/log/messages
Note
rsyslogd
and netconsole.service
use port 514. To use a different port, change the following line in /etc/rsyslog.conf
to the required port number:
$UDPServerRun <PORT>
/etc/sysconfig/netconsole
file:
SYSLOGPORT=514
netconsole
configuration and troubleshooting tips, see Netconsole Kernel Documentation.
1.6. Using Network Kernel Tunables with sysctl
sysctl
utility, you can adjust network configuration on a running system and directly affect the networking performance.
sysctl
commands. For permanent changes that persist across system restarts, add lines to the /etc/sysctl.conf
file.
sysctl
parameters, enter as root
:
~]# sysctl -a
sysctl
, see the Using PTP with Multiple Interfaces section in the System Administrator's Guide.
1.7. Managing Data Using the ncat utility
IPv4
and IPv6
, open connections, send packets, perform port scanning, and supports higher-level features such as SSL
, and connection broker.
nc
command can also be entered as ncat
, using the identical options. For more information about the ncat options, see the New networking utility (ncat) section in the Migration Planning Guide and the ncat(1) man page.
Installing ncat
root
:
~]# yum install ncat
Brief Selection of ncat Use Cases
Example 1.1. Enabling Communication between a Client and a Server
- Set a client machine to listen for connections on TCP port 8080:
~]$
ncat -l 8080
- On a server machine, specify the IP address of the client and use the same port number:
~]$
ncat 10.0.11.60 8080
You can send messages on either side of the connection and they appear on both local and remote machines. - Press
Ctrl+D
to close the TCP connection.
Note
nc
commands with the –u
option. For example:
~]$ ncat -u -l 8080
Example 1.2. Sending Files
- On a client machine, to listen a specific port transferring a file to the server machine:
~]$
ncat -l 8080 > outputfile
- On a server machine, specify the IP address of the client, the port and the file which is to be transferred:
~]$
ncat -l 10.0.11.60 8080 < inputfile
Note
~]$ ncat -l 8080 < inputfile
~]$ ncat -l 10.0.11.60 8080 > outputfile
Example 1.3. Creating an HTTP proxy server
~]$ ncat -l --proxy-type http localhost 8080
Example 1.4. Port Scanning
–z
option and specify a range of ports to scan:
~]$ ncat -z 10.0.11.60 80-90
Connection to 192.168.0.1 80 port [tcp/http] succeeded!
Example 1.5. Setting up Secure Client-Server Communication Using SSL
SSL
on a server:
~]$ ncat -e /bin/bash -k -l 8080 --ssl
~]$ ncat --ssl 10.0.11.60 8080
Note
SSL
connection, the server requires the --ssl-cert
and --ssl-key
options, and the client requires the --ssl-verify
and --ssl-trustfile
options. For information on OpenSSL
, see the Using OpenSSL section in the Security Guide.
Part II. Managing IP Networking
Chapter 2. Getting Started with NetworkManager
2.1. Overview of NetworkManager
ifcfg
type configuration files are still supported. See Section 2.6, “Using NetworkManager with Network Scripts” for more information.
2.1.1. Benefits of Using NetworkManager
- Making Network management easier: NetworkManager ensures that network connectivity works. When it detects that there is no network configuration in a system but there are network devices, NetworkManager creates temporary connections to provide connectivity.
- Providing easy setup of connection to the user: NetworkManager offers management through different tools — GUI, nmtui, nmcli —. See Section 2.5, “NetworkManager Tools”.
- Supporting configuration flexibility. For example, configuring a WiFi interface, NetworkManager scans and shows the available wifi networks. You can select an interface, and NetworkManager displays the required credentials providing automatic connection after the reboot process. NetworkManager can configure network aliases, IP addresses, static routes, DNS information, and VPN connections, as well as many connection-specific parameters. You can modify the configuration options to reflect your needs.
- Offering an API through D-Bus which allows applications to query and control network configuration and state. In this way, applications can check or configure networking through D-BUS. For example, the
web console
interface, which monitors and configures servers through a web browser, uses the NetworkManager D-BUS interface to configure networking. - Maintaining the state of devices after the reboot process and taking over interfaces which are set into managed mode during restart.
- Handling devices which are not explicitly set unmanaged but controlled manually by the user or another network service.
2.2. Installing NetworkManager
root
:
~]# yum install NetworkManager
2.3. Checking the Status of NetworkManager
~]$ systemctl status NetworkManager
NetworkManager.service - Network Manager
Loaded: loaded (/lib/systemd/system/NetworkManager.service; enabled)
Active: active (running) since Fri, 08 Mar 2013 12:50:04 +0100; 3 days ago
systemctl status
command displays Active: inactive (dead)
when NetworkManager is not running.
2.4. Starting NetworkManager
~]# systemctl start NetworkManager
~]# systemctl enable NetworkManager
2.5. NetworkManager Tools
Application or Tool | Description |
---|---|
nmcli | A command-line tool which enables users and scripts to interact with NetworkManager. Note that nmcli can be used on systems without a GUI such as servers to control all aspects of NetworkManager. It has the same functionality as GUI tools. |
nmtui | A simple curses-based text user interface (TUI) for NetworkManager |
nm-connection-editor | A graphical user interface tool for certain tasks not yet handled by the control-center utility such as configuring bonds and teaming connections. You can add, remove, and modify network connections stored by NetworkManager. To start it, enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
|
control-center | A graphical user interface tool provided by the GNOME Shell, available for desktop users. It incorporates a Network settings tool. To start it, press the Super key to enter the Activities Overview, type Network and then press Enter. The Network settings tool appears. |
network connection icon | A graphical user interface tool provided by the GNOME Shell representing network connection states as reported by NetworkManager. The icon has multiple states that serve as visual indicators for the type of connection you are currently using. |
2.6. Using NetworkManager with Network Scripts
network scripts
refers to the script /etc/init.d/network
and any other installed scripts it calls. Although NetworkManager provides the default networking service, scripts and NetworkManager can run in parallel and work together. Red Hat recommends to test them first.
Running Network Script
systemctl
command: systemctl start|stop|restart|status
network
The systemctl utility clears any existing environment variables and ensures correct execution.
/etc/init.d/network
checks with NetworkManager to avoid tampering with NetworkManager's connections. NetworkManager is intended to be the primary application using sysconfig configuration files, and /etc/init.d/network
is intended to be secondary.
/etc/init.d/network
script runs:
- manually - using one of the
systemctl
commands
,start|stop|restart
networkor - on boot and shutdown if the network service is enabled - as a result of the
systemctl enable network
command.
ifup
and ifdown
scripts manually.
Note
systemctl reload network.service
command does not work due to technical limitations of initscripts. To apply a new configuration for the network service, use the restart
command:
~]# systemctl restart network.service
This brings down and brings up all the Network Interface Cards (NICs) to load the new configuration. For more information, see the Red Hat Knowledgebase solution Reload and force-reload options for network service.
Using Custom Commands in Network Scripts
/sbin/ifup-local
, ifdown-pre-local
, and ifdown-local
scripts are only executed if these devices are controlled by the /etc/init.d/network
service. The ifup-local
file does not exist by default. If required, create it under the /sbin/
directory.
ifup-local
script is readable only by the initscripts and not by NetworkManager. To run a custom script using NetworkManager, create it under the dispatcher.d/
directory. See the section called “Running Dispatcher scripts”.
Important
network scripts
and with NetworkManager. If NetworkManager is enabled, the ifup
and ifdown
script will ask NetworkManager whether NetworkManager manages the interface in question, which is found from the “DEVICE=” line in the ifcfg
file.
- calling ifup
- When you call
ifup
and the device is managed by NetworkManager, there are two options:- If the device is not already connected, then
ifup
asks NetworkManager to start the connection. - If the device is already connected, then nothing to do.
- calling ifdown
- When you call
ifdown
and the device is managed by NetworkManager:ifdown
asks NetworkManager to terminate the connection.
ifup
or ifdown
, the script starts the connection using the older, non-NetworkManager mechanism that it has used since the time before NetworkManager existed.
Running Dispatcher scripts
/etc/NetworkManager/dispatcher.d/
directory exists and NetworkManager runs scripts there, in alphabetical order. Each script must be an executable file owned by root
and must have write permission
only for the file owner. For more information about running NetworkManager dispatcher scripts, see the Red Hat Knowledgebase solution How to write a NetworkManager dispatcher script to apply ethtool commands.
2.7. Using NetworkManager with sysconfig files
/etc/sysconfig/
directory is a location for configuration files and scripts. Most network configuration information is stored there, with the exception of VPN, mobile broadband and PPPoE configuration, which are stored in the /etc/NetworkManager/
subdirectories. For example, interface-specific information is stored in the ifcfg
files in the /etc/sysconfig/network-scripts/
directory.
/etc/sysconfig/network
file. Information for VPNs, mobile broadband and PPPoE connections is stored in /etc/NetworkManager/system-connections/
.
ifcfg
file, NetworkManager is not automatically aware of the change and has to be prompted to notice the change. If you use one of the tools to update NetworkManager profile settings, NetworkManager does not implement those changes until you reconnect using that profile. For example, if configuration files have been changed using an editor, NetworkManager must read the configuration files again.
root
to reload all connection profiles:
~]# nmcli connection reload
ifcfg-ifname
:
~]# nmcli con load /etc/sysconfig/network-scripts/ifcfg-ifname
~]# nmcli dev disconnect
interface-name
~]# nmcli con up
interface-name
ifup
commands are used. See Section 2.6, “Using NetworkManager with Network Scripts” for the explanation of the network scripts.
ifup
script is a generic script which does a few things and then calls interface-specific scripts such as ifup-device_name
, ifup-wireless
, ifup-ppp
, and so on. When a user runs ifup enp1s0
manually:
ifup
looks for a file called/etc/sysconfig/network-scripts/ifcfg-enp1s0
;- if the
ifcfg
file exists,ifup
looks for theTYPE
key in that file to determine which type-specific script to call; ifup
callsifup-wireless
orifup-device_name
based onTYPE
;- the type-specific scripts do type-specific setup;
- the type-specific scripts let common functions perform
IP
-related tasks likeDHCP
or static setup.
/etc/init.d/network
reads through all the ifcfg
files and for each one that has ONBOOT=yes
, it checks whether NetworkManager is already starting the DEVICE from that ifcfg
file. If NetworkManager is starting that device or has already started it, nothing more is done for that file, and the next ONBOOT=yes
file is checked. If NetworkManager is not yet starting that device, the initscripts continue with their traditional behavior and call ifup
for that ifcfg
file.
ifcfg
file that has ONBOOT=yes
is expected to be started on system bootup, either by NetworkManager or by the initscripts. This ensures that some legacy network types which NetworkManager does not handle (such as ISDN or analog dial-up modems) as well as any new application not yet supported by NetworkManager are still correctly started by the initscripts even though NetworkManager is unable to handle them.
Important
/etc
directory, or in the same location as the live files, because the script literally does ifcfg-*
. Only these extensions are excluded: .old
, .orig
, .rpmnew
, .rpmorig
, and .rpmsave
.
2.8. Additional Resources
man(1)
man page — Describes man pages and how to find them.NetworkManager(8)
man page — Describes the network management daemon.NetworkManager.conf(5)
man page — Describes theNetworkManager
configuration file./usr/share/doc/initscripts-version/sysconfig.txt
— Describesifcfg
configuration files and their directives as understood by the legacy network service./usr/share/doc/initscripts-version/examples/networking/
— A directory containing example configuration files.ifcfg(8)
man page — Describes briefly theifcfg
command.
Chapter 3. Configuring IP Networking
3.1. Selecting Network Configuration Methods
- To configure a network interface using NetworkManager, use one of the following tools:
- the text user interface tool, nmtui. For more details, see Section 3.2, “Configuring IP Networking with nmtui”.
- the command-line tool, nmcli. For more details, see Section 3.3, “Configuring IP Networking with nmcli”.
- the graphical user interface tools, GNOME GUI. For more details, see Section 3.4, “ Configuring IP Networking with GNOME GUI ”.
- To configure a network interface without using NetworkManager:
- edit the
ifcfg
files manually. For more details, see Section 3.5, “Configuring IP Networking with ifcfg Files”. - use the
ip
commands. This can be used to assign IP addresses to an interface, but changes are not persistent across reboots; when you reboot, you will lose any changes. For more details, see Section 3.6, “Configuring IP Networking with ip Commands”.
- To configure the network settings when the root filesystem is not local:
- use the kernel command-line. For more details, see Section 3.7, “Configuring IP Networking from the Kernel Command line”.
3.2. Configuring IP Networking with nmtui
Prerequisites
- The nmtui tool is used in a terminal window. It is contained in the NetworkManager-tui package, but it is not installed along with NetworkManager by default. To install NetworkManager-tui:
~]#
yum install NetworkManager-tui
- To verify that NetworkManager is running, see Section 2.3, “Checking the Status of NetworkManager”.
Procedure
- Start the nmtui tool:
~]$
The text user interface appears.nmtui
Figure 3.1. The NetworkManager Text User Interface starting menu
- To navigate, use the arrow keys or press Tab to step forwards and press Shift+Tab to step back through the options. Press Enter to select an option. The Space bar toggles the status of a check box.
Procedure
- Select the
Activate a connection
menu entry.Figure 3.2. Activate a Connection
- Select the modified connection. On the right, click the
Deactivate
button.Figure 3.3. Deactivate the Modified Connection
- Choose the connection again and click the
Activate
button.Figure 3.4. Reactivate the Modified Connection
If no connection name is supplied, the selection menu appears. If the connection name is supplied and correctly identified, the relevant Edit connection screen appears.nmtui edit
connection-name
If no connection name is supplied, the selection menu appears. If the connection name is supplied and correctly identified, the relevant connection is activated. Any invalid command prints a usage message.nmtui connect
connection-name
802.1X
.
3.3. Configuring IP Networking with nmcli
- For servers, headless machines, and terminals, nmcli can be used to control NetworkManager directly, without GUI, including creating, editing, starting and stopping network connections and viewing network status.
- For scripts, nmcli supports a terse output format which is better suited for script processing. It is a way to integrate network configuration instead of managing network connections manually.
nmcli [OPTIONS] OBJECT { COMMAND | help }where OBJECT can be one of the following options:
general
, networking
, radio
, connection
, device
, agent
, and monitor
. You can use any prefix of these options in your commands. For example, nmcli con help
, nmcli c help
, nmcli connection help
generate the same output.
- -t, terse
- This mode can be used for computer script processing as you can see a terse output displaying only the values.
Example 3.1. Viewing a terse output
nmcli -t device
ens3:ethernet:connected:Profile 1 lo:loopback:unmanaged: - -f, field
- This option specifies what fields can be displayed in output. For example, NAME,UUID,TYPE,AUTOCONNECT,ACTIVE,DEVICE,STATE. You can use one or more fields. If you want to use more, do not use space after comma to separate the fields.
Example 3.2. Specifying Fields in the output
~]$
or even better for scripting:nmcli -f DEVICE,TYPE device
DEVICE TYPE ens3 ethernet lo loopback~]$
nmcli -t -f DEVICE,TYPE device
ens3:ethernet lo:loopback - -p, pretty
- This option causes nmcli to produce human-readable output. For example, values are aligned and headers are printed.
Example 3.3. Viewing an output in pretty mode
nmcli -p device
===================== Status of devices ===================== DEVICE TYPE STATE CONNECTION -------------------------------------------------------------- ens3 ethernet connected Profile 1 lo loopback unmanaged -- - -h, help
- Prints help information.
nmcli help
- This command lists the available options and object names to be used in subsequent commands.
nmcli object help
- This command displays the list of available actions related to a specified object. For example,
nmcli c help
3.3.1. Brief Selection of nmcli Examples
Example 3.4. Checking the overall status of NetworkManager
~]$ nmcli general status
STATE CONNECTIVITY WIFI-HW WIFI WWAN-HW WWAN
connected full enabled enabled enabled enabled
In terse mode:
~]$ nmcli -t -f STATE general
connected
Example 3.5. Viewing NetworkManager logging status
~]$ nmcli general logging
LEVEL DOMAINS
INFO PLATFORM,RFKILL,ETHER,WIFI,BT,MB,DHCP4,DHCP6,PPP,WIFI_SCAN,IP4,IP6,A
UTOIP4,DNS,VPN,SHARING,SUPPLICANT,AGENTS,SETTINGS,SUSPEND,CORE,DEVICE,OLPC,
WIMAX,INFINIBAND,FIREWALL,ADSL,BOND,VLAN,BRIDGE,DBUS_PROPS,TEAM,CONCHECK,DC
B,DISPATCH
Example 3.6. Viewing all connections
~]$ nmcli connection show
NAME UUID TYPE DEVICE
Profile 1 db1060e9-c164-476f-b2b5-caec62dc1b05 ethernet ens3
ens3 aaf6eb56-73e5-4746-9037-eed42caa8a65 ethernet --
Example 3.7. Viewing only currently active connections
~]$ nmcli connection show --active
NAME UUID TYPE DEVICE
Profile 1 db1060e9-c164-476f-b2b5-caec62dc1b05 ethernet ens3
Example 3.8. Viewing only devices recognized by NetworkManager and their state
~]$ nmcli device status
DEVICE TYPE STATE CONNECTION
ens3 ethernet connected Profile 1
lo loopback unmanaged --
nmcli command | abbreviation | |
---|---|---|
nmcli general status | nmcli g | |
nmcli general logging | nmcli g log | |
nmcli connection show | nmcli con show | |
nmcli connection show --active | nmcli con show -a | |
nmcli device status | nmcli dev |
3.3.2. Starting and Stopping a Network Interface Using nmcli
nmcli con up id bond0 nmcli con up id port0 nmcli dev disconnect bond0 nmcli dev disconnect ens3
Note
nmcli connection down
command, deactivates a connection from a device without preventing the device from further auto-activation. The nmcli device disconnect
command, disconnects a device and prevent the device from automatically activating further connections without manual intervention.
3.3.3. Understanding the nmcli Options
connection.type
- A connection type. Allowed values are: adsl, bond, bond-slave, bridge, bridge-slave, bluetooth, cdma, ethernet, gsm, infiniband, olpc-mesh, team, team-slave, vlan, wifi, wimax. Each connection type has type-specific command options. You can see the
TYPE_SPECIFIC_OPTIONS
list in the nmcli(1) man page. For example:- A
gsm
connection requires the access point name specified in anapn
.nmcli c add connection.type gsm apn access_point_name
- A
wifi
device requires the service set identifier specified in assid
.nmcli c add connection.type wifi ssid My identifier
connection.interface-name
- A device name relevant for the connection.
nmcli con add connection.interface-name enp1s0 type ethernet
connection.id
- A name used for the connection profile. If you do not specify a connection name, one will be generated as follows:
connection.type -connection.interface-name
Theconnection.id
is the name of a connection profile and should not be confused with the interface name which denotes a device (wlp61s0
,ens3
,em1
). However, users can name the connections after interfaces, but they are not the same thing. There can be multiple connection profiles available for a device. This is particularly useful for mobile devices or when switching a network cable back and forth between different devices. Rather than edit the configuration, create different profiles and apply them to the interface as needed. Theid
option also refers to the connection profile name.
show
, up
, down
are:
id
- An identification string assigned by the user to a connection profile. Id can be used in nmcli connection commands to identify a connection. The NAME field in the command output always denotes the connection id. It refers to the same connection profile name that the con-name does.
uuid
- A unique identification string assigned by the system to a connection profile. The
uuid
can be used innmcli connection
commands to identify a connection.
3.3.4. Using the nmcli Interactive Connection Editor
~]$ nmcli con edit
You will be prompted to enter a valid connection type from the list displayed. After entering a connection type you will be placed at the nmcli prompt. If you are familiar with the connection types you can add a valid connection type
option to the nmcli con edit
command and be taken straight to the nmcli prompt. The format is as follows for editing an existing connection profile: nmcli con edit [id | uuid | path] IDFor editing a new connection profile:
nmcli con edit [type new-connection-type] [con-name new-connection-name]
help
at the nmcli prompt to see a list of valid commands. Use the describe
command to get a description of settings and their properties: describe setting.propertyFor example:
nmcli> describe team.config
3.3.5. Creating and Modifying a Connection Profile with nmcli
nmcli c add {ARGUMENTS}
nmcli c add
accepts two different types of parameters:
- Property names
- the names which NetworkManager uses to describe the connection internally. The most important are:
- connection.type
nmcli c add connection.type
bond
- connection.interface-name
nmcli c add connection.interface-name
enp1s0
- connection.id
nmcli c add connection.id
"My Connection"
See thenm-settings(5)
man page for more information on properties and their settings.
- Aliases names
- the human-readable names which are translated to properties internally. The most common are:
- type (the connection.type property)
nmcli c add type
bond
- ifname (the connection.interface-name property)
nmcli c add ifname
enp1s0
- con-name (the connection.id property)
nmcli c add con-name
"My Connection"
nmcli
, to create a connection required using the aliases
. For example, ifname
enp1s0 and con-name
My Connection. A command in the following format could be used: nmcli c add type ethernet ifname enp1s0 con-name "My Connection"
property names
and the aliases
can be used interchangeably. The following examples are all valid and equivalent: nmcli c add type ethernet ifname enp1s0 con-name "My Connection" ethernet.mtu 1600
nmcli c add connection.type ethernet ifname enp1s0 con-name "My Connection" ethernet.mtu 1600
nmcli c add connection.type ethernet connection.interface-name enps1s0 connection.id "My Connection" ethernet.mtu 1600
type
argument is mandatory for all connection types and ifname
is mandatory for all types except bond
, team
, bridge
and vlan
.
- type type_name
- connection type. For example:
nmcli c add type
bond
- ifname interface_name
- interface to bind the connection to. For example:
nmcli c add ifname interface_name type ethernet
nmcli c modify
connection.id
from My Connection to My favorite connection
and the connection.interface-name
to enp1s0
, issue the command as follows: nmcli c modify "My Connection" connection.id "My favorite connection" connection.interface-name enp1s0
Note
property names
. The aliases
are used only for compatibility reasons.
nmcli c modify "My favorite connection" ethernet.mtu 1600
nmcli con up con-name
For example:
nmcli con up My-favorite-connection
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/16)
3.3.6. Connecting to a Network Using nmcli
~]$ nmcli con show
NAME UUID TYPE DEVICE
Auto Ethernet 9b7f2511-5432-40ae-b091-af2457dfd988 802-3-ethernet --
ens3 fb157a65-ad32-47ed-858c-102a48e064a2 802-3-ethernet ens3
MyWiFi 91451385-4eb8-4080-8b82-720aab8328dd 802-11-wireless wlp61s0
Note that the NAME
field in the output always denotes the connection ID (name). It is not the interface name even though it might look the same. In the second connection shown above, ens3
in the NAME field is the connection ID given by the user to the profile applied to the interface ens3. In the last connection shown, the user has assigned the connection ID MyWiFi
to the interface wlp61s0.
~]$ nmcli device status
DEVICE TYPE STATE CONNECTION
ens3 ethernet disconnected --
ens9 ethernet disconnected --
lo loopback unmanaged --
3.3.7. Adding and Configuring a Dynamic Ethernet Connection with nmcli
Adding a Dynamic Ethernet Connection
IP
configuration, allowing DHCP
to assign the network configuration: nmcli connection add type ethernet con-name connection-name ifname interface-name
~]$ nmcli con add type ethernet con-name my-office ifname ens3
Connection 'my-office' (fb157a65-ad32-47ed-858c-102a48e064a2) successfully added.
~]$ nmcli con up my-office
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
Review the status of the devices and connections:
~]$ nmcli device status
DEVICE TYPE STATE CONNECTION
ens3 ethernet connected my-office
ens9 ethernet disconnected --
lo loopback unmanaged --
Configuring a Dynamic Ethernet Connection
DHCP
server, modify the dhcp-hostname
property:
~]$ nmcli con modify my-office my-office ipv4.dhcp-hostname host-name ipv6.dhcp-hostname host-name
IPv4
client ID sent by a host to a DHCP
server, modify the dhcp-client-id
property:
~]$ nmcli con modify my-office my-office ipv4.dhcp-client-id client-ID-string
There is no dhcp-client-id
property for IPv6
, dhclient creates an identifier for IPv6
. See the dhclient(8)
man page for details.
DNS
servers sent to a host by a DHCP
server, modify the ignore-auto-dns
property:
~]$ nmcli con modify my-office my-office ipv4.ignore-auto-dns yes ipv6.ignore-auto-dns yes
nm-settings(5)
man page for more information on properties and their settings.
Example 3.9. Configuring a Dynamic Ethernet Connection Using the Interactive Editor
~]$ nmcli con edit type ethernet con-name ens3
===| nmcli interactive connection editor |===
Adding a new '802-3-ethernet' connection
Type 'help' or '?' for available commands.
Type 'describe [<setting>.<prop>]' for detailed property description.
You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, ipv4, ipv6, dcb
nmcli> describe ipv4.method
=== [method] ===
[NM property description]
IPv4 configuration method. If 'auto' is specified then the appropriate automatic method (DHCP, PPP, etc) is used for the interface and most other properties can be left unset. If 'link-local' is specified, then a link-local address in the 169.254/16 range will be assigned to the interface. If 'manual' is specified, static IP addressing is used and at least one IP address must be given in the 'addresses' property. If 'shared' is specified (indicating that this connection will provide network access to other computers) then the interface is assigned an address in the 10.42.x.1/24 range and a DHCP and forwarding DNS server are started, and the interface is NAT-ed to the current default network connection. 'disabled' means IPv4 will not be used on this connection. This property must be set.
nmcli> set ipv4.method auto
nmcli> save
Saving the connection with 'autoconnect=yes'. That might result in an immediate activation of the connection.
Do you still want to save? [yes] yes
Connection 'ens3' (090b61f7-540f-4dd6-bf1f-a905831fc287) successfully saved.
nmcli> quit
~]$
The default action is to save the connection profile as persistent. If required, the profile can be held in memory only, until the next restart, by means of the save temporary
command.
3.3.8. Adding and Configuring a Static Ethernet Connection with nmcli
Adding a Static Ethernet Connection
IPv4
configuration: nmcli connection add type ethernet con-name connection-name ifname interface-name ip4 address gw4 address
IPv6
address and gateway information can be added using the ip6
and gw6
options.
IPv4
address and gateway:
~]$Optionally, at the same time specifynmcli con add type ethernet con-name test-lab ifname ens9 ip4 10.10.10.10/24 \
gw4 10.10.10.254
IPv6
address and gateway for the device:
~]$nmcli con add type ethernet con-name test-lab ifname ens9 ip4 10.10.10.10/24 \
gw4 10.10.10.254 ip6 abbe::cafe gw6 2001:db8::1
Connection 'test-lab' (05abfd5e-324e-4461-844e-8501ba704773) successfully added.
IPv4
DNS
server addresses:
~]$ nmcli con mod test-lab ipv4.dns "8.8.8.8 8.8.4.4"
Note that this will replace any previously set DNS
servers. To set two IPv6
DNS
server addresses:
~]$ nmcli con mod test-lab ipv6.dns "2001:4860:4860::8888 2001:4860:4860::8844"
Note that this will replace any previously set DNS
servers. Alternatively, to add additional DNS
servers to any previously set, use the +
prefix:
~]$ nmcli con mod test-lab +ipv4.dns "8.8.8.8 8.8.4.4"
~]$ nmcli con mod test-lab +ipv6.dns "2001:4860:4860::8888 2001:4860:4860::8844"
~]$ nmcli con up test-lab ifname ens9
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
Review the status of the devices and connections:
~]$ nmcli device status
DEVICE TYPE STATE CONNECTION
ens3 ethernet connected my-office
ens9 ethernet connected test-lab
lo loopback unmanaged --
~]$ nmcli -p con show test-lab
===============================================================================
Connection profile details (test-lab)
===============================================================================
connection.id: test-lab
connection.uuid: 05abfd5e-324e-4461-844e-8501ba704773
connection.interface-name: ens9
connection.type: 802-3-ethernet
connection.autoconnect: yes
connection.timestamp: 1410428968
connection.read-only: no
connection.permissions:
connection.zone: --
connection.master: --
connection.slave-type: --
connection.secondaries:
connection.gateway-ping-timeout: 0
[output truncated]
The use of the -p, --pretty
option adds a title banner and section breaks to the output.
Example 3.10. Configuring a Static Ethernet Connection Using the Interactive Editor
~]$ nmcli con edit type ethernet con-name ens3
===| nmcli interactive connection editor |===
Adding a new '802-3-ethernet' connection
Type 'help' or '?' for available commands.
Type 'describe [>setting<.>prop<]' for detailed property description.
You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, ipv4, ipv6, dcb
nmcli> set ipv4.addresses 192.168.122.88/24
Do you also want to set 'ipv4.method' to 'manual'? [yes]: yes
nmcli>
nmcli> save temporary
Saving the connection with 'autoconnect=yes'. That might result in an immediate activation of the connection.
Do you still want to save? [yes] no
nmcli> save
Saving the connection with 'autoconnect=yes'. That might result in an immediate activation of the connection.
Do you still want to save? [yes] yes
Connection 'ens3' (704a5666-8cbd-4d89-b5f9-fa65a3dbc916) successfully saved.
nmcli> quit
~]$
The default action is to save the connection profile as persistent. If required, the profile can be held in memory only, until the next restart, by means of the save temporary
command.
connection.autoconnect
to yes
. NetworkManager will also write out settings to /etc/sysconfig/network-scripts/ifcfg-my-office
where the corresponding BOOTPROTO will be set to none
and ONBOOT to yes
.
3.3.9. Locking a Profile to a Specific Device Using nmcli
nmcli connection add type ethernet con-name connection-name ifname interface-nameTo make a profile usable for all compatible Ethernet interfaces:
nmcli connection add type ethernet con-name connection-name ifname "*"Note that you have to use the
ifname
argument even if you do not want to set a specific interface. Use the wildcard character *
to specify that the profile can be used with any compatible device.
nmcli connection add type ethernet con-name "connection-name" ifname "*" mac 00:00:5E:00:53:00
3.3.10. Adding a Wi-Fi Connection with nmcli
~]$ nmcli dev wifi list
SSID MODE CHAN RATE SIGNAL BARS SECURITY
FedoraTest Infra 11 54 MB/s 98 ▂▄▆█ WPA1
Red Hat Guest Infra 6 54 MB/s 97 ▂▄▆█ WPA2
Red Hat Infra 6 54 MB/s 77 ▂▄▆_ WPA2 802.1X
* Red Hat Infra 40 54 MB/s 66 ▂▄▆_ WPA2 802.1X
VoIP Infra 1 54 MB/s 32 ▂▄__ WEP
MyCafe Infra 11 54 MB/s 39 ▂▄__ WPA2
IP
configuration, but allowing automatic DNS
address assignment:
~]$To set a WPA2 password, for example “caffeine”:nmcli con add con-name MyCafe ifname wlp61s0 type wifi ssid MyCafe \
ip4 192.168.100.101/24 gw4 192.168.100.1
~]$See the Red Hat Enterprise Linux 7 Security Guide for information on password security.nmcli con modify MyCafe wifi-sec.key-mgmt wpa-psk
~]$nmcli con modify MyCafe wifi-sec.psk caffeine
~]$ nmcli radio wifi [on | off ]
Changing a Specific Property Using nmcli
mtu
:
~]$ nmcli connection show id 'MyCafe' | grep mtu
802-11-wireless.mtu: auto
To change the property of a setting:
~]$ nmcli connection modify id 'MyCafe' 802-11-wireless.mtu 1350
To verify the change:
~]$ nmcli connection show id 'MyCafe' | grep mtu
802-11-wireless.mtu: 1350
802-3-ethernet
and 802-11-wireless
as the setting, and mtu
as a property of the setting. See the nm-settings(5)
man page for more information on properties and their settings.
3.3.11. Configuring NetworkManager to Ignore Certain Devices
lo
(loopback) device. However, you can set certain devices as unmanaged
to configure that NetworkManager ignores these devices. With this setting, you can manually manage these devices, for example, using a script.
3.3.11.1. Permanently Configuring a Device as Unmanaged in NetworkManager
unmanaged
based on several criteria, such as the interface name, MAC address, or device type. This procedure describes how to permanently set the enp1s0
interface as unmanaged in NetworkManager.
unmanaged
, see Section 3.3.11.2, “Temporarily Configuring a Device as Unmanaged in NetworkManager”.
Procedure
- Optional: Display the list of devices to identify the device you want to set as
unmanaged
:#
nmcli device status
DEVICE TYPE STATE CONNECTION enp1s0 ethernet disconnected -- ... - Create the
/etc/NetworkManager/conf.d/99-unmanaged-devices.conf
file with the following content:[keyfile] unmanaged-devices=interface-name:enp1s0
To set multiple devices as unmanaged, separate the entries in theunmanaged-devices
parameter with semicolon:[keyfile] unmanaged-devices=interface-name:interface_1;interface-name:interface_2;...
- Reload the
NetworkManager
service:#
systemctl reload NetworkManager
Verification Steps
- Display the list of devices:
#
nmcli device status
DEVICE TYPE STATE CONNECTION enp1s0 ethernet unmanaged -- ...Theunmanaged
state next to theenp1s0
device indicates that NetworkManager does not manage this device.
Additional Resources
3.3.11.2. Temporarily Configuring a Device as Unmanaged in NetworkManager
unmanaged
based on several criteria, such as the interface name, MAC address, or device type. This procedure describes how to temporarily set the enp1s0
interface as unmanaged
in NetworkManager.
unmanaged
, see Section 3.3.11.1, “Permanently Configuring a Device as Unmanaged in NetworkManager”.
Procedure
- Optional: Display the list of devices to identify the device you want to set as
unmanaged
:#
nmcli device status
DEVICE TYPE STATE CONNECTION enp1s0 ethernet disconnected -- ... - Set the
enp1s0
device to theunmanaged
state:#
nmcli device set enp1s0 managed no
Verification Steps
- Display the list of devices:
#
nmcli device status
DEVICE TYPE STATE CONNECTION enp1s0 ethernet unmanaged -- ...Theunmanaged
state next to theenp1s0
device indicates that NetworkManager does not manage this device.
Additional Resources
3.4. Configuring IP Networking with GNOME GUI
- the GNOME control-center application
- the GNOME nm-connection-editor application
3.4.1. Connecting to a Network Using the control-center GUI
- Press the Super key to enter the Activities Overview, type Settings and then press Enter. Then, select the
Network
tab on the left-hand side, and the Network settings tool appears. Proceed to the section called “Configuring New Connections with control-center”. - Click on the GNOME Shell network connection icon in the top right-hand corner of the screen to open its menu.
Figure 3.5. Network Configuration using the control-center application
- A list of categorized networks you are currently connected to (such as Wired and Wi-Fi).
- A list of all Available Networks that NetworkManager has detected.
- Options for connecting to any configured Virtual Private Networks (VPNs)and
- An option for selecting the Network Settings menu entry.
3.4.2. Configuring New and Editing Existing Connections Using a GUI
- the GNOME control-center application
- the GNOME nm-connection-editor application
3.4.2.1. Configuring New and Editing Existing Connections Using control-center
Configuring New Connections with control-center
- Press the Super key to enter the Activities Overview, type Settings and then press Enter. Then, select the
Network
tab on the left-hand side. The Network settings tool appears on the right-hand side menu:Figure 3.6. Opening the Network Settings Window
- Click the plus button to add a new connection.To configure:
- Wired connections, click the plus button next to Wired entry and proceed to Section 3.4.6, “Configuring a Wired (Ethernet) Connection with a GUI”.
- VPN connections, click the plus button next to VPN entry and proceed to Section 3.4.8.1, “Establishing a VPN Connection with control-center”
For Wi-Fi connections, click the Wi-fi entry in theSettings
menu and proceed to Section 3.4.7, “Configuring a Wi-Fi Connection with a GUI”
Editing an Existing Connection with control-center
IP
addressing, DNS
, and routing configuration.
Figure 3.7. Configure Networks Using the Network Connection Details Window
3.4.2.2. Configuring New and Editing Existing Connections Using nm-connection-editor
Configuring a New Connection with nm-connection-editor
Procedure
- Enter
nm-connection-editor
in a terminal:~]$ nm-connection-editor
TheNetwork Connections
window appears. - Click the plus button to choose a connection type:
Figure 3.8. Adding a connection type using nm-connection-editor
Figure 3.9. Choosing a connection type with nm-connection-editor
To create and configure:- Bond connections, click the Bond entry and proceed to Section 7.8.1, “Establishing a Bond Connection”;
- Bridge connections, click the Bridge entry and proceed to Section 9.4.1, “Establishing a Bridge Connection with a GUI”;
- VLAN connections, click the VLAN entry and proceed to Section 10.5.1, “Establishing a VLAN Connection”; or,
- Team connections, click the Team entry and proceed to Section 8.14, “Creating a Network Team Using a GUI”.
Editing an Existing Connection with nm-connection-editor
3.4.3. Common Configuration Options Using nm-connection-editor
nm-connection-editor
utility, there are five common configuration options to the most connection types (ethernet, wifi, mobile broadband, DSL) following the procedure below:
Procedure
- Enter
nm-connection-editor
in a terminal:~]$ nm-connection-editor
TheNetwork Connections
window appears. Click the plus button to choose a connection type or the gear wheel icon to edit an existing connection. - Select the General tab in the Editing dialog:
Figure 3.10. Configuration options in nm-connection-editor
- Connection name — Enter a descriptive name for your network connection. This name is used to list this connection in the menu of the Network window.
- Connection priority for auto-activation — If the connection is set to autoconnect, the number is activated (
0
by default). The higher number means higher priority. - Automatically connect to this network when it is available — Select this box if you want NetworkManager to auto-connect to this connection when it is available. See the section called “Editing an Existing Connection with control-center” for more information.
- All users may connect to this network — Select this box to create a connection available to all users on the system. Changing this setting may require root privileges. See Section 3.4.5, “Managing System-wide and Private Connection Profiles with a GUI” for details.
- Automatically connect to VPN when using this connection — Select this box if you want NetworkManager to auto-connect to a VPN connection when it is available. Select the VPN from the drop-down menu.
- Firewall Zone — Select the firewall zone from the drop-down menu. See the Red Hat Enterprise Linux 7 Security Guide for more information on firewall zones.
Note
3.4.4. Connecting to a Network Automatically with a GUI
- the GNOME control-center application
- the GNOME nm-connection-editor application
3.4.4.1. Connecting to a Network Automatically with control-center
Procedure
- Press the Super key to enter the Activities Overview, type
Settings
and then press Enter. Then, select the Network tab on the left-hand side. The Network settings tool appears on the right-hand side menu, see the section called “Configuring New Connections with control-center”. - Select the network interface from the right-hand-side menu.
- Click on the gear wheel icon of a connection profile on the right-hand side menu. The Network details window appears.
- Select the Details menu entry, see the section called “Editing an Existing Connection with control-center”.
- Select Connect automatically to cause NetworkManager to auto-connect to the connection whenever NetworkManager detects that it is available. Clear the check box if you do not want NetworkManager to connect automatically. If the check box is clear, you will have to select that connection manually in the network connection icon's menu to cause it to connect.
3.4.4.2. Connecting to a Network Automatically with nm-connection-editor
Automatically connect to this network when it is available
check box in the General tab.
3.4.5. Managing System-wide and Private Connection Profiles with a GUI
nm-settings(5)
man page for more information on the connection
settings permissions
property. You can control access to a connection profile using the following graphical user interface tools:
- the nm-connection-editor application
- the control-center application
3.4.5.1. Managing Permissions for a Connection Profile with nm-connection-editor
3.4.5.2. Managing Permissions for a Connection Profile with control-center
Details
window.
Note
user-em2
with the Connect Automatically check box selected but with the Make available to other users not selected, then the connection will not be available at boot time.
- Clear the Make available to other users check box, which changes the connection to be modifiable and usable only by the user doing the changing.
- Use the polkit framework to restrict permissions of general network operations on a per-user basis.
polkit(8)
man page for more information on polkit.
3.4.6. Configuring a Wired (Ethernet) Connection with a GUI
- the control-center application
- the nm-connection-editor application
3.4.6.1. Configuring a Wired Connection Using control-center
Procedure
- Press the Super key to enter the Activities Overview, type Settings and then press Enter. Then, select the
Network
menu entry on the left-hand side, and the Network settings tool appears, see the section called “Configuring New Connections with control-center”. - Select the Wired network interface if it is not already highlighted.The system creates and configures a single wired connection profile called Wired by default. A profile is a named collection of settings that can be applied to an interface. More than one profile can be created for an interface and applied as needed. The default profile cannot be deleted but its settings can be changed.
- Edit the default Wired profile by clicking the gear wheel icon.
Basic Configuration Options
Figure 3.11. Basic Configuration options of a Wired Connection
- Name — Enter a descriptive name for your network connection. This name will be used to list this connection in the menu of the Network window.
- MAC Address — Select the MAC address of the interface this profile must be applied to.
- Cloned Address — If required, enter a different MAC address to use.
- MTU — If required, enter a specific maximum transmission unit (MTU) to use. The MTU value represents the size in bytes of the largest packet that the link layer will transmit. This value defaults to
1500
and does not generally need to be specified or changed.
Making Further Wired Configurations
IPv4
settings for the connection, click the IPv4 menu entry and proceed to Section 5.4, “Configuring IPv4 Settings”orIPv6
settings for the connection, click the IPv6 menu entry and proceed to Section 5.5, “Configuring IPv6 Settings”.- port-based Network Access Control (PNAC), click the 802.1X Security menu entry and proceed to Section 5.2, “Configuring 802.1X Security”;
Saving Your New (or Modified) Wired Connection
Creating a New Wired Connection
3.4.6.2. Configuring a Wired Connection with nm-connection-editor
- Enter the nm-connection-editor in a terminal.
~]$ nm-connection-editor
TheNetwork Connections
window appears. - Select the ethernet connection you want to edit and click the gear wheel icon:
Figure 3.12. Edit a wired connection
TheEditing
dialog appears.- To connect to a network automatically and restrict connections, click the
General
tab, see Section 3.4.3, “Common Configuration Options Using nm-connection-editor”. - To configure the networking settings, click the
Ethernet
tab, see the section called “Configuring 802.3 Link Settings with nm-connection-editor”. - To configure 802.1X Security for a wired connection, click the
802.1X Security
tab, see Section 5.2.4, “Configuring 802.1X Security for Wired with nm-connection-editor”. - To configure the IPV4 settings, click the
IPV4 Settings
tab, see the section called “Setting the Method for IPV4 Using nm-connection-editor”. - To configure the IPV6 settings, click the
IPV6 Settings
tab, see Section 5.5, “Configuring IPv6 Settings”.
3.4.7. Configuring a Wi-Fi Connection with a GUI
Wi-Fi
(also known as wireless or 802.11a/b/g/n) connection to an Access Point. An Access Point is a device that allows wireless devices to connect to a network.
Connecting Quickly to an Available Access Point
Procedure
- Click on the network connection icon to activate the network connection icon's menu, see Section 3.4.1, “Connecting to a Network Using the control-center GUI”.
- Locate the Service Set Identifier (SSID) of the access point in the list of Wi-Fi networks.
- Click on the SSID of the network. A padlock symbol indicates the access point requires authentication. If the access point is secured, a dialog prompts you for an authentication key or password.NetworkManager tries to auto-detect the type of security used by the access point. If there are multiple possibilities, NetworkManager guesses the security type and presents it in the Wi-Fi security drop-down menu.
- For WPA-PSK security (WPA with a passphrase) no choice is necessary.
- For WPA Enterprise (802.1X) you have to specifically select the security, because that cannot be auto-detected.Note that if you are unsure, try connecting to each type in turn.
- Enter the key or passphrase in the Password field. Certain password types, such as a 40-bit WEP or 128-bit WPA key, are invalid unless they are of a requisite length. The Connect button will remain inactive until you enter a key of the length required for the selected security type. To learn more about wireless security, see Section 5.2, “Configuring 802.1X Security”.
Connecting to a Hidden Wi-Fi Network
Procedure
- Press the Super key to enter the Activities Overview, type Settings and then press Enter. Then, select the
Wi-Fi
menu entry on the left-hand side. - Select Connect to Hidden Network. There are two options:
- If you have connected to the hidden network before:
- Use the Connection drop-down to select the network.
- Click.
- If not, proceed as follows:
- Leave the Connection drop-down as .
- Enter the SSID of the hidden network.
- Select its Wi-Fi security method.
- Enter the correct authentication secrets.
- Click.
Configuring a New Wi-Fi Connection
Procedure
- Select the Wi-Fi menu entry of Settings.
- Click the Wi-Fi connection name that you want to connect to (by default, the same as the SSID).
- If the SSID is not in range, see the section called “Connecting to a Hidden Wi-Fi Network” for more information.
- If the SSID is in range, click the
Wi-Fi
connection profile on the right-hand side menu. A padlock symbol indicates a key or password is required. If requested, enter the authentication details.
Editing an Existing Wi-Fi Connection
Procedure
- Press the Super key to enter the Activities Overview, type Settings and press Enter.
- Select Wi-Fi from the left-hand-side menu entry.
- Select the gear wheel icon to the right of the Wi-Fi connection name that you want to edit, and the editing connection dialog appears. Note that if the network is not currently in range, click History to display past connections. The Details window shows the connection details.
Basic Configuration Options for a Wi-Fi Connection
Figure 3.13. Basic Configuration Options for a Wi-Fi Connection
- SSID
- The Service Set Identifier (SSID) of the access point (AP).
- BSSID
- The Basic Service Set Identifier (BSSID) is the MAC address, also known as a hardware address, of the specific wireless access point you are connecting to when in Infrastructure mode. This field is blank by default, and you are able to connect to a wireless access point by SSID without having to specify its BSSID. If the BSSID is specified, it will force the system to associate to a specific access point only.For ad-hoc networks, the BSSID is generated randomly by the mac80211 subsystem when the ad-hoc network is created. It is not displayed by NetworkManager
- MAC address
- Select the MAC address, also known as a hardware address, of the Wi-Fi interface to use.A single system could have one or more wireless network adapters connected to it. The MAC address field therefore allows you to associate a specific wireless adapter with a specific connection (or connections).
- Cloned Address
- A cloned MAC address to use in place of the real hardware address. Leave blank unless required.
- Connect automatically — Select this box if you want NetworkManager to auto-connect to this connection when it is available. See the section called “Editing an Existing Connection with control-center” for more information.
- Make available to other users — Select this box to create a connection available to all users on the system. Changing this setting may require root privileges. See Section 3.4.5, “Managing System-wide and Private Connection Profiles with a GUI” for details.
Making Further Wi-Fi Configurations
- security authentication for the wireless connection, click Security and proceed to Section 5.2, “Configuring 802.1X Security”.
IPv4
settings for the connection, click IPv4 and proceed to Section 5.4, “Configuring IPv4 Settings”orIPv6
settings for the connection, click IPv6 and proceed to Section 5.5, “Configuring IPv6 Settings”.
Saving Your New (or Modified) Connection
3.4.8. Configuring a VPN Connection with a GUI
IPsec
, provided by Libreswan, is the preferred method for creating a VPN. Libreswan is an open-source, user-space IPsec
implementation for VPN. Configuring an IPsec
VPN using the command line is documented in the Red Hat Enterprise Linux 7 Security Guide.
3.4.8.1. Establishing a VPN Connection with control-center
IPsec
, provided by Libreswan, is the preferred method for creating a VPN in Red Hat Enterprise Linux 7. For more information, see Section 3.4.8, “Configuring a VPN Connection with a GUI”.
root
:
~]# yum install NetworkManager-libreswan-gnome
- it adds an Authentication Header for routing and authentication purposes;
- it encrypts the packet data; and,
- it encloses the data in packets according to the Encapsulating Security Payload (ESP) protocol, which constitutes the decryption and handling instructions.
Adding a New IPsec VPN Connection
Procedure
- Press the Super key to enter the Activities Overview, type Settings and press Enter. Then, select the Network menu entry and the Network settings tool appears, see the section called “Configuring New Connections with control-center”.
- Click the plus button in the VPN entry.
- The Add VPN window appears. For manually configuration, select IPsec based VPN.
Figure 3.14. Configuring VPN on IPsec mode
- In the
Identity
configuration form, you can specify the fields in theGeneral
andAdvanced
sections:Figure 3.15. General and Advanced sections
- In
General
section, you can specify:
- Gateway
- The name or
IP
address of the remote VPN gateway. - User name
- If required, enter the user name associated with the VPN user's identity for authentication.
- User password
- If required, enter the password associated with the VPN user's identity for authentication.
- Group name
- The name of a VPN group configured on the remote gateway. In case it is blank, the IKEv1 Main mode is used instead of the default Aggressive mode.
- Secret
- It is a pre-shared key which is used to initialize the encryption before the user's authentication. If required, enter the password associated with the group name.
- The following configuration settings are available under the
Advanced
section:
- Phase1 Algorithms
- If required, enter the algorithms to be used to authenticate and set up an encrypted channel.
- Phase2 Algorithms
- If required, enter the algorithms to be used for the
IPsec
negotiations. - Domain
- If required, enter the Domain Name.
Note
IPsec
VPN without using NetworkManager, see Section 3.4.8, “Configuring a VPN Connection with a GUI”.
Editing an Existing VPN Connection
Procedure
- Press the Super key to enter the Activities Overview, type Settings and press Enter. Then, select the Network menu entry and the Network settings tool appears, see the section called “Configuring New Connections with control-center”.
- Select the VPN connection you want to edit and click the gear wheel icon and edit the
General
andAdvanced
sections, see Section 3.4.8.1, “Establishing a VPN Connection with control-center”.
Saving Your New (or Modified) Connection and Making Further Configurations
IPv4
settings for the connection, click the IPv4 Settings tab and proceed to Section 5.4, “Configuring IPv4 Settings”.
3.4.8.2. Configuring a VPN Connection with nm-connection-editor
Procedure
- Enter nm-connection-editor in a terminal. The Network Connections window appears, see Section 3.4.3, “Common Configuration Options Using nm-connection-editor”.
- Click the plus button. The Choose a Connection Type menu opens.
- Select from themenu entry, the
IPsec based VPN
option. - Click
Create
to open theEditing
dialog and proceed to the section called “Adding a New IPsec VPN Connection” to edit theGeneral
andAdvanced
sections.
3.4.9. Configuring a Mobile Broadband Connection with a GUI
- 2G — GPRS (General Packet Radio Service), EDGE (Enhanced Data Rates for GSM Evolution), or CDMA (Code Division Multiple Access).
- 3G — UMTS (Universal Mobile Telecommunications System), HSPA (High Speed Packet Access), or EVDO (EVolution Data-Only).
3.4.9.1. Configuring a Mobile Broadband Connection with nm-connection-editor
Adding a New Mobile Broadband Connection
Procedure
- Enter nm-connection-editor in a terminal. The Network Connections window appears, see Section 3.4.3, “Common Configuration Options Using nm-connection-editor”.
- Click the plus button. The Choose a Connection Type menu opens.
- Select themenu entry.
- Click Set up a Mobile Broadband Connection assistant.to open the
- Under Create a connection for this mobile broadband device, choose the 2G- or 3G-capable device you want to use with the connection. If the drop-down menu is inactive, this indicates that the system was unable to detect a device capable of mobile broadband. In this case, click Cancel, ensure that you do have a mobile broadband-capable device attached and recognized by the computer and then retry this procedure. Click the Continue button.
- Select the country where your service provider is located from the list and click the Continue button.
- Select your provider from the list or enter it manually. Click the Continue button.
- Select your payment plan from the drop-down menu and confirm the Access Point Name (APN) is correct. Click the Continue button.
- Review and confirm the settings and then click the Apply button.
- Edit the mobile broadband-specific settings by referring to the section called “Configuring the Mobile Broadband Tab”
Editing an Existing Mobile Broadband Connection
Procedure
- Enter
nm-connection-editor
in a terminal. The Network Connections window appears. - Select the Mobile Broadband tab.
- Select the connection you want to edit and click the gear wheel icon. See Section 3.4.3, “Common Configuration Options Using nm-connection-editor” for more information.
- Edit the mobile broadband-specific settings by referring to the section called “Configuring the Mobile Broadband Tab”
Configuring the Mobile Broadband Tab
- Number
- The number that is dialed to establish a PPP connection with the GSM-based mobile broadband network. This field may be automatically populated during the initial installation of the broadband device. You can usually leave this field blank and enter the APN instead.
- Username
- Enter the user name used to authenticate with the network. Some providers do not provide a user name, or accept any user name when connecting to the network.
- Password
- Enter the password used to authenticate with the network. Some providers do not provide a password, or accept any password.
- APN
- Enter the Access Point Name (APN) used to establish a connection with the GSM-based network. Entering the correct APN for a connection is important because it often determines:
- how the user is billed for their network usage;
- whether the user has access to the Internet, an intranet, or a subnetwork.
- Network ID
- Entering a Network ID causes NetworkManager to force the device to register only to a specific network. This can be used to ensure the connection does not roam when it is not possible to control roaming directly.
- Type
- Any — The default value of Any leaves the modem to select the fastest network.3G (UMTS/HSPA) — Force the connection to use only 3G network technologies.2G (GPRS/EDGE) — Force the connection to use only 2G network technologies.Prefer 3G (UMTS/HSPA) — First attempt to connect using a 3G technology such as HSPA or UMTS, and fall back to GPRS or EDGE only upon failure.Prefer 2G (GPRS/EDGE) — First attempt to connect using a 2G technology such as GPRS or EDGE, and fall back to HSPA or UMTS only upon failure.
- Allow roaming if home network is not available
- Uncheck this box if you want NetworkManager to terminate the connection rather than transition from the home network to a roaming one, thereby avoiding possible roaming charges. If the box is checked, NetworkManager will attempt to maintain a good connection by transitioning from the home network to a roaming one, and vice versa.
- PIN
- If your device's SIM (Subscriber Identity Module) is locked with a PIN (Personal Identification Number), enter the PIN so that NetworkManager can unlock the device. NetworkManager must unlock the SIM if a PIN is required in order to use the device for any purpose.
APN
, Network ID
, or Type
options.
Saving Your New (or Modified) Connection and Making Further Configurations
- Point-to-point settings for the connection, click the PPP Settings tab and proceed to Section 5.6, “Configuring PPP (Point-to-Point) Settings”;
IPv4
settings for the connection, click the IPv4 Settings tab and proceed to Section 5.4, “Configuring IPv4 Settings”; or,IPv6
settings for the connection, click the IPv6 Settings tab and proceed to Section 5.5, “Configuring IPv6 Settings”.
3.4.10. Configuring a DSL Connection with a GUI
3.4.10.1. Configuring a DSL Connection with nm-connection-editor
Adding a New DSL Connection
Procedure
- Enter nm-connection-editor in a terminal. The Network Connections window appears, see Section 3.4.3, “Common Configuration Options Using nm-connection-editor”.
- Click the plus button.
- The Choose a Connection Type list appears.
- Selectand press the button.
- The Editing DSL Connection 1 window appears.
Editing an Existing DSL Connection
Procedure
- Enter nm-connection-editor in a terminal. The Network Connections window appears.
- Select the connection you want to edit and click the gear wheel icon. See Section 3.4.3, “Common Configuration Options Using nm-connection-editor” for more information.
Configuring the DSL Tab
- Username
- Enter the user name used to authenticate with the service provider.
- Service
- Leave blank unless otherwise directed by your service provider.
- Password
- Enter the password supplied by the service provider.
Saving Your New (or Modified) Connection and Making Further Configurations
- The MAC address and MTU settings, click the Wired tab and proceed to the section called “Basic Configuration Options ”.
- Point-to-point settings for the connection, click the PPP Settings tab and proceed to Section 5.6, “Configuring PPP (Point-to-Point) Settings”.
IPv4
settings for the connection, click the IPv4 Settings tab and proceed to Section 5.4, “Configuring IPv4 Settings”.
3.5. Configuring IP Networking with ifcfg Files
ifcfg
files.
ifcfg-name
, where the suffix name refers to the name of the device that the configuration file controls. By convention, the ifcfg
file's suffix is the same as the string given by the DEVICE
directive in the configuration file itself.
Configuring an Interface with Static Network Settings Using ifcfg Files
ifcfg
files, for an interface with the name enp1s0
, create a file with the name ifcfg-enp1s0
in the /etc/sysconfig/network-scripts/
directory, that contains:
- For
IPv4
configurationDEVICE=enp1s0 BOOTPROTO=none ONBOOT=yes PREFIX=24 IPADDR=10.0.1.27
- For
IPv6
configurationDEVICE=enp1s0 BOOTPROTO=none ONBOOT=yes IPV6INIT=yes IPV6ADDR=2001:db8::2/48
You do not need to specify the network or broadcast address as this is calculated automatically by ipcalc.For moreIPv6
ifcfg configuration options, see nm-settings-ifcfg-rh(5) man page.
Important
HWADDR
directive can influence the device naming procedure.
Configuring an Interface with Dynamic Network Settings Using ifcfg Files
ifcfg
files:
- Create a file with the name
ifcfg-em1
in the/etc/sysconfig/network-scripts/
directory, that contains:DEVICE=em1 BOOTPROTO=dhcp ONBOOT=yes
- To configure an interface to send a different host name to the
DHCP
server, add the following line to theifcfg
file:DHCP_HOSTNAME=hostname
To configure an interface to send a different fully qualified domain name (FQDN) to theDHCP
server, add the following line to theifcfg
file:DHCP_FQDN=fully.qualified.domain.name
Note
Only one directive, eitherDHCP_HOSTNAME
orDHCP_FQDN
, should be used in a givenifcfg
file. In case bothDHCP_HOSTNAME
andDHCP_FQDN
are specified, only the latter is used. - To configure an interface to use particular
DNS
servers, add the following lines to theifcfg
file:PEERDNS=no DNS1=ip-address DNS2=ip-address
where ip-address is the address of aDNS
server. This will cause the network service to update/etc/resolv.conf
with the specifiedDNS
servers specified. Only oneDNS
server address is necessary, the other is optional. - To configure static routes in the
ifcfg
file, see Section 4.5, “Configuring Static Routes in ifcfg files”.By default, NetworkManager calls theDHCP
client, dhclient, when a profile has been set to obtain addresses automatically by settingBOOTPROTO
todhcp
in an interface configuration file. IfDHCP
is required, an instance of dhclient is started for every Internet protocol,IPv4
andIPv6
, on an interface. If NetworkManager is not running, or is not managing an interface, then the legacy network service will call instances of dhclient as required. For more details on dynamic IP addresses, see Section 1.2, “Comparing Static to Dynamic IP Addressing”. - To apply the configuration:
- Reload the updated connection files:
# nmcli connection reload
- Re-activate the connection:
# nmcli connection up connection_name
3.5.1. Managing System-wide and Private Connection Profiles with ifcfg Files
USERS
directive in the ifcfg
files. If the USERS
directive is not present, the network profile will be available to all users. As an example, the following command in an ifcfg
file will make the connection available only to the users listed: USERS="joe bob alice"
USERCTL
directive to manage the device:
- If you set yes, non-
root
users are allowed to control this device. - If you set no, non-
root
users are not allowed to control this device.
3.6. Configuring IP Networking with ip Commands
ip
command, but but changes are not persistent across reboots; when you reboot, you will lose any changes.
man ip(8)
page. The package name in Red Hat Enterprise Linux 7 is iproute. If necessary, you can check that the ip utility is installed by checking its version number as follows:
~]$ ip -V
ip utility, iproute2-ss130716
ip link set ifname down
Note
ip link set ifname
command sets a network interface in IFF_UP
state and enables it from the kernel's scope. This is different from the ifup ifname
command for initscripts or NetworkManager's activation state of a device. In fact, NetworkManager always sets an interface up even if it is currently disconnected. Disconnecting the device through the nmcli tool, does not remove the IFF_UP
flag. In this way, NetworkManager gets notifications about the carrier state.
ifconfig
utility because the net-tools package (which provides ifconfig
) does not support InfiniBand addresses.
ip help
command. For example: ip link help
and ip addr help
.
Note
ifcfg
files) or add the commands to a script.
IP
addresses to an interface with the following form: ip addr [ add | del ] address dev ifname
Assigning a Static Address Using ip Commands
IP
address to an interface:
~]#Further examples and command options can be found in theip address add 10.0.0.3/24 dev enp1s0
You can view the address assignment of a specific device: ~]#ip addr show dev enp1s0
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether f0:de:f1:7b:6e:5f brd ff:ff:ff:ff:ff:ff inet 10.0.0.3/24 brd 10.0.0.255 scope global global enp1s0 valid_lft 58682sec preferred_lft 58682sec inet6 fe80::f2de:f1ff:fe7b:6e5f/64 scope link valid_lft forever preferred_lft forever
ip-address(8)
manual page.
Configuring Multiple Addresses Using ip Commands
~]#ip address add 192.168.2.223/24 dev enp1s0
~]#ip address add 192.168.4.223/24 dev enp1s0
~]#ip addr
3: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:fb:77:9e brd ff:ff:ff:ff:ff:ff inet 192.168.2.223/24 scope global enp1s0 inet 192.168.4.223/24 scope global enp1s0
ip(8)
manual page.
Note
3.7. Configuring IP Networking from the Kernel Command line
- Install the dracut utility. For information on using dracut, see Red Hat Enterprise Linux System Administrator's Guide
- Set the configuration using the
ip
option on the kernel command line:ip<client-IP-number>:[<server-id>]:<gateway-IP-number>:<netmask>:<client-hostname>:<interface>:{dhcp|dhcp6|auto6|on|any|none|off}
For example:dhcp
- DHCP configurationdhpc6
- DHCP IPv6 configurationauto6
- automatic IPv6 configurationon
,any
- any protocol available in the kernel (default)none
,off
- no autoconfiguration, static network configuration
ip=192.168.180.120:192.168.180.100:192.168.180.1:255.255.255.0::enp1s0:off
- Set the name server configuration:
nameserver=srv1 [nameserver=srv2 [nameserver=srv3 […]]]
ifcfg
files that can be copied to the /etc/sysconfig/network-scripts/
file.
3.8. Enabling IP Multicast with IGMP
ip maddr show
subcommand, for example:
~]$ ip maddr show dev br0
8: br0
inet 224.0.0.1
inet6 ff02::1
inet6 ff01::1
[output truncated]
MULTICAST
string in the ip link show
command output, for example:
~]$ ip link show br0
8: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000
link/ether 6c:0b:84:67:fe:63 brd ff:ff:ff:ff:ff:ff
~]#ip link set multicast off dev br0
~]$ip link show br0
8: br0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000 link/ether 6c:0b:84:67:fe:63 brd ff:ff:ff:ff:ff:ff
MULTICAST
string indicates that multicast is disabled.
~]#ip link set multicast on dev br0
~]$ip link show br0
8: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000 link/ether 6c:0b:84:67:fe:63 brd ff:ff:ff:ff:ff:ff
ip(8)
man page for more information.
/proc/net/igmp
file:
~]$ cat /proc/net/igmp
Note
firewalld
by default. To enable IGMP for a zone:
~]# firewall-cmd --zone=zone-name --add-protocol=igmp
3.9. Additional Resources
Installed Documentation
- ip(8) man page — Describes the ip utility's command syntax.
- nmcli(1) man page — Describes NetworkManager's command‐line tool.
- nmcli-examples(5) man page — Gives examples of nmcli commands.
- nm-settings(5) man page — Describes NetworkManager properties and their settings.
- nm-settings-ifcfg-rh(5) man page — Describes ifcfg-rh settings plug-in.
Online Documentation
- Red Hat Enterprise Linux 7 Security Guide
- Describes
IPsec
based VPN and its configuration. Describes the use of authenticatedDNS
queries using DNSSEC. - RFC 1518 — Classless Inter-Domain Routing (CIDR)
- Describes the CIDR Address Assignment and Aggregation Strategy, including variable-length subnetting.
- RFC 1918 — Address Allocation for Private Internets
- Describes the range of
IPv4
addresses reserved for private use. - RFC 3330 — Special-Use IPv4 Addresses
- Describes the global and other specialized
IPv4
address blocks that have been assigned by the Internet Assigned Numbers Authority (IANA).
Chapter 4. Configuring Static Routes and the Default Gateway
4.1. Introduction to Understanding Routing and Gateway
DHCP
, an address of a gateway that leads to an upstream network or the Internet is usually assigned. This gateway is usually referred to as the default gateway as it is the gateway to use if no better route is known to the system (and present in the routing table). Network administrators often use the first or last host IP
address in the network as the gateway address; for example, 192.168.10.1
or 192.168.10.254
. Not to be confused by the address which represents the network itself; in this example, 192.168.10.0
, or the subnet's broadcast address; in this example 192.168.10.255
. The default gateway is traditionally a network router. The default gateway is for any and all traffic which is not destined for the local network and for which no preferred route is specified in the routing table.
Note
4.2. Configuring Static Routes Using nmcli
- the nmcli command line
- the nmcli interactive editor
Example 4.1. Configuring Static Routes Using nmcli
~]# nmcli connection modify enp1s0 +ipv4.routes "192.168.122.0/24 10.10.10.1"
This will direct traffic for the 192.168.122.0/24
subnet to the gateway at 10.10.10.1
Example 4.2. Configuring Static Routes with nmcli Editor
~]$ nmcli con edit ens3
===| nmcli interactive connection editor |===
Editing existing '802-3-ethernet' connection: 'ens3'
Type 'help' or '?' for available commands.
Type 'describe [<setting>.<prop>]' for detailed property description.
You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, dcb, ipv4, ipv6, tc, proxy
nmcli> set ipv4.routes 192.168.122.0/24 10.10.10.1
nmcli> save persistent
Connection 'ens3' (23f8b65a-8f3d-41a0-a525-e3bc93be83b8) successfully updated.
nmcli> quit
4.3. Configuring Static Routes with GUI
- Routes
IP
address of a remote network, sub-net, or host.IP
address entered above.IP
address of the gateway leading to the remote network, sub-net, or host entered above.- When Automatic is ON, routes from
RA
orDHCP
are used, but you can also add additional static routes. When OFF, only static routes you define are used. - Use this connection only for resources on its network
- Select this check box to prevent the connection from becoming the default route. Typical examples are where a connection is a VPN tunnel or a leased line to a head office and you do not want any Internet-bound traffic to pass over the connection. Selecting this option means that only traffic specifically destined for routes learned automatically over the connection or entered here manually will be routed over the connection.
4.4. Configuring Static Routes with ip commands
ip route
command.
IP
routing table, use the ip route
command. For example:
~]$ ip route default via 192.168.122.1 dev ens9 proto static metric 1024 192.168.122.0/24 dev ens9 proto kernel scope link src 192.168.122.107 192.168.122.0/24 dev enp1s0 proto kernel scope link src 192.168.122.126
ip route
commands take the following form: ip route [ add | del | change | append | replace ] destination-address
See the ip-route(8)
man page for more details on the options and formats.
~]#ip route add
192.0.2.1 via 10.0.0.1 [dev
interface]
IP
address of the host in dotted decimal notation, 10.0.0.1 is the next hop address and interface is the exit interface leading to the next hop.
IP
address representing a range of IP
addresses:
~]#where 192.0.2.0 is theip route add
192.0.2.0/24 via 10.0.0.1 [dev
interface]
IP
address of the destination network in dotted decimal notation and /24 is the network prefix. The network prefix is the number of enabled bits in the subnet mask. This format of network address slash network prefix length is sometimes referred to as classless inter-domain routing (CIDR) notation.
~]# ip route del
192.0.2.1
ip route
do not persist across system reboots. To permanently configure static routes, you can configure them by creating a route-interface
file in the /etc/sysconfig/network-scripts/
directory for the interface. For example, static routes for the enp1s0 interface would be stored in the /etc/sysconfig/network-scripts/route-enp1s0
file. Any changes that you make to a route-interface
file do not take effect until you restart either the network service or the interface. The route-interface
file has two formats:
- ip command arguments, see the section called “Static Routes Using the IP Command Arguments Format”.and
- network/netmask directives, see the section called “Static Routes Using the Network/Netmask Directives Format”.
ip-route(8)
man page for more information on the ip route
command.
4.5. Configuring Static Routes in ifcfg files
/etc/sysconfig/network-scripts/
directory. The file name should be of the format route-interface
. There are two types of commands to use in the configuration files:
Static Routes Using the IP Command Arguments Format
/etc/sysconfig/network-scripts/route-enp1s0
, define a route to a default gateway on the first line. This is only required if the gateway is not set through DHCP
and is not set globally in the /etc/sysconfig/network
file:
default via 192.168.1.1 dev
interface
IP
address of the default gateway. The interface is the interface that is connected to, or can reach, the default gateway. The dev
option can be omitted, it is optional. Note that this setting takes precedence over a setting in the /etc/sysconfig/network
file.
10.10.10.0/24 via 192.168.1.1 [dev
interface]
IP
address leading to the remote network. It is preferably the next hop address but the address of the exit interface will work. The “next hop” means the remote end of a link, for example a gateway or router. The dev
option can be used to specify the exit interface interface but it is not required. Add as many static routes as required.
route-interface
file using the ip command arguments format. The default gateway is 192.168.0.1
, interface enp1s0 and a leased line or WAN connection is available at 192.168.0.10
. The two static routes are for reaching the 10.10.10.0/24
network and the 172.16.1.10/32
host:
default via 192.168.0.1 dev enp1s0 10.10.10.0/24 via 192.168.0.10 dev enp1s0 172.16.1.10/32 via 192.168.0.10 dev enp1s0
192.168.0.0/24
network will be directed out the interface attached to that network. Packets going to the 10.10.10.0/24
network and 172.16.1.10/32
host will be directed to 192.168.0.10
. Packets to unknown, remote, networks will use the default gateway therefore static routes should only be configured for remote networks or hosts if the default route is not suitable. Remote in this context means any networks or hosts that are not directly attached to the system.
IPv6
configuration, an example of a route6-interface
file in ip route
format:
2001:db8:1::/48 via 2001:db8::1 metric 2048 2001:db8:2::/48
ip route
format can be used to specify a source address. For example:
10.10.10.0/24 via 192.168.0.10 src 192.168.0.2
Important
DHCP
and if the same gateway with the same metric is specified in a configuration file, an error during start-up, or when bringing up an interface, will occur. The follow error message may be shown: "RTNETLINK answers: File exists". This error may be ignored.
Static Routes Using the Network/Netmask Directives Format
route-interface
files. The following is a template for the network/netmask format, with instructions following afterwards:
ADDRESS0=10.10.10.0 NETMASK0=255.255.255.0 GATEWAY0=192.168.1.1
ADDRESS0=10.10.10.0
is the network address of the remote network or host to be reached.NETMASK0=255.255.255.0
is the netmask for the network address defined withADDRESS0=10.10.10.0
.GATEWAY0=192.168.1.1
is the default gateway, or anIP
address that can be used to reachADDRESS0=10.10.10.0
route-interface
file using the network/netmask directives format. The default gateway is 192.168.0.1
but a leased line or WAN connection is available at 192.168.0.10
. The two static routes are for reaching the 10.10.10.0/24
and 172.16.1.0/24
networks:
ADDRESS0=10.10.10.0 NETMASK0=255.255.255.0 GATEWAY0=192.168.0.10 ADDRESS1=172.16.1.10 NETMASK1=255.255.255.0 GATEWAY1=192.168.0.10
ADDRESS0
, ADDRESS1
, ADDRESS2
, and so on.
4.5.1. Understanding Policy-routing
Policy-routing
also known as source-routing, is a mechanism for more flexible routing configurations. Routing decisions are commonly made based on the destination IP address of a package. Policy-routing
allows more flexibility to select routes based on other routing properties, such as source IP address, source port, protocol type. Routing tables stores route information about networks. They are identified by either numeric values or names, which can be configured in the /etc/iproute2/rt_tables
file. The default table is identified with 254
. Using policy-routing
, you also need rules. Rules are used to select a routing table, based on certain properties of packets.
ip route
format can be used to define an existing policy-based routing configuration, which specifies multiple routing tables:
10.10.10.0/24 via 192.168.0.10 table 1 10.10.10.0/24 via 192.168.0.10 table 2
/etc/sysconfig/network-scripts/rule-enp1s0
file for IPv4
or to the /etc/sysconfig/network-scripts/rule6-enp1s0
file for IPv6
.
ipv4.route-table
forIPv4
andipv6.route-table
forIPv6
.
DHCP
, autoconf6
, DHCP6
are placed in that specific table. In addition, all routes for subnets that have already configured addresses, are placed in the corresponding routing table. For example, if you configure the 192.168.1.10/24 address, the 192.168.1.0/24 subnet is contained in ipv4.route-table.
policy-routing
rules, see the ip-rule(8)
man page. For routing tables, see the ip-route(8)
man page.
4.6. Configuring the Default Gateway
/etc/sysconfig/network
file first and then the network interface ifcfg
files for interfaces that are “up”. The ifcfg
files are parsed in numerically ascending order, and the last GATEWAY directive to be read is used to compose a default route in the routing table.
/etc/sysconfig/network
file is deprecated, and specifying the gateway should now only be done in per-interface configuration files.
DHCP
. In special cases where it is necessary to influence NetworkManager's selection of the exit interface to be used to reach a gateway, make use of the DEFROUTE=no
command in the ifcfg
files for those interfaces which do not lead to the default gateway.
Chapter 5. Configuring Network Connection Settings
5.1. Configuring 802.3 Link Settings
802-3-ethernet.auto-negotiate
802-3-ethernet.speed
802-3-ethernet.duplex
- Ignore link negotiation
- Enforce auto-negotiation activation
- Manually set the
speed
andduplex
link settings
Ignoring link negotiation
802-3-ethernet.auto-negotiate = no 802-3-ethernet.speed = 0 802-3-ethernet.duplex = NULL
Important
auto-negotiate
parameter is set to no
, but the speed
and duplex
values are not set, that does not mean that auto-negotiation is disabled.
Enforcing auto-negotiation activation
802-3-ethernet.auto-negotiate = yes 802-3-ethernet.speed = 0 802-3-ethernet.duplex = NULL
Manually setting the link speed and duplex
speed
and duplex
settings on the link.
speed
and duplex
link settings, set the aforementioned parameters as follows:
802-3-ethernet.auto-negotiate = no 802-3-ethernet.speed = [speed in Mbit/s] 802-3-ethernet.duplex = [half |full]
Important
speed
and the duplex
values, otherwise NetworkManager does not update the link configuration.
- the nmcli tool
- the nm-connection-editor utility
Configuring 802.3 Link Settings with the nmcli Tool
Procedure
- Create a new ethernet connection for the enp1s0 device.
- Set the 802.3 link setting to a configuration of your choice. For details, see Section 5.1, “Configuring 802.3 Link Settings”For example, to manually set the
speed
option 100 Mbit/s andduplex
to full:nmcli connection add con-name MyEthernet type ethernet ifname enp1s0 \ 802-3-ethernet.auto-negotiate no \ 802-3-ethernet.speed 100 \ 802-3-ethernet.duplex full
Configuring 802.3 Link Settings with nm-connection-editor
Procedure
- Enter nm-connection-editor in a terminal.
- Select the ethernet connection you want to edit and click the gear wheel icon to move to the editing dialog. See Section 3.4.3, “Common Configuration Options Using nm-connection-editor” for more information.
- Select the link negotiation of your choice.
Ignore
: link configuration is skipped (default).Automatic
: link auto-negotiation is enforced on the device.Manual
: theSpeed
andDuplex
options can be specified to enforce the link negotiation.
Figure 5.1. Configure 802.3 link settings using nm-connection-editor
5.2. Configuring 802.1X Security
DHCP
servers were configured not to lease IP
addresses to unauthorized users, but for various reasons this practice is both impractical and insecure, and thus is no longer recommended. Instead, 802.1X security is used to ensure a logically-secure network through port-based authentication.
5.2.1. Configuring 802.1X Security for Wi-Fi with nmcli
Procedure
- Set the authenticated
key-mgmt
(key management) protocol. It configures the keying mechanism for a securewifi
connection. See the nm-settings(5) man page for more details on properties. - Configure the 802-1x authentication settings. For the Transport Layer Security (TLS) authentication, see the section called “Configuring TLS Settings”.
802-1x authentication setting | Name | |
---|---|---|
802-1x.identity | Identity | |
802-1x.ca-cert | CA certificate | |
802-1x.client-cert | User certificate | |
802-1x.private-key | Private key | |
802-1x.private-key-password | Private key password |
nmcli c add type wifi ifname wlo61s0 con-name 'My Wifi Network' \ 802-11-wireless.ssid 'My Wifi' \ 802-11-wireless-security.key-mgmt wpa-eap \ 802-1x.eap tls \ 802-1x.identity identity@example.com \ 802-1x.ca-cert /etc/pki/my-wifi/ca.crt \ 802-1x.client-cert /etc/pki/my-wifi/client.crt \ 802-1x.private-key /etc/pki/my-wifi/client.key \ 802-1x.private-key-password s3cr3t
5.2.2. Configuring 802.1X Security for Wired with nmcli
wired
connection using the nmcli tool, follow the same procedure as for a wireless
connection, except the 802-11-wireless.ssid
and 802-11-wireless-security.key-mgmt
settings.
5.2.3. Configuring 802.1X Security for Wi-Fi with a GUI
Procedure
- Open the Network window (see Section 3.4.1, “Connecting to a Network Using the control-center GUI”).
- Select a Wireless network interface from the right-hand-side menu. If necessary, set the symbolic power button to ON and check that your hardware switch is on.
- Either select the connection name of a new connection, or click the gear wheel icon of an existing connection profile, for which you want to configure 802.1X security. In the case of a new connection, complete any authentication steps to complete the connection and then click the gear wheel icon.
- Select Security.The following configuration options are available:
- Security
- Password
- Enter the password to be used in the authentication process.
- From the drop-down menu select one of the following security methods:, , or .
5.2.4. Configuring 802.1X Security for Wired with nm-connection-editor
Procedure
- Enter the nm-connection-editor in a terminal.
~]$ nm-connection-editor
TheNetwork Connections
window appears. - Select the ethernet connection you want to edit and click the gear wheel icon, see Section 3.4.6.2, “Configuring a Wired Connection with nm-connection-editor”.
- Select Security and set the symbolic power button to ON to enable settings configuration.
- Select from one of following authentication methods:
- Select TLS for Transport Layer Security and proceed to the section called “Configuring TLS Settings”;
- Select FAST for Flexible Authentication through Secure Tunneling and proceed to the section called “Configuring Tunneled TLS Settings”;
- Select Tunneled TLS for Tunneled Transport Layer Security, otherwise known as TTLS, or EAP-TTLS and proceed to the section called “Configuring Tunneled TLS Settings”;
- Select Protected EAP (PEAP) for Protected Extensible Authentication Protocol and proceed to the section called “Configuring Protected EAP (PEAP) Settings”.
Configuring TLS Settings
- Identity
- Provide the identity of this server.
- User certificate
- Click to browse for, and select, a personal X.509 certificate file encoded with Distinguished Encoding Rules (DER) or Privacy Enhanced Mail (PEM).
- CA certificate
- Click to browse for, and select, an X.509 certificate authority certificate file encoded with Distinguished Encoding Rules (DER) or Privacy Enhanced Mail (PEM).
- Private key
- Click to browse for, and select, a private key file encoded with Distinguished Encoding Rules (DER), Privacy Enhanced Mail (PEM), or the Personal Information Exchange Syntax Standard (PKCS #12).
- Private key password
- Enter the password for the private key in the Private key field. Select Show password to make the password visible as you type it.
Configuring FAST Settings
- Anonymous Identity
- Provide the identity of this server.
- PAC provisioning
- Select the check box to enable and then select from, , and .
- PAC file
- Click to browse for, and select, a protected access credential (PAC) file.
- Inner authentication
- Username
- Enter the user name to be used in the authentication process.
- Password
- Enter the password to be used in the authentication process.
Configuring Tunneled TLS Settings
- Anonymous identity
- This value is used as the unencrypted identity.
- CA certificate
- Click to browse for, and select, a Certificate Authority's certificate.
- Inner authentication
- Username
- Enter the user name to be used in the authentication process.
- Password
- Enter the password to be used in the authentication process.
Configuring Protected EAP (PEAP) Settings
- Anonymous Identity
- This value is used as the unencrypted identity.
- CA certificate
- Click to browse for, and select, a Certificate Authority's certificate.
- PEAP version
- The version of Protected EAP to use. Automatic, 0 or 1.
- Inner authentication
- Username
- Enter the user name to be used in the authentication process.
- Password
- Enter the password to be used in the authentication process.
5.3. Using MACsec
with wpa_supplicant and NetworkManager
Media Access Control Security
(MACsec
, IEEE 802.1AE) encrypts and authenticates all traffic in LANs with the GCM-AES-128 algorithm. MACsec
can protect not only IP
but also Address Resolution Protocol (ARP), Neighbor Discovery (ND), or DHCP
. While IPsec
operates on the network layer (layer 3) and SSL
or TLS
on the application layer (layer 7), MACsec
operates in the data link layer (layer 2). Combine MACsec
with security protocols for other networking layers to take advantage of different security features that these standards provide.
MACsec
with a switch that performs authentication using a pre-shared Connectivity Association Key/CAK Name (CAK/CKN) pair:
Procedure
- Create a CAK/CKN pair. For example, the following command generates a 16-byte key in hexadecimal notation:
~]$
dd if=/dev/urandom count=16 bs=1 2> /dev/null | hexdump -e '1/2 "%02x"'
- Create the
wpa_supplicant.conf
configuration file and add the following lines to it:ctrl_interface=/var/run/wpa_supplicant eapol_version=3 ap_scan=0 fast_reauth=1 network={ key_mgmt=NONE eapol_flags=0 macsec_policy=1 mka_cak=0011... # 16 bytes hexadecimal mka_ckn=2233... # 32 bytes hexadecimal }
Use the values from the previous step to complete themka_cak
andmka_ckn
lines in thewpa_supplicant.conf
configuration file.See thewpa_supplicant.conf(5)
man page for more information. - Assuming you are using wlp61s0 to connect to your network, start wpa_supplicant using the following command:
~]#
wpa_supplicant -i wlp61s0 -Dmacsec_linux -c wpa_supplicant.conf
wpa_supplicant.conf
file, Red Hat recommends using the nmcli
command to configure wpa_supplicant equivalently as in the previous steps. The following example assumes that you already have a 16-byte hexadecimal CAK ($MKA_CAK
) and a 32-byte hexadecimal CKN ($MKA_CKN
):
~]# nmcli connection add type macsec \ con-name test-macsec+ ifname macsec0 \ connection.autoconnect no \ macsec.parent wlp61s0 macsec.mode psk \ macsec.mka-cak $MKA_CAK \ macsec.mka-cak-flags 0 \ macsec.mka-ckn $MKA_CKN ~]# nmcli connection up test-macsec+
MACsec
network, use case scenarios, and configuration examples.
5.4. Configuring IPv4 Settings
Configuring IPv4 Settings with control-center
Procedure
- Press the Super key to enter the Activities Overview, type Settings and then press Enter. Then, select the
Network
tab on the left-hand side, and the Network settings tool appears. Proceed to the section called “Configuring New Connections with control-center”. - Select the connection that you want to edit and click on the gear wheel icon. The
Editing
dialog appears. - Click the IPv4 menu entry.
IP
address,DNS
and route information as required. The IPv4 menu entry is available when you create and modify one of the following connection types: wired, wireless, mobile broadband, VPN or DSL.
DHCP
to obtain a dynamic IP
address from a DHCP
server, you can simply set Addresses to .
Setting the Method for IPV4 Using nm-connection-editor
IPv4
settings:
Procedure
- Enter nm-connection-editor in a terminal.
- For an existing connection type, click the gear wheel icon.
Figure 5.2. Editing a connection
- Click IPv4 Settings.
Figure 5.3. Configuring IPv4 Settings
Available IPv4 Methods by Connection Type
IPv4
connection methods. All of the methods are listed here according to which connection type, or types, they are associated with:
- Wired, Wireless and DSL Connection Methods
DHCP
server to assignIP
addresses. You do not need to fill in the DHCP client ID field.DHCP
server to assignIP
addresses but you want to assignDNS
servers manually.IP
addresses manually.DHCP
server and you do not want to assignIP
addresses manually. Random addresses will be assigned as per RFC 3927 with prefix169.254/16
.10.42.x.1/24
range, aDHCP
server andDNS
server are started, and the interface is connected to the default network connection on the system with network address translation (NAT).IPv4
is disabled for this connection.- Mobile Broadband Connection Methods
IP
address andDNS
servers automatically.IP
address automatically, but you want to manually specifyDNS
servers.- VPN Connection Methods
IP
address andDNS
servers automatically.IP
address automatically, but you want to manually specifyDNS
servers.- DSL Connection Methods
IP
address andDNS
servers automatically.IP
address automatically, but you want to manually specifyDNS
servers.
DHCP
to obtain a dynamic IP
address from a DHCP
server, you can simply set Method to .
5.5. Configuring IPv6 Settings
IPv6
settings, follow the procedure described in Section 5.4, “Configuring IPv4 Settings” and click the IPv6
menu entry.
- Method
IPv6
settings for this connection.DNS
servers manually.DHCPv6
directly to create a stateful configuration.IP
addresses manually.DHCP
server and you do not want to assignIP
addresses manually. Random addresses will be assigned as per RFC 4862 with prefixFE80::0
.- Addresses
DNS
servers.
5.6. Configuring PPP (Point-to-Point) Settings
Authentication Methods
- Use point-to-point encryption (MPPE)
- Microsoft Point-To-Point Encryption protocol (RFC 3078).
- Allow BSD data compression
- PPP BSD Compression Protocol (RFC 1977).
- Allow Deflate data compression
- PPP Deflate Protocol (RFC 1979).
- Use TCP header compression
- Compressing TCP/IP Headers for Low-Speed Serial Links (RFC 1144).
- Send PPP echo packets
- LCP Echo-Request and Echo-Reply Codes for loopback tests (RFC 1661).
Note
Chapter 6. Configure Host Names
6.1. Understanding Host Names
hostname
: static, pretty, and transient.
hostname
, which can be chosen by the user, and is stored in the /etc/hostname
file. The “transient” hostname
is a dynamic host name maintained by the kernel. It is initialized to the static host name by default, whose value defaults to “localhost”. It can be changed by DHCP
or mDNS
at runtime. The “pretty” hostname
is a free-form UTF8 host name for presentation to the user.
Note
DNS
, such as host.example.com
. It is also recommended that the static and transient names consists only of 7 bit ASCII lower-case characters, no spaces or dots, and limits itself to the format allowed for DNS
domain name labels, even though this is not a strict requirement. Older specifications do not permit the underscore, and so their use is not recommended.
a-z
, A-Z
, 0-9
, “-
”, “_
” and “.
” only, to not begin or end in a dot, and to not have two dots immediately following each other. The size limit of 64 characters is enforced.
6.1.1. Recommended Naming Practices
.yourcompany
) to the public register. Therefore, Red Hat strongly recommends that you do not use a domain name that is not delegated to you, even on a private network, as this can result in a domain name that resolves differently depending on network configuration. As a result, network resources can become unavailable. Using domain names that are not delegated to you also makes DNSSEC more difficult to deploy and maintain, as domain name collisions require manual configuration to enable DNSSEC validation. See the ICANN FAQ on domain name collision for more information on this issue.
6.2. Configuring Host Names Using Text User Interface, nmtui
~]$ nmtui
The text user interface appears. Any invalid command prints a usage message.
Figure 6.1. The NetworkManager Text User Interface starting menu
/etc/hostname
file.
Important
systemd-hostnamed
service to read and write the static host name, which is stored in the /etc/hostname
file. Due to this, manual modifications done to the /etc/hostname
file are no longer picked up automatically by NetworkManager; you should change the system host name through the hostnamectl utility. Also, the use of the HOSTNAME
variable in the /etc/sysconfig/network
file is now deprecated.
6.3. Configuring Host Names Using hostnamectl
6.3.1. View All the Host Names
~]$ hostnamectl status
The status
option is implied by default if no option is given.
6.3.2. Set All the Host Names
root
:
~]# hostnamectl set-hostname name
This will alter the pretty, static, and transient host names alike. The static and transient host names will be simplified forms of the pretty host name. Spaces will be replaced with “-
” and special characters will be removed.
6.3.3. Set a Particular Host Name
root
with the relevant option:
~]# hostnamectl set-hostname name [option...]
Where option is one or more of: --pretty
, --static
, and --transient
.
--static
or --transient
options are used together with the --pretty
option, the static and transient host names will be simplified forms of the pretty host name. Spaces will be replaced with “-
” and special characters will be removed. If the --pretty
option is not given, no simplification takes place.
~]# hostnamectl set-hostname "Stephen's notebook" --pretty
6.3.4. Clear a Particular Host Name
root
with the relevant option:
~]# hostnamectl set-hostname "" [option...]
Where "" is a quoted empty string and where option is one or more of: --pretty
, --static
, and --transient
.
6.3.5. Changing Host Names Remotely
hostnamectl
command on a remote system, use the -H, --host
option as follows:
~]# hostnamectl set-hostname -H
[username]@hostname
Where hostname is the remote host you want to configure. The username is optional. The hostnamectl tool will use SSH
to connect to the remote system.
6.4. Configuring Host Names Using nmcli
/etc/hostname
file.
~]$ nmcli general hostname
To set the static host name to my-server, issue the following command as root
:
~]# nmcli general hostname my-server
6.5. Additional Resources
hostnamectl(1)
man page — Describes hostnamectl including the commands and command options.hostname(1)
man page — Contains an explanation of thehostname
anddomainname
commands.hostname(5)
man page — Contains an explanation of the host name file, its contents, and use.hostname(7)
man page — Contains an explanation of host name resolution.machine-info(5)
man page — Describes the local machine information file and the environment variables it contains.machine-id(5)
man page — Describes the local machine ID configuration file.systemd-hostnamed.service(8)
man page — Describes thesystemd-hostnamed
system service used by hostnamectl.
Chapter 7. Configure Network Bonding
Warning
Note
7.1. Understanding the Default Behavior of Controller and Port Interfaces
NetworkManager
daemon, and especially when fault finding, keep the following in mind:
- Starting the controller interface does not automatically start the port interfaces.
- Starting a port interface always starts the controller interface.
- Stopping the controller interface also stops the port interfaces.
- A controller without ports can start static
IP
connections. - A controller without ports waits for ports when starting
DHCP
connections. - A controller with a
DHCP
connection waiting for ports completes when a port with a carrier is added. - A controller with a
DHCP
connection waiting for ports continues waiting when a port without a carrier is added.
7.2. Configure Bonding Using the Text User Interface, nmtui
~]$ nmtui
The text user interface appears. Any invalid command prints a usage message.
- From the starting menu, select Edit a connection. Select Add, the New Connection screen opens.
Figure 7.1. The NetworkManager Text User Interface Add a Bond Connection menu
- Select Bond and then Create; the Edit connection screen for the bond will open.
Figure 7.2. The NetworkManager Text User Interface Configuring a Bond Connection menu
- At this point port interfaces will need to be added to the bond; to add these select Add, the New Connection screen opens. Once the type of Connection has been chosen select the Create button.
Figure 7.3. The NetworkManager Text User Interface Configuring a New Bond Slave Connection menu
- The port's Edit Connection display appears; enter the required port's device name or MAC address in the Device section. If required, enter a clone MAC address to be used as the bond's MAC address by selecting Show to the right of the Ethernet label. Select the OK button to save the port.
Note
If the device is specified without a MAC address the Device section will be automatically populated once the Edit Connection window is reloaded, but only if it successfully finds the device.Figure 7.4. The NetworkManager Text User Interface Configuring a Bond Slave Connection menu
- The name of the bond port appears in the Slaves section. Repeat the above steps to add further port connections.
- Review and confirm the settings before selecting the OK button.
Figure 7.5. The NetworkManager Text User Interface Completed Bond
7.3. Network Bonding Using the NetworkManager Command Line Tool, nmcli
Note
bond
connection with the nmcli tool, issue the following command:
~]$ nmcli con add type bond ifname mybond0
Connection 'bond-mybond0' (5f739690-47e8-444b-9620-1895316a28ba) successfully added.
con-name
was given for the bond, the connection name was derived from the interface name by prepending the type.
~]$ nmcli con add type bond ifname mybond0 bond.options "mode=balance-rr,miimon=100"
Connection 'bond-mybond0' (5f739690-47e8-444b-9620-1895316a28ba) successfully added.
- Create a new connection, see Section 3.3.5, “Creating and Modifying a Connection Profile with nmcli” for details.
- Set the controller property to the
bond
interface name, or to the name of the controller connection:
~]$ nmcli con add type ethernet ifname ens3 master mybond0
Connection 'bond-slave-ens3' (220f99c6-ee0a-42a1-820e-454cbabc2618) successfully added.
port
interface, repeat the previous command with the new interface. For example:
~]$ nmcli con add type ethernet ifname ens7 master mybond0
Connection 'bond-slave-ens7' (ecc24c75-1c89-401f-90c8-9706531e0231) successfully added.
~]$ nmcli con up bond-slave-ens7
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/14)
~]$ nmcli con up bond-slave-ens3
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/15)
active_slave
option and the primary
option of the bond at runtime, without deactivating the connection. For example to change the active_slave
option, issue the following command:
~]$ nmcli dev mod bond0 +bond.options "active_slave=ens7"
Connection successfully reapplied to device 'bond0'.
or to change the primary
option:
~]$ nmcli dev mod bond0 +bond.options "primary=ens3"
Connection successfully reapplied to device 'bond0'.
Note
active_slave
option sets the currently active port whereas the primary
option of the bond specifies the active port to be automatically selected by kernel when a new port is added or a failure of the active port occurs.
7.4. Using the Command Line Interface (CLI)
bonding
kernel module and a special network interface called a channel bonding interface.
7.4.1. Check if Bonding Kernel Module is Installed
root
:
~]# modprobe --first-time bonding
This activation will not persist across system restarts. See the Red Hat Enterprise Linux Kernel Administration Guide for an explanation of persistent module loading. Note that given a correct configuration file using the BONDING_OPTS
directive, the bonding module will be loaded as required and therefore does not need to be loaded separately.
~]$ modinfo bonding
See the modprobe(8)
man page for more command options.
7.4.2. Create a Channel Bonding Interface
/etc/sysconfig/network-scripts/
directory called ifcfg-bondN
, replacing N with the number for the interface, such as 0
.
DEVICE
directive is bondN
, replacing N with the number for the interface, and TYPE=Bond
. In addition, set BONDING_MASTER=yes
.
Example 7.1. Example ifcfg-bond0 Interface Configuration File
DEVICE=bond0 NAME=bond0 TYPE=Bond BONDING_MASTER=yes IPADDR=192.168.1.1 PREFIX=24 ONBOOT=yes BOOTPROTO=none BONDING_OPTS="bonding parameters separated by spaces" NM_CONTROLLED="no"
Important
BONDING_OPTS="bonding parameters"
directive in the ifcfg-bondN
interface file. Do not specify options for the bonding device in /etc/modprobe.d/bonding.conf
, or in the deprecated /etc/modprobe.conf
file.
max_bonds
parameter is not interface specific and should not be set when using ifcfg-bondN
files with the BONDING_OPTS
directive as this directive will cause the network scripts to create the bond interfaces as required.
NM_CONTROLLED="no"
setting is not present, NetworkManager might override settings in this configuration file.
7.4.3. Creating Port Interfaces
master
) and the interfaces to be bonded are referred to as the ports (slaves
). After the channel bonding interface is created, the network interfaces to be bound together must be configured by adding the MASTER
and SLAVE
directives to the configuration files of the ports. The configuration files for each of the port interfaces can be nearly identical.
Example 7.2. Example Port Interface Configuration File
enp1s0
and enp2s0
, they can both look like the following example:
DEVICE=device_name NAME=bond0-slave TYPE=Ethernet BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes NM_CONTROLLED="no"
ONBOOT=yes
for an interface, they may race with each other and a plain TYPE=Ethernet
profile may be activated instead of a bond port.
Note
NM_CONTROLLED="no"
setting is not present, NetworkManager might override settings in this configuration file.
7.4.4. Activating a Channel Bond
root
, issue the following commands:
~]# ifup ifcfg-enp1s0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)
~]# ifup ifcfg-enp2s0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/8)
ifdown device_nameThen when complete, open all the ports, which will open the bond (provided it was not set “down”).
root
:
~]# nmcli con load /etc/sysconfig/network-scripts/ifcfg-device
Alternatively, to reload all interfaces:
~]# nmcli con reload
The default behavior is for NetworkManager not to be aware of the changes and to continue using the old configuration data. This is set by the monitor-connection-files
option in the NetworkManager.conf
file. See the NetworkManager.conf(5)
manual page for more information.
~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp1s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP mode DEFAULT qlen 1000
link/ether 52:54:00:e9:ce:d2 brd ff:ff:ff:ff:ff:ff
3: enp2s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP mode DEFAULT qlen 1000
link/ether 52:54:00:38:a6:4c brd ff:ff:ff:ff:ff:ff
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 52:54:00:38:a6:4c brd ff:ff:ff:ff:ff:ff
7.4.5. Creating Multiple Bonds
BONDING_OPTS
directive. This configuration method is used so that multiple bonding devices can have different configurations. To create multiple channel bonding interfaces, proceed as follows:
- Create multiple
ifcfg-bondN
files with theBONDING_OPTS
directive; this directive will cause the network scripts to create the bond interfaces as required. - Create, or edit existing, interface configuration files to be bonded and include the
SLAVE
directive. - Assign the interfaces to be bonded, the port interfaces, to the channel bonding interfaces by means of the
MASTER
directive.
Example 7.3. Example multiple ifcfg-bondN interface configuration files
DEVICE=bondN NAME=bondN TYPE=Bond BONDING_MASTER=yes IPADDR=192.168.1.1 PREFIX=24 ONBOOT=yes BOOTPROTO=none BONDING_OPTS="bonding parameters separated by spaces"
ifcfg-bond0
and ifcfg-bond1
, with appropriate IP
addresses.
MASTER=bondN
directive. For example, continuing on from the example above, if two interfaces per bond are required, then for two bonds create four interface configuration files and assign the first two using MASTER=bond0
and the next two using MASTER=bond1
.
7.5. Verifying Network Configuration Bonding for Redundancy
Procedure
- Ping the destination IP from the bond interface. For example:
~]#
ping -I bond0 DSTADDR
- View which interface is in
active
mode:~]#
enp1s0 is thecat /sys/class/net/bond0/bonding/active_slave
enp1s0active
port interface. - Set the
active
port interface down:~]#
ip link set enp1s0 down
- Check if the
backup
interface is up:~]#
enp2s0 is now thecat /sys/class/net/bond0/bonding/active_slave
enp2s0active
port interface. - Check if you can still ping the destination IP from the bond interface:
~]#
ping -I bond0 DSTADDR
7.6. Overview of Bonding Modes and the Required Settings on the Switch
Bonding Mode | Configuration on the Switch |
---|---|
0 - balance-rr | Requires static Etherchannel enabled (not LACP-negotiated) |
1 - active-backup | Requires autonomous ports |
2 - balance-xor | Requires static Etherchannel enabled (not LACP-negotiated) |
3 - broadcast | Requires static Etherchannel enabled (not LACP-negotiated) |
4 - 802.3ad | Requires LACP-negotiated Etherchannel enabled |
5 - balance-tlb | Requires autonomous ports |
6 - balance-alb | Requires autonomous ports |
7.7. Using Channel Bonding
miimon
or arp_interval
and the arp_ip_target
parameters. See Section 7.7.1, “Bonding Module Directives” for a list of available options and how to quickly determine the best ones for your bonded interface.
7.7.1. Bonding Module Directives
BONDING_OPTS="bonding parameters"
directive in your bonding interface configuration file (ifcfg-bond0
for example). Parameters to bonded interfaces can be configured without unloading (and reloading) the bonding module by manipulating files in the sysfs
file system.
sysfs
is a virtual file system that represents kernel objects as directories, files and symbolic links. sysfs
can be used to query for information about kernel objects, and can also manipulate those objects through the use of normal file system commands. The sysfs
virtual file system is mounted under the /sys/
directory. All bonding interfaces can be configured dynamically by interacting with and manipulating files under the /sys/class/net/
directory.
ifcfg-bond0
by following the instructions in Section 7.4.2, “Create a Channel Bonding Interface”. Insert the SLAVE=yes
and MASTER=bond0
directives in the configuration files for each interface bonded to bond0
. Once this is completed, you can proceed to testing the parameters.
ifup bondN
as root
:
~]# ifup bond0
ifcfg-bond0
bonding interface file, you will be able to see bond0
listed in the output of running ip link show
as root
:
~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp1s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP mode DEFAULT qlen 1000
link/ether 52:54:00:e9:ce:d2 brd ff:ff:ff:ff:ff:ff
3: enp2s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP mode DEFAULT qlen 1000
link/ether 52:54:00:38:a6:4c brd ff:ff:ff:ff:ff:ff
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 52:54:00:38:a6:4c brd ff:ff:ff:ff:ff:ff
~]$ cat /sys/class/net/bonding_masters
bond0
/sys/class/net/bondN/bonding/
directory. First, the bond you are configuring must be taken down:
~]# ifdown bond0
root
:
~]# echo 1000 > /sys/class/net/bond0/bonding/miimon
balance-alb
mode, run either:
~]# echo 6 > /sys/class/net/bond0/bonding/mode
~]# echo balance-alb > /sys/class/net/bond0/bonding/mode
ifup bondN
. If you decide to change the options, take the interface down, modify its parameters using sysfs
, bring it back up, and re-test.
BONDING_OPTS=
directive of the /etc/sysconfig/network-scripts/ifcfg-bondN
file for the bonding interface you are configuring. Whenever that bond is brought up (for example, by the system during the boot sequence if the ONBOOT=yes
directive is set), the bonding options specified in the BONDING_OPTS
will take effect for that bond.
parm
in modinfo bonding
output, or for more detailed information, see https://www.kernel.org/doc/Documentation/networking/bonding.txt.
Bonding Interface Parameters
-
ad_select=value
- Specifies the 802.3ad aggregation selection logic to use. Possible values are:The
stable
or0
— Default setting. The active aggregator is chosen by largest aggregate bandwidth. Reselection of the active aggregator occurs only when all ports of the active aggregator are down or if the active aggregator has no ports.bandwidth
or1
— The active aggregator is chosen by largest aggregate bandwidth. Reselection occurs if:- A port is added to or removed from the bond;
- Any port's link state changes;
- Any port's 802.3ad association state changes;
- The bond's administrative state changes to up.
count
or2
— The active aggregator is chosen by the largest number of ports. Reselection occurs as described for thebandwidth
setting above.
bandwidth
andcount
selection policies permit failover of 802.3ad aggregations when partial failure of the active aggregator occurs. This keeps the aggregator with the highest availability, either in bandwidth or in number of ports, active at all times. -
arp_interval=time_in_milliseconds
- Specifies, in milliseconds, how often
ARP
monitoring occurs.Important
It is essential that botharp_interval
andarp_ip_target
parameters are specified, or, alternatively, themiimon
parameter is specified. Failure to do so can cause degradation of network performance in the event that a link fails.If using this setting while inmode=0
ormode=2
(the two load-balancing modes), the network switch must be configured to distribute packets evenly across the NICs. For more information on how to accomplish this, see https://www.kernel.org/doc/Documentation/networking/bonding.txt.The value is set to0
by default, which disables it. -
arp_ip_target=ip_address[,ip_address_2,…ip_address_16]
- Specifies the target
IP
address ofARP
requests when thearp_interval
parameter is enabled. Up to 16IP
addresses can be specified in a comma separated list. -
arp_validate=value
- Validate source/distribution of
ARP
probes; default isnone
. Other valid values areactive
,backup
, andall
. -
downdelay=time_in_milliseconds
- Specifies (in milliseconds) how long to wait after link failure before disabling the link. The value must be a multiple of the value specified in the
miimon
parameter. The value is set to0
by default, which disables it. -
fail_over_mac=value
- Specifies whether active-backup mode should set all ports to the same MAC address at the point of assignment (the traditional behavior), or, when enabled, perform special handling of the bond's MAC address in accordance with the selected policy. Possible values are:
none
or0
— Default setting. This setting disablesfail_over_mac
, and causes bonding to set all ports of an active-backup bond to the same MAC address at the point of assignment.active
or1
— The “active”fail_over_mac
policy indicates that the MAC address of the bond should always be the MAC address of the currently active port. The MAC address of the ports is not changed; instead, the MAC address of the bond changes during a failover.This policy is useful for devices that cannot ever alter their MAC address, or for devices that refuse incoming broadcasts with their own source MAC (which interferes with the ARP monitor). The disadvantage of this policy is that every device on the network must be updated by gratuitous ARP, as opposed to the normal method of switches snooping incoming traffic to update their ARP tables. If the gratuitous ARP is lost, communication may be disrupted.When this policy is used in conjunction with the MII monitor, devices which assert link up prior to being able to actually transmit and receive are particularly susceptible to loss of the gratuitous ARP, and an appropriate updelay setting may be required.follow
or2
— The “follow”fail_over_mac
policy causes the MAC address of the bond to be selected normally (normally the MAC address of the first port added to the bond). However, the second and subsequent ports are not set to this MAC address while they are in a backup role; a port is programmed with the bond's MAC address at failover time (and the formerly active port receives the newly active port's MAC address).This policy is useful for multiport devices that either become confused or incur a performance penalty when multiple ports are programmed with the same MAC address.
- lacp_rate=value
- Specifies the rate at which link partners should transmit LACPDU packets in 802.3ad mode. Possible values are:
slow
or0
— Default setting. This specifies that partners should transmit LACPDUs every 30 seconds.fast
or1
— Specifies that partners should transmit LACPDUs every 1 second.
-
miimon=time_in_milliseconds
- Specifies (in milliseconds) how often MII link monitoring occurs. This is useful if high availability is required because MII is used to verify that the NIC is active. To verify that the driver for a particular NIC supports the MII tool, type the following command as root:
~]#
ethtool interface_name | grep "Link detected:"
In this command, replace interface_name with the name of the device interface, such asenp1s0
, not the bond interface. If MII is supported, the command returns:Link detected: yes
If using a bonded interface for high availability, the module for each NIC must support MII. Setting the value to0
(the default), turns this feature off. When configuring this setting, a good starting point for this parameter is100
.Important
It is essential that botharp_interval
andarp_ip_target
parameters are specified, or, alternatively, themiimon
parameter is specified. Failure to do so can cause degradation of network performance in the event that a link fails. -
mode=value
- Allows you to specify the bonding policy. The value can be one of:
balance-rr
or0
— Sets a round-robin policy for fault tolerance and load balancing. Transmissions are received and sent out sequentially on each bonded port interface beginning with the first one available.active-backup
or1
— Sets an active-backup policy for fault tolerance. Transmissions are received and sent out through the first available bonded port interface. Another bonded port interface is only used if the active bonded port interface fails.balance-xor
or2
— Transmissions are based on the selected hash policy. The default is to derive a hash by XOR of the source and destination MAC addresses multiplied by the modulo of the number of port interfaces. In this mode traffic destined for specific peers will always be sent over the same interface. As the destination is determined by the MAC addresses this method works best for traffic to peers on the same link or local network. If traffic has to pass through a single router then this mode of traffic balancing will be suboptimal.broadcast
or3
— Sets a broadcast policy for fault tolerance. All transmissions are sent on all port interfaces.802.3ad
or4
— Sets an IEEE 802.3ad dynamic link aggregation policy. Creates aggregation groups that share the same speed and duplex settings. Transmits and receives on all ports in the active aggregator. Requires a switch that is 802.3ad compliant.balance-tlb
or5
— Sets a Transmit Load Balancing (TLB) policy for fault tolerance and load balancing. The outgoing traffic is distributed according to the current load on each port interface. Incoming traffic is received by the current port. If the receiving port fails, another port takes over the MAC address of the failed port. This mode is only suitable for local addresses known to the kernel bonding module and therefore cannot be used behind a bridge with virtual machines.balance-alb
or6
— Sets an Adaptive Load Balancing (ALB) policy for fault tolerance and load balancing. Includes transmit and receive load balancing forIPv4
traffic. Receive load balancing is achieved throughARP
negotiation. This mode is only suitable for local addresses known to the kernel bonding module and therefore cannot be used behind a bridge with virtual machines.
For details about required settings on the upstream switch, see Section 7.6, “Overview of Bonding Modes and the Required Settings on the Switch”. -
primary=interface_name
- Specifies the interface name, such as
enp1s0
, of the primary device. Theprimary
device is the first of the bonding interfaces to be used and is not abandoned unless it fails. This setting is particularly useful when one NIC in the bonding interface is faster and, therefore, able to handle a bigger load.This setting is only valid when the bonding interface is inactive-backup
mode. See https://www.kernel.org/doc/Documentation/networking/bonding.txt for more information. -
primary_reselect=value
- Specifies the reselection policy for the primary port. This affects how the primary port is chosen to become the active port when failure of the active port or recovery of the primary port occurs. This parameter is designed to prevent flip-flopping between the primary port and other ports. Possible values are:
always
or0
(default) — The primary port becomes the active port whenever it comes back up.better
or1
— The primary port becomes the active port when it comes back up, if the speed and duplex of the primary port is better than the speed and duplex of the current active port.failure
or2
— The primary port becomes the active port only if the current active port fails and the primary port is up.
Theprimary_reselect
setting is ignored in two cases:- If no ports are active, the first port to recover is made the active port.
- When initially assigned to a bond, the primary port is always made the active port.
Changing theprimary_reselect
policy throughsysfs
will cause an immediate selection of the best active port according to the new policy. This may or may not result in a change of the active port, depending upon the circumstances -
resend_igmp=range
- Specifies the number of IGMP membership reports to be issued after a failover event. One membership report is issued immediately after the failover, subsequent packets are sent in each 200ms interval.The valid range is
0
to255
; the default value is1
. A value of0
prevents the IGMP membership report from being issued in response to the failover event.This option is useful for bonding modesbalance-rr
(mode 0),active-backup
(mode 1),balance-tlb
(mode 5) andbalance-alb
(mode 6), in which a failover can switch the IGMP traffic from one port to another. Therefore a fresh IGMP report must be issued to cause the switch to forward the incoming IGMP traffic over the newly selected port. -
updelay=time_in_milliseconds
- Specifies (in milliseconds) how long to wait before enabling a link. The value must be a multiple of the value specified in the
miimon
parameter. The value is set to0
by default, which disables it. -
use_carrier=number
- Specifies whether or not
miimon
should use MII/ETHTOOL ioctls ornetif_carrier_ok()
to determine the link state. Thenetif_carrier_ok()
function relies on the device driver to maintains its state withnetif_carrier_on/off
; most device drivers support this function.The MII/ETHTOOL ioctls tools utilize a deprecated calling sequence within the kernel. However, this is still configurable in case your device driver does not supportnetif_carrier_on/off
.Valid values are:1
— Default setting. Enables the use ofnetif_carrier_ok()
.0
— Enables the use of MII/ETHTOOL ioctls.
Note
If the bonding interface insists that the link is up when it should not be, it is possible that your network device driver does not supportnetif_carrier_on/off
. -
xmit_hash_policy=value
- Selects the transmit hash policy used for port selection in
balance-xor
and802.3ad
modes. Possible values are:0
orlayer2
— Default setting. This parameter uses the XOR of hardware MAC addresses to generate the hash. The formula used is:(source_MAC_address XOR destination_MAC) MODULO slave_count
This algorithm will place all traffic to a particular network peer on the same port, and is 802.3ad compliant.1
orlayer3+4
— Uses upper layer protocol information (when available) to generate the hash. This allows for traffic to a particular network peer to span multiple ports, although a single connection will not span multiple ports.The formula for unfragmented TCP and UDP packets used is:((source_port XOR dest_port) XOR ((source_IP XOR dest_IP) AND
0xffff
) MODULO slave_countFor fragmented TCP or UDP packets and all otherIP
protocol traffic, the source and destination port information is omitted. For non-IP
traffic, the formula is the same as thelayer2
transmit hash policy.This policy intends to mimic the behavior of certain switches; particularly, Cisco switches with PFC2 as well as some Foundry and IBM products.The algorithm used by this policy is not 802.3ad compliant.2
orlayer2+3
— Uses a combination of layer2 and layer3 protocol information to generate the hash.Uses XOR of hardware MAC addresses andIP
addresses to generate the hash. The formula is:(((source_IP XOR dest_IP) AND
0xffff
) XOR ( source_MAC XOR destination_MAC )) MODULO slave_countThis algorithm will place all traffic to a particular network peer on the same port. For non-IP
traffic, the formula is the same as for the layer2 transmit hash policy.This policy is intended to provide a more balanced distribution of traffic than layer2 alone, especially in environments where a layer3 gateway device is required to reach most destinations.This algorithm is 802.3ad compliant.
7.8. Creating a Bond Connection Using a GUI
7.8.1. Establishing a Bond Connection
Procedure 7.1. Adding a New Bond Connection_Using nm-connection-editor
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Click the Add button. The Choose a Connection Type window appears. Select Bond and click Create. The Editing Bond connection 1 window appears.
Figure 7.6. The NetworkManager Graphical User Interface Add a Bond menu
- On the Bond tab, click and select the type of interface you want to use with the bond connection. Click the button. Note that the dialog to select the port type only comes up when you create the first port; after that, it will automatically use that same type for all further ports.
- The Editing bond0 slave 1 window appears. Use the Device MAC address drop-down menu to select the MAC address of the interface to be bonded. The first port's MAC address will be used as the MAC address for the bond interface. If required, enter a clone MAC address to be used as the bond's MAC address. Click the button.
Figure 7.7. The NetworkManager Graphical User Interface Add a Bond Connection menu
- The name of the bonded port appears in the Bonded connections window. Click the button to add further port connections.
- Review and confirm the settings and then click the Save button.
- Edit the bond-specific settings by referring to Section 7.8.1.1, “Configuring the Bond Tab” below.
Procedure 7.2. Editing an Existing Bond Connection
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Select the connection you want to edit and click the Edit button.
- Select the General tab.
- Configure the connection name, auto-connect behavior, and availability settings.Five settings in the Editing dialog are common to all connection types, see the General tab:
- Connection name — Enter a descriptive name for your network connection. This name will be used to list this connection in the menu of the Network window.
- Automatically connect to this network when it is available — Select this box if you want NetworkManager to auto-connect to this connection when it is available. See the section called “Editing an Existing Connection with control-center” for more information.
- All users may connect to this network — Select this box to create a connection available to all users on the system. Changing this setting may require root privileges. See Section 3.4.5, “Managing System-wide and Private Connection Profiles with a GUI” for details.
- Automatically connect to VPN when using this connection — Select this box if you want NetworkManager to auto-connect to a VPN connection when it is available. Select the VPN from the drop-down menu.
- Firewall Zone — Select the firewall zone from the drop-down menu. See the Red Hat Enterprise Linux 7 Security Guide for more information on firewall zones.
- Edit the bond-specific settings by referring to Section 7.8.1.1, “Configuring the Bond Tab” below.
Saving Your New (or Modified) Connection and Making Further Configurations
IPv4
settings for the connection, click the IPv4 Settings tab and proceed to Section 5.4, “Configuring IPv4 Settings”orIPv6
settings for the connection, click the IPv6 Settings tab and proceed to Section 5.5, “Configuring IPv6 Settings”.
7.8.1.1. Configuring the Bond Tab
- Mode
- The mode that is used to share traffic over the port connections which make up the bond. The default is Round-robin. Other load sharing modes, such as
802.3ad
, can be selected by means of the drop-down list. - Link Monitoring
- The method of monitoring the ports ability to carry network traffic.
- Round-robin
- Sets a round-robin policy for fault tolerance and load balancing. Transmissions are received and sent out sequentially on each bonded port interface beginning with the first one available. This mode might not work behind a bridge with virtual machines without additional switch configuration.
- Active backup
- Sets an active-backup policy for fault tolerance. Transmissions are received and sent out through the first available bonded port interface. Another bonded port interface is only used if the active bonded port interface fails. Note that this is the only mode available for bonds of InfiniBand devices.
- XOR
- Sets an XOR (exclusive-or) policy. Transmissions are based on the selected hash policy. The default is to derive a hash by XOR of the source and destination MAC addresses multiplied by the modulo of the number of port interfaces. In this mode traffic destined for specific peers will always be sent over the same interface. As the destination is determined by the MAC addresses this method works best for traffic to peers on the same link or local network. If traffic has to pass through a single router then this mode of traffic balancing will be suboptimal.
- Broadcast
- Sets a broadcast policy for fault tolerance. All transmissions are sent on all port interfaces. This mode might not work behind a bridge with virtual machines without additional switch configuration.
- 802.3ad
- Sets an IEEE
802.3ad
dynamic link aggregation policy. Creates aggregation groups that share the same speed and duplex settings. Transmits and receives on all ports in the active aggregator. Requires a network switch that is802.3ad
compliant. - Adaptive transmit load balancing
- Sets an adaptive Transmit Load Balancing (TLB) policy for fault tolerance and load balancing. The outgoing traffic is distributed according to the current load on each port interface. Incoming traffic is received by the current port. If the receiving port fails, another port takes over the MAC address of the failed port. This mode is only suitable for local addresses known to the kernel bonding module and therefore cannot be used behind a bridge with virtual machines.
- Adaptive load balancing
- Sets an Adaptive Load Balancing (ALB) policy for fault tolerance and load balancing. Includes transmit and receive load balancing for
IPv4
traffic. Receive load balancing is achieved throughARP
negotiation. This mode is only suitable for local addresses known to the kernel bonding module and therefore cannot be used behind a bridge with virtual machines.
- MII (Media Independent Interface)
- The state of the carrier wave of the interface is monitored. This can be done by querying the driver, by querying MII registers directly, or by using ethtool to query the device. Three options are available:
- Monitoring Frequency
- The time interval, in milliseconds, between querying the driver or MII registers.
- Link up delay
- The time in milliseconds to wait before attempting to use a link that has been reported as up. This delay can be used if some gratuitous
ARP
requests are lost in the period immediately following the link being reported as “up”. This can happen during switch initialization for example. - Link down delay
- The time in milliseconds to wait before changing to another link when a previously active link has been reported as “down”. This delay can be used if an attached switch takes a relatively long time to change to backup mode.
- ARP
- The address resolution protocol (
ARP
) is used to probe one or more peers to determine how well the link-layer connections are working. It is dependent on the device driver providing the transmit start time and the last receive time.Two options are available:- Monitoring Frequency
- The time interval, in milliseconds, between sending
ARP
requests. - ARP targets
- A comma separated list of
IP
addresses to sendARP
requests to.
7.9. Additional Resources
Installed Documentation
nmcli(1)
man page — Describes NetworkManager's command‐line tool.nmcli-examples(5)
man page — Gives examples of nmcli commands.nm-settings(5)
man page — Description of settings and parameters of NetworkManager connections.
Online Documentation
- Red Hat Enterprise Linux System Administrator's Guide
- Explains the use of kernel module capabilities.
- https://access.redhat.com/site/node/28421/Configuring_VLAN_devices_over_a_bonded_interface
- A Red Hat Knowledgebase article about Configuring VLAN devices over a bonded interface.
Chapter 8. Configure Network Teaming
8.1. Understanding Network Teaming
bonding
. The term Network Teaming has been chosen to refer to this new implementation of the concept. The existing bonding driver is unaffected, Network Teaming is offered as an alternative and does not replace bonding in Red Hat Enterprise Linux 7.
Note
teamd
, which uses the libteam
library is also available. One instance of teamd
can control one instance of the Team driver. The daemon implements the load-balancing and active-backup logic, such as round-robin, by using additional code referred to as “runners”. By separating the code in this way, the Network Teaming implementation presents an easily extensible and scalable solution for load-balancing and redundancy requirements. For example, custom runners can be relatively easily written to implement new logic through teamd
, and even teamd
is optional, users can write their own application to use libteam.
teamd
using D-bus. teamdctl provides a D-Bus wrapper around the teamd
D-Bus API. By default, teamd
listens and communicates using Unix Domain Sockets but still monitors D-Bus. This is to ensure that teamd
can be used in environments where D-Bus is not present or not yet loaded. For example, when booting over teamd
links, D-Bus would not yet be loaded. The teamdctl utility can be used during run time to read the configuration, the state of link-watchers, check and change the state of ports, add and remove ports, and to change ports between active and backup states.
libteam
user-space library does not directly interact with the API, but uses libnl or teamnl to interact with the driver API.
Note
port
is also known as slave
. Port
is preferred when using teamd
directly while slave
is used when using NetworkManager to refer to interfaces which create a team.
8.2. Understanding the Default Behavior of Controller and Port Interfaces
NetworkManager
daemon, and especially when fault finding, keep the following in mind:
- Starting the controller interface does not automatically start the port interfaces.
- Starting a port interface always starts the controller interface.
- Stopping the controller interface also stops the port interfaces.
- A controller without ports can start static
IP
connections. - A controller without ports waits for ports when starting
DHCP
connections. - A controller with a
DHCP
connection waiting for ports completes when a port with a carrier is added. - A controller with a
DHCP
connection waiting for ports continues waiting when a port without a carrier is added.
Warning
8.3. Comparison of Network Teaming to Bonding
Feature | Bonding | Team |
---|---|---|
broadcast Tx policy | Yes | Yes |
round-robin Tx policy | Yes | Yes |
active-backup Tx policy | Yes | Yes |
LACP (802.3ad) support | Yes (active only) | Yes |
Hash-based Tx policy | Yes | Yes |
User can set hash function | No | Yes |
Tx load-balancing support (TLB) | Yes | Yes |
LACP hash port select | Yes | Yes |
load-balancing for LACP support | No | Yes |
Ethtool link monitoring | Yes | Yes |
ARP link monitoring | Yes | Yes |
NS/NA (IPv6) link monitoring | No | Yes |
ports up/down delays | Yes | Yes |
port priorities and stickiness (“primary” option enhancement) | No | Yes |
separate per-port link monitoring setup | No | Yes |
multiple link monitoring setup | Limited | Yes |
lockless Tx/Rx path | No (rwlock) | Yes (RCU) |
VLAN support | Yes | Yes |
user-space runtime control | Limited | Full |
Logic in user-space | No | Yes |
Extensibility | Hard | Easy |
Modular design | No | Yes |
Performance overhead | Low | Very Low |
D-Bus interface | No | Yes |
multiple device stacking | Yes | Yes |
zero config using LLDP | No | (in planning) |
NetworkManager support | Yes | Yes |
8.4. Understanding the Network Teaming Daemon and the "Runners"
teamd
, uses libteam to control one instance of the team driver. This instance of the team driver adds instances of a hardware device driver to form a “team” of network links. The team driver presents a network interface, team0 for example, to the other parts of the kernel. The interfaces created by instances of the team driver are given names such as team0, team1, and so forth in the documentation. This is for ease of understanding and other names can be used. The logic common to all methods of teaming is implemented by teamd
; those functions that are unique to the different load sharing and backup methods, such as round-robin, are implemented by separate units of code referred to as “runners”. Because words such as “module” and “mode” already have specific meanings in relation to the kernel, the word “runner” was chosen to refer to these units of code. The user specifies the runner in the JSON format configuration file and the code is then compiled into an instance of teamd
when the instance is created. A runner is not a plug-in because the code for a runner is compiled into an instance of teamd
as it is being created. Code could be created as a plug-in for teamd
should the need arise.
- broadcast (data is transmitted over all ports)
- round-robin (data is transmitted over all ports in turn)
- active-backup (one port or link is used while others are kept as a backup)
- loadbalance (with active Tx load balancing and BPF-based Tx port selectors)
- lacp (implements the 802.3ad Link Aggregation Control Protocol)
- ethtool (Libteam lib uses ethtool to watch for link state changes). This is the default if no other link-watcher is specified in the configuration file.
- arp_ping (The arp_ping utility is used to monitor the presence of a far-end hardware address using ARP packets.)
- nsna_ping (Neighbor Advertisements and Neighbor Solicitation from the
IPv6
Neighbor Discovery protocol are used to monitor the presence of a neighbor's interface)
8.5. Install the Network Teaming Daemon
teamd
, is not installed by default. To install teamd
, issue the following command as root
:
~]# yum install teamd
8.6. Converting a Bond to a Team
ifcfg
format to team configuration files in either ifcfg
or JSON format. Note that firewall rules, alias interfaces, and anything that might be tied to the original interface name can break after the renaming because the tool will only change the ifcfg
file, nothing else.
~]$ bond2team --examples
New files will be created in a directory whose name starts with /tmp/bond2team.XXXXXX/
, where XXXXXX is a random string. After creating the new configuration files, move the old bonding files to a backup folder and then move the new files to the /etc/sysconfig/network-scripts/
directory.
Example 8.1. Convert a Bond to a Team
bond0
configuration to team ifcfg
, issue a command as root
:
~]# /usr/bin/bond2team --master bond0
Note that this will retain the name bond0
. To use a new name to save the configuration, use the --rename
as follows:
~]# /usr/bin/bond2team --master bond0 --rename team0
add the --json
option to output JSON format files instead of ifcfg
files. See the teamd.conf(5)
man page for examples of JSON format.
Example 8.2. Convert a Bond to a Team and Specify the File Path
bond0
configuration to team ifcfg
, and to manually specify the path to the ifcfg
file, issue a command as root
:
~]# /usr/bin/bond2team --master bond0 --configdir /path/to/ifcfg-file
add the --json
option to output JSON format files instead of ifcfg
files.
Example 8.3. Create a Team Configuration Using Bond2team
~]# /usr/bin/bond2team --bonding_opts "mode=1 miimon=500"
Ports can also be supplied on the command line as follows:
~]#/usr/bin/bond2team --bonding_opts "mode=1 miimon=500 primary=enp1s0 \
primary_reselect-0" --port enp1s0 --port enp2s0 --port enp3s0 --port enp4s0
bond2team(1)
man page for further details. For an explanation of bonding parameters, see Section 7.7, “Using Channel Bonding”
8.7. Selecting Interfaces to Use as Ports for a Network Team
~]$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP > mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP > mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 52:54:00:6a:02:8a brd ff:ff:ff:ff:ff:ff
3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP > mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 52:54:00:9b:6d:2a brd ff:ff:ff:ff:ff:ff
From the available interfaces, determine which are suitable for adding to your network team and then proceed to Section 8.8, “Selecting Network Team Configuration Methods”
8.8. Selecting Network Team Configuration Methods
teamd
, proceed to Section 8.10.2, “Creating a Network Team Using teamd”.
8.9. Configure a Network Team Using the Text User Interface, nmtui
~]$ nmtui
The text user interface appears. Any invalid command prints a usage message.
- From the starting menu, select Edit a connection. Select Add, the New Connection screen opens.
Figure 8.1. The NetworkManager Text User Interface Add a Team Connection menu
- Select Team, the Edit connection screen opens.
Figure 8.2. The NetworkManager Text User Interface Configuring a Team Connection menu
- To add port interfaces to the team select Add, the New Connection screen opens. Once the type of Connection has been chosen select the Create button to cause the team's Edit Connection display to appear.
Figure 8.3. The NetworkManager Text User Interface Configuring a new Team Port Interface Connection menu
- Enter the required port's device name or MAC address in the Device section. If required, enter a clone MAC address to be used as the team's MAC address by selecting Show to the right of the Ethernet label. Select the OK button.
Note
If the device is specified without a MAC address the Device section will be automatically populated once the Edit Connection window is reloaded, but only if it successfully finds the device.Figure 8.4. The NetworkManager Text User Interface Configuring a Team's Port Interface Connection menu
- The name of the teamed port appears in the Slaves section. Repeat the above steps to add further port connections.
- If custom port settings are to be applied select the Edit button under the JSON configuration section. This will launch a vim console where changes may be applied. Once finished write the changes from vim and then confirm that the displayed JSON string under JSON configuration matches what is intended.
- Review and confirm the settings before selecting the OK button.
Figure 8.5. The NetworkManager Text User Interface Configuring a Team Connection menu
8.10. Configure a Network Team Using the Command Line
8.10.1. Configure Network Teaming Using nmcli
~]$ nmcli connection show
NAME UUID TYPE DEVICE
enp2s0 0e8185a1-f0fd-4802-99fb-bedbb31c689b 802-3-ethernet --
enp1s0 dfe1f57b-419d-4d1c-aaf5-245deab82487 802-3-ethernet --
~]$ nmcli device status
DEVICE TYPE STATE CONNECTION
virbr0 bridge connected virbr0
ens3 ethernet connected ens3
~]$ nmcli connection add type team ifname ServerA
Connection 'team-ServerA' (b954c62f-5fdd-4339-97b0-40efac734c50) successfully added.
NetworkManager will set its internal parameter connection.autoconnect
to yes
and as no IP
address was given ipv4.method
will be set to auto
. NetworkManager will also write a configuration file to /etc/sysconfig/network-scripts/ifcfg-team-ServerA
where the corresponding ONBOOT will be set to yes
and BOOTPROTO will be set to dhcp
.
~]$ nmcli con show team-ServerA
connection.id: team-ServerA
connection.uuid: b954c62f-5fdd-4339-97b0-40efac734c50
connection.interface-name: ServerA
connection.type: team
connection.autoconnect: yes
…
ipv4.method: auto
[output truncated]
As no JSON configuration file was specified the default values apply. See the teamd.conf(5)
man page for more information on the team JSON parameters and their default values. Notice that the name was derived from the interface name by prepending the type. Alternatively, specify a name with the con-name
option as follows:
~]$ nmcli connection add type team con-name Team0 ifname ServerB
Connection 'Team0' (5f7160a1-09f6-4204-8ff0-6d96a91218a7) successfully added.
~]$ nmcli con show
NAME UUID TYPE DEVICE
team-ServerA b954c62f-5fdd-4339-97b0-40efac734c50 team ServerA
enp2s0 0e8185a1-f0fd-4802-99fb-bedbb31c689b 802-3-ethernet --
enp1s0 dfe1f57b-419d-4d1c-aaf5-245deab82487 802-3-ethernet --
Team0 5f7160a1-09f6-4204-8ff0-6d96a91218a7 team ServerB
nmcli con mod old-team-name connection.id new-team-name
nmcli connection modify team-name team.config JSON-config
You can specify the team configuration either as a JSON string or provide a file name containing the configuration. The file name can include the path. In both cases, what is stored in the team.config
property is the JSON string. In the case of a JSON string, use single quotes around the string and paste the entire string to the command line.
team.config
property: nmcli con show team-name | grep team.config
team.config
property is set, all the other team properties are updated accordingly.
team.config
property is updated to match the new values.
team.link-watchers
property which allows to specify one or multiple link-watchers
, enter a command in the following format: nmcli connection modify team-name team.link-watchers
"name=ethtool delay-up=5, name=nsna_ping target-host=target.host"
The required link-watchers
are separated by comma and the attributes which belong to the same link-watcher
are separated by space.
team.runner
and the team.link-watchers
properties, enter a command in the following format: nmcli connection modify team-name team.runner
activebackup team.link-watchers
"name=ethtool delay-up=5, name=nsna_ping target-host=target.host"
team.config
property to the corresponding JSON string: nmcli connection modify team-name team.config
'{"runner": {"name": "activebackup"}, "link_watch": [{"name": "ethtool", "delay_up": 5},{"name": "nsna_ping", "target_host ": "target.host"}]'
Team0
, with the name Team0-port1, issue a command as follows:
~]$ nmcli con add type ethernet con-name Team0-port1 ifname enp1s0 slave-type team master Team0
Connection 'Team0-port1' (ccd87704-c866-459e-8fe7-01b06cf1cffc) successfully added.
~]$ nmcli con add type ethernet con-name Team0-port2 ifname enp2s0 slave-type team master Team0
Connection 'Team0-port2' (a89ccff8-8202-411e-8ca6-2953b7db52dd) successfully added.
nmcli only supports Ethernet ports.
~]$ nmcli connection up Team0-port1
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/2)
~]$ nmcli connection up Team0-port2
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)
~]$ ip link
3: Team0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 52:54:00:76:6f:f0 brd ff:ff:ff:ff:ff:f
Alternatively, issue a command to open the team as follows:
~]$ nmcli connection up Team0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
8.10.2. Creating a Network Team Using teamd
Note
root
privileges. To list the available example configurations, enter the following command:
~]$ ls /usr/share/doc/teamd-*/example_configs/
activebackup_arp_ping_1.conf activebackup_multi_lw_1.conf loadbalance_2.conf
activebackup_arp_ping_2.conf activebackup_nsna_ping_1.conf loadbalance_3.conf
activebackup_ethtool_1.conf broadcast.conf random.conf
activebackup_ethtool_2.conf lacp_1.conf roundrobin_2.conf
activebackup_ethtool_3.conf loadbalance_1.conf roundrobin.conf
To view one of the included files, such as activebackup_ethtool_1.conf
, enter the following command:
~]$ cat /usr/share/doc/teamd-*/example_configs/activebackup_ethtool_1.conf
{
"device": "team0",
"runner": {"name": "activebackup"},
"link_watch": {"name": "ethtool"},
"ports": {
"enp1s0": {
"prio": -10,
"sticky": true
},
"enp2s0": {
"prio": 100
}
}
}
Create a working configurations directory to store teamd
configuration files. For example, as normal user, enter a command with the following format:
~]$ mkdir ~/teamd_working_configs
Copy the file you have chosen to your working directory and edit it as necessary. As an example, you could use a command with the following format:
~]$ cp /usr/share/doc/teamd-*/example_configs/activebackup_ethtool_1.conf \ ~/teamd_working_configs/activebackup_ethtool_1.conf
To edit the file to suit your environment, for example to change the interfaces to be used as ports for the network team, open the file for editing as follows:
~]$ vi ~/teamd_working_configs/activebackup_ethtool_1.conf
Make any necessary changes and save the file. See the vi(1)
man page for help on using the vi editor or use your preferred editor.
~]$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 52:54:00:d5:f7:d4 brd ff:ff:ff:ff:ff:ff
3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 52:54:00:d8:04:70 brd ff:ff:ff:ff:ff:ff
In this example we see that both the interfaces we plan to use are “UP”.
root
in the following format:
~]# ip link set down em1
Repeat for each interface as necessary.
root
user, change to the working configurations directory (teamd_working_configs in this example):
~]# cd /home/userteamd_working_configs
Then issue a command in the following format:
~]# teamd -g -f activebackup_ethtool_1.conf -d
Using team device "team0".
Using PID file "/var/run/teamd/team0.pid"
Using config file "/home/user/teamd_working_configs/activebackup_ethtool_1.conf"
The -g
option is for debug messages, -f
option is to specify the configuration file to load, and the -d
option is to make the process run as a daemon after startup. See the teamd(8)
man page for other options.
root
:
~]# teamdctl team0 state
setup:
runner: activebackup
ports:
em1
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
em2
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
runner:
active port: em1
root
in the following format:
~]# ip addr add 192.168.23.2/24 dev team0
~]$ ip addr show team0
4: team0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 16:38:57:60:20:6f brd ff:ff:ff:ff:ff:ff
inet 192.168.23.2/24 scope global team0
valid_lft forever preferred_lft forever
inet6 2620:52:0:221d:1438:57ff:fe60:206f/64 scope global dynamic
valid_lft 2591880sec preferred_lft 604680sec
inet6 fe80::1438:57ff:fe60:206f/64 scope link
valid_lft forever preferred_lft forever
root
in the following format:
~]# ip link set dev team0 up
root
in the following format:
~]# ip link set dev team0 down
root
user, issue a command in the following format:
~]# teamd -t team0 -k
The -k
option is to specify that the instance of the daemon associated with the device team0 is to be killed. See the teamd(8)
man page for other options.
teamd
, issue the following command:
~]$ teamd -h
In addition, see the teamd(8)
man page.
8.10.3. Creating a Network Team Using ifcfg Files
ifcfg
files, create a file in the /etc/sysconfig/network-scripts/
directory as follows:
DEVICE=team0 DEVICETYPE=Team ONBOOT=yes BOOTPROTO=none IPADDR=192.168.11.1 PREFIX=24 TEAM_CONFIG='{"runner": {"name": "activebackup"}, "link_watch": {"name": "ethtool"}}'This creates the interface to the team, in other words, this is the
master
.
/etc/sysconfig/network-scripts/
directory as follows:
DEVICE=enp1s0 HWADDR=D4:85:64:01:46:9E DEVICETYPE=TeamPort ONBOOT=yes TEAM_MASTER=team0 TEAM_PORT_CONFIG='{"prio": 100}'Add additional port interfaces similar to the above as required, changing the DEVICE and HWADDR field to match the ports (the network devices) being added. If port priority is not specified by
prio
it defaults to 0
; it accepts negative and positive values in the range -32,767
to +32,767
.
HWADDR
directive will influence the device naming procedure. This is explained in Chapter 11, Consistent Network Device Naming.
root
:
~]# ifup team0
To view the network team, issue the following command:
~]$ ip link show
8.10.4. Add a Port to a Network Team Using iputils
root
:
~]#Add additional ports as required. Team driver will bring ports up automatically.ip link set dev em1 down
~]#ip link set dev em1 master team0
8.10.5. Listing the ports of a Team Using teamnl
root
:
~]# teamnl team0 ports
em2: up 100 fullduplex
em1: up 100 fullduplex
8.10.6. Configuring Options of a Team Using teamnl
root
:
~]# teamnl team0 options
To configure a team to use active backup mode, issue the following command as root
:
~]# teamnl team0 setoption mode activebackup
8.10.7. Add an Address to a Network Team Using iputils
root
:
~]# ip addr add 192.168.252.2/24 dev team0
8.10.8. open an Interface to a Network Team Using iputils
root
:
~]# ip link set team0 up
8.10.9. Viewing the Active Port Options of a Team Using teamnl
activeport
option in a network team, using the teamnl utility, issue the following command as root
:
~]# teamnl team0 getoption activeport
0
8.10.10. Setting the Active Port Options of a Team Using teamnl
activeport
option in a network team, using the teamnl utility, issue the following command as root
:
~]# teamnl team0 setoption activeport 5
To check the change in team port options, issue the following command as root
:
~]# teamnl team0 getoption activeport
5
8.11. Controlling teamd with teamdctl
teamd
for statistics or configuration information, or to make changes, the control tool teamdctl is used.
root
:
~]# teamdctl team0 state view
For a more verbose output:
~]# teamdctl team0 state view -v
~]# teamdctl team0 state dump
~]# teamdctl team0 config dump
~]# teamdctl team0 port config dump em1
8.11.1. Add a Port to a Network Team
root
:
~]# teamdctl team0 port add em1
Important
teamdctl
directly to add a port, the port must be set to down. Otherwise the teamdctl team0 port add em1
command will fail.
8.11.2. Remove a Port From a Network Team
root
:
~]# teamdctl team0 port remove em1
8.11.3. Applying a Sticky Setting to a Port in a Network Team
teamdctl
command to apply a sticky setting to ensure that a specific port is used as an active link when it is available.
Prerequisites
- You already created a team of network interfaces. As a result, you have a port (em1) that you want to update the configuration of.
Procedure
- Update the configuration of the sticky setting for em1:
~]#
teamdctl team0 port config update em1 '{ "prio": 100, "sticky": true }'
- Remove em1:
~]#
teamdctl team0 port remove em1
- Add em1 again so that the sticky setting takes effect:
~]#
teamdctl team0 port add em1
Note that the old configuration will be overwritten and that any options omitted will be reset to the default values. See theteamdctl(8)
man page for more team daemon control tool command examples.
8.11.4. View the Configuration of a Port in a Network Team
root
:
~]# teamdctl team0 port config dump em1
This will dump the JSON format configuration of the port to standard output.
8.12. Verifying Network Configuration Teaming for Redundancy
Procedure
- Ping the destination IP from the team interface. For example:
~]#
ping -I team0 DSTADDR
- View which interface is in
active
mode:~]#
enp1s0 is theteamdctl team0 state
setup: runner: activebackup ports: enp1s0 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 0 enp2s0 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 0 runner: active port: enp1s0active
interface. - Temporarily remove the network cable from the host.
Note
There is no method to properly test link failure events using software utilities. Tools that deactivate connections, such asip
ornmcli
, show only the driver’s ability to handle port configuration changes and not actual link failure events. - Check if the
backup
interface is up:~]#
enp2s0 is now theteamdctl team0 state
setup: runner: activebackup ports: enp1s0 link watches: link summary: down instance[link_watch_0]: name: ethtool link: down down count: 1 enp2s0 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 0 runner: active port: enp2s0active
interface. - Check if you can still ping the destination IP from the team interface:
~]#
ping -I team0 DSTADDR
8.13. Configure teamd Runners
teamd
runners, see Section 8.4, “Understanding the Network Teaming Daemon and the "Runners"”.
8.13.1. Configure the broadcast Runner
root
, add the following to the team JSON format configuration file:
{ "device": "team0", "runner": {"name": "broadcast"}, "ports": {"em1": {}, "em2": {}} }
teamd.conf(5)
man page for more information.
8.13.2. Configure the random Runner
root
, add the following to the team JSON format configuration file:
{ "device": "team0", "runner": {"name": "random"}, "ports": {"em1": {}, "em2": {}} }
teamd.conf(5)
man page for more information.
8.13.3. Configure the Round-robin Runner
root
, add the following to the team JSON format configuration file:
{ "device": "team0", "runner": {"name": "roundrobin"}, "ports": {"em1": {}, "em2": {}} }A very basic configuration for round-robin.
teamd.conf(5)
man page for more information.
8.13.4. Configure the activebackup Runner
{ "device": "team0", "runner": { "name": "activebackup" }, "link_watch": { "name": "ethtool" }, "ports": { "em1": { "prio": -10, "sticky": true }, "em2": { "prio": 100 } } }This example configuration uses the active-backup runner with ethtool as the link watcher. Port em2 has higher priority. The sticky flag ensures that if em1 becomes active, it stays active as long as the link remains up.
{ "device": "team0", "runner": { "name": "activebackup" }, "link_watch": { "name": "ethtool" }, "ports": { "em1": { "prio": -10, "sticky": true, "queue_id": 4 }, "em2": { "prio": 100 } } }This example configuration adds a queue ID of
4
. It uses active-backup runner with ethtool as the link watcher. Port em2 has higher priority. But the sticky flag ensures that if em1 becomes active, it will stay active as long as the link remains up.
root
, add the following to the team JSON format configuration file:
{ "device": "team0", "runner": { "name": "activebackup" }, "link_watch": { "name": "ethtool", "delay_up": 2500, "delay_down": 1000 }, "ports": { "em1": { "prio": -10, "sticky": true }, "em2": { "prio": 100 } } }This example configuration uses the active-backup runner with ethtool as the link watcher. Port em2 has higher priority. But the sticky flag ensures that if em1 becomes active, it stays active while the link remains up. Link changes are not propagated to the runner immediately, but delays are applied.
teamd.conf(5)
man page for more information.
8.13.5. Configure the loadbalance Runner
teamd
.
root
, add the following to the team JSON format configuration file:
{ "device": "team0", "runner": { "name": "loadbalance", "tx_hash": ["eth", "ipv4", "ipv6"] }, "ports": {"em1": {}, "em2": {}} }Configuration for hash-based passive transmit (Tx) load balancing.
root
, add the following to the team JSON format configuration file:
{ "device": "team0", "runner": { "name": "loadbalance", "tx_hash": ["eth", "ipv4", "ipv6"], "tx_balancer": { "name": "basic" } }, "ports": {"em1": {}, "em2": {}} }Configuration for active transmit (Tx) load balancing using basic load balancer.
teamd.conf(5)
man page for more information.
8.13.6. Configure the LACP (802.3ad) Runner
root
, add the following to the team JSON format configuration file:
{ "device": "team0", "runner": { "name": "lacp", "active": true, "fast_rate": true, "tx_hash": ["eth", "ipv4", "ipv6"] }, "link_watch": {"name": "ethtool"}, "ports": {"em1": {}, "em2": {}} }Configuration for connection to a link aggregation control protocol (LACP) capable counterpart. The LACP runner should use ethtool to monitor the status of a link. Note that only ethtool can be used for link monitoring because, for example in the case of arp_ping, the link would never come up. The reason is that the link has to be established first and only after that can packets, ARP included, go through. Using ethtool prevents this because it monitors each link layer individually.
"tx_balancer": { "name": "basic" }
teamd.conf(5)
man page for more information.
8.13.7. Configure Monitoring of the Link State
root
privileges.
8.13.7.1. Configure Ethtool for link-state Monitoring
"link_watch": { "name": "ethtool", "delay_up": 2500 }
"link_watch": { "name": "ethtool", "delay_down": 1000 }
8.13.7.2. Configure ARP Ping for Link-state Monitoring
teamd
sends an ARP REQUEST to an address at the remote end of the link in order to determine if the link is up. The method used is the same as the arping utility but it does not use that utility.
{ "device": "team0", "runner": {"name": "activebackup"}, "link_watch":{ "name": "arp_ping", "interval": 100, "missed_max": 30, "source_host": "192.168.23.2", "target_host": "192.168.23.1" }, "ports": { "em1": { "prio": -10, "sticky": true }, "em2": { "prio": 100 } } }This configuration uses arp_ping as the link watcher. The
missed_max
option is a limit value of the maximum allowed number of missed replies (ARP replies for example). It should be chosen in conjunction with the interval
option in order to determine the total time before a link is reported as down.
root
:
~]# teamdctl port config update em2 JSON-config-fileNote that the old configuration will be overwritten and that any options omitted will be reset to the default values. See the
teamdctl(8)
man page for more team daemon control tool command examples.
8.13.7.3. Configure IPv6 NA/NS for Link-state Monitoring
{ "device": "team0", "runner": {"name": "activebackup"}, "link_watch": { "name": "nsna_ping", "interval": 200, "missed_max": 15, "target_host": "fe80::210:18ff:feaa:bbcc" }, "ports": { "em1": { "prio": -10, "sticky": true }, "em2": { "prio": 100 } } }
"link_watch": { "name": "nsna_ping", "interval": 200 }Value is positive number in milliseconds. It should be chosen in conjunction with the
missed_max
option in order to determine the total time before a link is reported as down.
"link_watch": { "name": "nsna_ping", "missed_max": 15 }Maximum number of missed NS/NA reply packets. If this number is exceeded, the link is reported as down. The
missed_max
option is a limit value of the maximum allowed number of missed replies (ARP replies for example). It should be chosen in conjunction with the interval
option in order to determine the total time before a link is reported as down.
IPv6
address target address for the NS/NA packets, add or edit a section as follows:
"link_watch": { "name": "nsna_ping", "target_host": "MyStorage" }The “target_host” option contains the host name to be converted to an
IPv6
address which will be used as the target address for the NS/NA packets. An IPv6
address can be used in place of a host name.
teamd.conf(5)
man page for more information.
8.13.8. Configure Port Selection Override
teamd
runner). On occasion however, it is helpful to direct certain classes of outgoing traffic to certain physical interfaces to implement slightly more complex policies. By default the team driver is multiqueue aware and 16 queues are created when the driver initializes. If more or less queues are required, the Netlink attribute tx_queues
can be used to change this value during the team driver instance creation.
queue_id
as follows:
{ "queue_id": 3 }These queue ID's can be used in conjunction with the tc utility to configure a multiqueue queue discipline and filters to bias certain traffic to be transmitted on certain port devices. For example, if using the above configuration and wanting to force all traffic bound to
192.168.1.100
to use enp1s0 in the team as its output device, issue commands as root
in the following format:
~]#This mechanism of overriding runner selection logic in order to bind traffic to a specific port can be used with all runners.tc qdisc add dev team0 handle 1 root multiq
~]#tc filter add dev team0 protocol ip parent 1: prio 1 u32 match ip dst \
192.168.1.100 action skbedit queue_mapping 3
8.13.9. Configure BPF-based Tx Port Selectors
eth
— Uses source and destination MAC addresses.vlan
— Uses VLAN ID.ipv4
— Uses source and destinationIPv4
addresses.ipv6
— Uses source and destinationIPv6
addresses.ip
— Uses source and destinationIPv4
andIPv6
addresses.l3
— Uses source and destinationIPv4
andIPv6
addresses.tcp
— Uses source and destinationTCP
ports.udp
— Uses source and destinationUDP
ports.sctp
— Uses source and destinationSCTP
ports.l4
— Uses source and destinationTCP
andUDP
andSCTP
ports.
"tx_hash": ["eth", "ipv4", "ipv6"]See Section 8.13.5, “Configure the loadbalance Runner” for an example.
8.14. Creating a Network Team Using a GUI
8.14.1. Establishing a Team Connection
Procedure 8.1. Adding a New Team Connection Using nm-connection-editor
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Click the Add button. The Choose a Connection Type window appears. Select Team and click Create. The Editing Team connection 1 window appears.
Figure 8.6. The NetworkManager Graphical User Interface Add a menu
- On the Team tab, click and select the type of interface you want to use with the team connection. Click the button. Note that the dialog to select the port type only comes up when you create the first port; after that, it will automatically use that same type for all further ports.
- The Editing team0 slave 1 window appears.
Figure 8.7. The NetworkManager Graphical User Interface Add a
Slave
Connection - If custom port settings are to be applied, click on the Team Port tab and enter a JSON configuration string or import it from a file.
- Click thebutton.
- The name of the teamed port appears in the Teamed connections window. Click the button to add further port connections.
- Review and confirm the settings and then click the Save button.
- Edit the team-specific settings by referring to Section 8.14.1.1, “Configuring the Team Tab” below.
Procedure 8.2. Editing an Existing Team Connection
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Select the connection you want to edit and click the Edit button.
- Select the General tab.
- Five settings in the Editing dialog are common to the most connection types. See the General tab:
- Connection name — Enter a descriptive name for your network connection. This name is used to list this connection in the menu of the Network window.
- Connection priority for auto-activation — If the connection is set to autoconnect, the number is activated (
0
by default). The higher number means higher priority. - Automatically connect to this network when it is available — Select this box if you want NetworkManager to auto-connect to this connection when it is available. See the section called “Editing an Existing Connection with control-center” for more information.
- All users may connect to this network — Select this box to create a connection available to all users on the system. Changing this setting may require root privileges. See Section 3.4.5, “Managing System-wide and Private Connection Profiles with a GUI” for details.
- Automatically connect to VPN when using this connection — Select this box if you want NetworkManager to auto-connect to a VPN connection when it is available. Select the VPN from the drop-down menu.
- Firewall Zone — Select the firewall zone from the drop-down menu. See the Red Hat Enterprise Linux 7 Security Guide for more information on firewall zones.
- Edit the team-specific settings by referring to Section 8.14.1.1, “Configuring the Team Tab” below.
Saving Your New (or Modified) Connection and Making Further Configurations
IPv4
settings for the connection, click the IPv4 Settings tab and proceed to Section 5.4, “Configuring IPv4 Settings”orIPv6
settings for the connection, click the IPv6 Settings tab and proceed to Section 5.5, “Configuring IPv6 Settings”.
8.14.1.1. Configuring the Team Tab
8.15. Additional Resources
Installed Documentation
teamd(8)
man page — Describes theteamd
service.teamdctl(8)
man page — Describes theteamd
control tool.teamd.conf(5)
man page — Describes theteamd
configuration file.teamnl(8)
man page — Describes theteamd
Netlink library.bond2team(1)
man page — Describes a tool to convert bonding options to team.
Online Documentation
- http://www.w3schools.com/js/js_json_syntax.asp
- An explanation of JSON syntax.
Chapter 9. Configure Network Bridging
9.1. Configure Bridging Using the Text User Interface, nmtui
~]$ nmtui
The text user interface appears. Any invalid command prints a usage message.
- From the starting menu, select Edit a connection. Select Add, the New Connection screen opens.
Figure 9.1. The NetworkManager Text User Interface Add a Bridge Connection menu
- Select Bridge, the Edit connection screen opens.
- To add port interfaces to the bridge select Add, the New Connection screen opens. Once the type of Connection has been chosen select the Create button to cause the bridge's Edit Connection display to appear.
Figure 9.2. The NetworkManager Text User Interface Adding a new Bridge
Slave
Connection menu - Enter the required port's device name or MAC address in the Device section. If required, enter a clone MAC address to be used as the bridge's MAC address by selecting Show to the right of the Ethernet label. Select the OK button.
Note
If the device is specified without a MAC address the Device section will be automatically populated once the Edit Connection window is reloaded, but only if it successfully finds the device.Figure 9.3. The NetworkManager Text User Interface Configuring a Bridge
Slave
Connection menu - The name of the bridge port appears in the Slaves section. Repeat the above steps to add further port connections.
- Review and confirm the settings before selecting the OK button.
Figure 9.4. The NetworkManager Text User Interface Configuring a Bridge menu
9.2. Using the NetworkManager Command Line Tool, nmcli
root
:
~]# nmcli con add type bridge ifname br0
Connection 'bridge-br0' (6ad5bba6-98a0-4f20-839d-c997ba7668ad) successfully added.
If no interface name is specified, the name will default to bridge, bridge-1, bridge-2, and so on.
~]$ nmcli con show
NAME UUID TYPE DEVICE
bridge-br0 79cf6a3e-0310-4a78-b759-bda1cc3eef8d bridge br0
enp1s0 4d5c449a-a6c5-451c-8206-3c9a4ec88bca 802-3-ethernet enp1s0
STP
for this bridge, issue a command as follows as root
:
~]# nmcli con modify bridge-br0 bridge.stp no
To re-enable 802.1D STP
for this bridge, issue a command as follows as root
:
~]# nmcli con modify bridge-br0 bridge.stp yes
802.1D STP
is 32768
. The lower number is preferred in root bridge selection. For example, a bridge with priority of 28672
would be selected as the root bridge in preference to a bridge with priority value of 32768
(the default). To create a bridge with a non-default value, issue a command as follows:
~]$ nmcli con add type bridge ifname br5 stp yes priority 28672
Connection 'bridge-br5' (86b83ad3-b466-4795-aeb6-4a66eb1856c7) successfully added.
The allowed values are in the range 0
to 65535
.
~]$ nmcli connection modify bridge-br5 bridge.priority 36864
The allowed values are in the range 0
to 65535
.
01:80:C2:00:00:00
to 01:80:C2:00:00:0F
, change the group-forward-mask
property. This property is a mask of 16 bits. Each bit corresponds to a group address in the above-mentioned range that must be forwarded. For example:
~]$ nmcli connection modify bridge-br5 bridge.group-forward-mask 8
Important
group-forward-mask
property cannot have any of the 0
, 1
, 2
bits set to 1
because those addresses are used for Spanning tree protocol (STP), Link Aggregation Control Protocol (LACP) and Ethernet MAC pause frames.
~]$ nmcli -f bridge con show bridge-br0
802.1D STP
are listed in the bridge section of the nmcli(1)
man page.
~]$ nmcli con add type ethernet ifname enp1s0 master bridge-br0
Connection 'bridge-slave-enp1s0' (70ffae80-7428-4d9c-8cbd-2e35de72476e) successfully added.
- Change its controller and port-type properties. For example to assign an existing VLAN connection named vlan100:
~]$
nmcli connection modify vlan100 master bridge-br0 slave-type bridge
- Reactivate the connection to apply the changes:
~]$
nmcli connection up vlan100
~]$ nmcli connection edit bridge-br0
You will be placed at the nmcli prompt.
nmcli>set bridge.priority 4096
nmcli>save
Connection 'bridge-br0' (79cf6a3e-0310-4a78-b759-bda1cc3eef8d) successfully saved. nmcli>quit
9.3. Using the Command Line Interface (CLI)
9.3.1. Check if Bridging Kernel Module is Installed
root
:
~]# modprobe --first-time bridge
modprobe: ERROR: could not insert 'bridge': Module already in kernel
To display information about the module, issue the following command:
~]$ modinfo bridge
See the modprobe(8)
man page for more command options.
9.3.2. Create a Network Bridge
/etc/sysconfig/network-scripts/
directory called ifcfg-brN
, replacing N with the number for the interface, such as 0
.
- The
DEVICE
directive is given an interface name as its argument in the formatbrN
, where N is replaced with the number of the interface. - The
TYPE
directive is given an argumentBridge
. This directive determines the device type and the argument is case sensitive. - The bridge interface configuration file is given an
IP
address whereas the physical interface configuration file must only have a MAC address (see below). - An extra directive,
DELAY=0
, is added to prevent the bridge from waiting while it monitors traffic, learns where hosts are located, and builds a table of MAC addresses on which to base its filtering decisions. The default delay of 15 seconds is not needed if no routing loops are possible.
Example 9.1. Example ifcfg-br0 Interface Configuration File
IP
address:
DEVICE=br0 TYPE=Bridge IPADDR=192.168.1.1 PREFIX=24 BOOTPROTO=none ONBOOT=yes DELAY=0
Example 9.2. Example ifcfg-enp1s0 Interface Configuration File
/etc/sysconfig/network-scripts/ifcfg-device_name
, where device_name is the name of the interface
DEVICE=device_name TYPE=Ethernet HWADDR=AA:BB:CC:DD:EE:FF BOOTPROTO=none ONBOOT=yes BRIDGE=br0
ifcfg-rh
, will create a name for the connection profile in the form “Type Interface”. In this example, this means the bridge will be named Bridge br0
. Alternately, if NAME=bridge-br0
is added to the ifcfg-br0
file the connection profile will be named bridge-br0
.
Note
DEVICE
directive, almost any interface name could be used as it does not determine the device type. TYPE=Ethernet
is not strictly required. If the TYPE
directive is not set, the device is treated as an Ethernet device (unless its name explicitly matches a different interface configuration file).
HWADDR
directive will influence the device naming procedure as explained in Chapter 11, Consistent Network Device Naming.
Warning
root
in the following format: ifup device
This command will detect if NetworkManager is running and call nmcli con load UUID
and then call nmcli con up UUID
.
root
:
~]# systemctl restart network
This command will stop the network service, start the network service, and then call ifup
for all ifcfg files with ONBOOT=yes
.
Note
monitor-connection-files
option in the NetworkManager.conf
file. See the NetworkManager.conf(5)
manual page for more information.
9.3.3. Network Bridge with Bond
DEVICE=interface_name TYPE=Ethernet SLAVE=yes MASTER=bond0 BOOTPROTO=none HWADDR=AA:BB:CC:DD:EE:FF
Note
interface_name
as the interface name is common practice but almost any name could be used.
/etc/sysconfig/network-scripts/ifcfg-bond0
, as follows:
DEVICE=bond0 ONBOOT=yes BONDING_OPTS='mode=1 miimon=100' BRIDGE=brbond0For further instructions and advice on configuring the bonding module and to view the list of bonding parameters, see Section 7.7, “Using Channel Bonding”.
/etc/sysconfig/network-scripts/ifcfg-brbond0
, as follows:
DEVICE=brbond0 ONBOOT=yes TYPE=Bridge IPADDR=192.168.1.1 PREFIX=24
MASTER=bond0
directive. These point to the configuration file named /etc/sysconfig/network-scripts/ifcfg-bond0
, which contains the DEVICE=bond0
directive. This ifcfg-bond0
in turn points to the /etc/sysconfig/network-scripts/ifcfg-brbond0
configuration file, which contains the IP
address, and acts as an interface to the virtual networks inside the host.
root
in the following format: ifup device
This command will detect if NetworkManager is running and call nmcli con load UUID
and then call nmcli con up UUID
.
root
:
~]# systemctl restart network
This command will stop the network service, start the network service, and then call ifup
for all ifcfg files with ONBOOT=yes
.
Note
monitor-connection-files
option in the NetworkManager.conf
file. See the NetworkManager.conf(5)
manual page for more information.
9.4. Configure Network Bridging Using a GUI
IP
configuration such as DHCP
or IPv6
autoconfiguration. Static IP
addressing is allowed to proceed before any ports or ports are connected or begin forwarding packets.
9.4.1. Establishing a Bridge Connection with a GUI
Procedure 9.1. Adding a New Bridge Connection Using nm-connection-editor
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Click the Add button. The Choose a Connection Type window appears. Select Bridge and click Create. The Editing Bridge connection 1 window appears.
Figure 9.5. Editing Bridge Connection 1
- Add port devices by referring to Procedure 9.3, “Adding a Port Interface to a Bridge” below.
Procedure 9.2. Editing an Existing Bridge Connection
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Select the Bridge connection you want to edit.
- Click the Edit button.
Configuring the Connection Name, Auto-Connect Behavior, and Availability Settings
- Connection name — Enter a descriptive name for your network connection. This name will be used to list this connection in the menu of the Network window.
- Automatically connect to this network when it is available — Select this box if you want NetworkManager to auto-connect to this connection when it is available. See the section called “Editing an Existing Connection with control-center” for more information.
- All users may connect to this network — Select this box to create a connection available to all users on the system. Changing this setting may require root privileges. See Section 3.4.5, “Managing System-wide and Private Connection Profiles with a GUI” for details.
- Automatically connect to VPN when using this connection — Select this box if you want NetworkManager to auto-connect to a VPN connection when it is available. Select the VPN from the dropdown menu.
- Firewall Zone — Select the Firewall Zone from the dropdown menu. See the Red Hat Enterprise Linux 7 Security Guide for more information on Firewall Zones.
9.4.1.1. Configuring the Bridge Tab
- Interface name
- The name of the interface to the bridge.
- Bridged connections
- One or more port interfaces.
- Aging time
- The time, in seconds, a MAC address is kept in the MAC address forwarding database.
- Enable IGMP snooping
- If required, select the check box to enable IGMP snooping on the device.
- Enable STP (Spanning Tree Protocol)
- If required, select the check box to enable
STP
. - Priority
- The bridge priority; the bridge with the lowest priority will be elected as the root bridge.
- Forward delay
- The time, in seconds, spent in both the Listening and Learning states before entering the Forwarding state. The default is 15 seconds.
- Hello time
- The time interval, in seconds, between sending configuration information in bridge protocol data units (BPDU).
- Max age
- The maximum time, in seconds, to store the configuration information from BPDUs. This value should be twice the Hello Time plus 1 but less than twice the Forwarding delay minus 1.
- Group forward mask
- This property is a mask of group addresses that allows group addresses to be forwarded. In most cases, group addresses in the range from
01:80:C2:00:00:00
to01:80:C2:00:00:0F
are not forwarded by the bridge device. This property is a mask of 16 bits, each corresponding to a group address in the above range, that must be forwarded. Note that theGroup forward mask
property cannot have any of the0
,1
,2
bits set to1
because those addresses are used for Spanning tree protocol (STP), Link Aggregation Control Protocol (LACP) and Ethernet MAC pause frames.
Procedure 9.3. Adding a Port Interface to a Bridge
- To add a port to a bridge, select the Bridge tab in the Editing Bridge connection 1 window. If necessary, open this window by following the procedure in Procedure 9.2, “Editing an Existing Bridge Connection”.
- Click Choose a Connection Type menu appears.. The
- Select the type of connection to be created from the list. Click Create. A window appropriate to the connection type selected appears.
Figure 9.6. The NetworkManager Graphical User Interface Add a Bridge Connection
- Select the Bridge Port tab. Configure Priority and Path cost as required. Note the STP priority for a bridge port is limited by the Linux kernel. Although the standard allows a range of
0
to255
, Linux only allows0
to63
. The default is32
in this case.Figure 9.7. The NetworkManager Graphical User Interface Bridge Port tab
- If required, select the Hairpin mode check box to enable forwarding of frames for external processing. Also known as virtual Ethernet port aggregator (VEPA) mode.
- An Ethernet port, click the Ethernet tab and proceed to the section called “Basic Configuration Options ”, or;
- A Bond port, click the Bond tab and proceed to Section 7.8.1.1, “Configuring the Bond Tab”, or;
- A Team port, click the Team tab and proceed to Section 8.14.1.1, “Configuring the Team Tab”, or;
- An VLAN port, click the VLAN tab and proceed to Section 10.5.1.1, “Configuring the VLAN Tab”, or;
Saving Your New (or Modified) Connection and Making Further Configurations
IPv4
settings for the connection, click the IPv4 Settings tab and proceed to Section 5.4, “Configuring IPv4 Settings”, or;IPv6
settings for the connection, click the IPv6 Settings tab and proceed to Section 5.5, “Configuring IPv6 Settings”.
Figure 9.8. The NetworkManager Graphical User Interface with Bridge
9.5. Ethernet Bridge Configuration Using iproute
priority
, cost
or state
.
ip
utility, issue the following command as root
:
~]# ip link set enp1s0 type bridge_slave option
ip
utility, issue the following command as root
:
~]# ip link help bridge_slave
Usage: ... bridge_slave [ state STATE ] [ priority PRIO ] [cost COST ]
[ guard {on | off} ]
[ hairpin {on | off} ]
[ fastleave {on | off} ]
[ root_block {on | off} ]
[ learning {on | off} ]
[ flood {on | off} ]
For more details on the port options, see the ip-link(8)
man page.
9.6. Additional Resources
nmcli(1)
man page — Describes NetworkManager's command‐line tool.nmcli-examples(5)
man page — Gives examples of nmcli commands.nm-settings(5)
man page — Description of settings and parameters of NetworkManager connections.ip-link(8)
man page — Description of the bridge port options.
Chapter 10. Configure 802.1Q VLAN tagging
- In the case of VLANs over bonds, it is important that the bond has ports and that they are “up” before opening the VLAN interface. Adding a VLAN interface to a bond without ports does not work.
- A VLAN port cannot be configured on a bond with the
fail_over_mac=follow
option, because the VLAN virtual device cannot change its MAC address to match the parent's new MAC address. In such a case, traffic would still be sent with the now incorrect source MAC address. - Sending VLAN tagged packets through a network switch requires the switch to be properly configured. For example, ports on Cisco switches must be assigned to one VLAN or be configured as trunk ports to accept tagged packets from multiple VLANs. Some vendor switches allow untagged frames of the native VLAN to be processed by a trunk port. Some devices allow you to enable or disable the native VLAN, other devices have it disabled by default. Consequence of this disparity may result in native VLAN misconfiguration between two different switches, posing a security risk. For example:One switch uses native VLAN 1 while the other uses native VLAN 10. If the frames are allowed to pass without the tag being inserted, an attacker is able to jump VLANs - this common network penetration technique is also known as VLAN hopping.To minimize security risks, configure your interface as follows:
- Switches
- Unless you need them, disable trunk ports.
- If you need trunk ports, disable native VLAN, so that untagged frames are not allowed.
- Red Hat Enterprise Linux server
- Use the nftables or ebtables utilities to drop untagged frames in ingress filtering.
- Some older network interface cards, loopback interfaces, Wimax cards, and some InfiniBand devices, are said to be VLAN challenged, meaning they cannot support VLANs. This is usually because the devices cannot cope with VLAN headers and the larger MTU size associated with tagged packets.
Note
10.1. Selecting VLAN Interface Configuration Methods
- To configure a VLAN interface using NetworkManager's text user interface tool, nmtui, proceed to Section 10.2, “Configure 802.1Q VLAN tagging Using the Text User Interface, nmtui”
- To configure a VLAN interface using NetworkManager's command-line tool, nmcli, proceed to Section 10.3, “Configure 802.1Q VLAN Tagging Using the Command Line Tool, nmcli”
- To configure a network interface manually, see Section 10.4, “Configure 802.1Q VLAN Tagging Using the Command Line”.
- To configure a network using graphical user interface tools, proceed to Section 10.5, “Configure 802.1Q VLAN Tagging Using a GUI”
10.2. Configure 802.1Q VLAN tagging Using the Text User Interface, nmtui
~]$ nmtui
The text user interface appears. Any invalid command prints a usage message.
Figure 10.1. The NetworkManager Text User Interface Add a VLAN Connection menu
Figure 10.2. The NetworkManager Text User Interface Configuring a VLAN Connection menu
10.3. Configure 802.1Q VLAN Tagging Using the Command Line Tool, nmcli
~]$ nmcli con show
NAME UUID TYPE DEVICE
System enp2s0 9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04 802-3-ethernet enp2s0
System enp1s0 5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 802-3-ethernet enp1s0
Note that the NAME field in the output always denotes the connection ID. It is not the interface name even though it might look the same. The ID can be used in nmcli connection
commands to identify a connection. Use the DEVICE name with other applications such as firewalld
.
10
, issue a command as follows:
~]$ nmcli con add type vlan ifname VLAN10 dev enp1s0 id 10
Connection 'vlan-VLAN10' (37750b4a-8ef5-40e6-be9b-4fb21a4b6d17) successfully added.
Note that as no con-name
was given for the VLAN interface, the name was derived from the interface name by prepending the type. Alternatively, specify a name with the con-name
option as follows:
~]$ nmcli con add type vlan con-name VLAN12 dev enp1s0 id 12
Connection 'VLAN12' (b796c16a-9f5f-441c-835c-f594d40e6533) successfully added.
Assigning Addresses to VLAN Interfaces
IPv4
address and gateway is as follows:
~]$nmcli con add type vlan con-name VLAN20 dev enp1s0 id 20 ip4 10.10.10.10/24 \
gw4 10.10.10.254
~]$ nmcli con add type vlan con-name VLAN30 dev enp1s0 id 30
~]$ nmcli con show
NAME UUID TYPE DEVICE
VLAN12 4129a37d-4feb-4be5-ac17-14a193821755 vlan enp1s0.12
System enp2s0 9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04 802-3-ethernet enp2s0
System enp1s0 5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 802-3-ethernet enp1s0
vlan-VLAN10 1be91581-11c2-461a-b40d-893d42fed4f4 vlan VLAN10
~]$ nmcli -p con show VLAN12
===============================================================================
Connection profile details (VLAN12)
===============================================================================
connection.id: VLAN12
connection.uuid: 4129a37d-4feb-4be5-ac17-14a193821755
connection.interface-name: --
connection.type: vlan
connection.autoconnect: yes
…
-------------------------------------------------------------------------------
802-3-ethernet.port: --
802-3-ethernet.speed: 0
802-3-ethernet.duplex: --
802-3-ethernet.auto-negotiate: yes
802-3-ethernet.mac-address: --
802-3-ethernet.cloned-mac-address: --
802-3-ethernet.mac-address-blacklist:
802-3-ethernet.mtu: auto
…
vlan.interface-name: --
vlan.parent: enp1s0
vlan.id: 12
vlan.flags: 0 (NONE)
vlan.ingress-priority-map:
vlan.egress-priority-map:
-------------------------------------------------------------------------------
===============================================================================
Activate connection details (4129a37d-4feb-4be5-ac17-14a193821755)
===============================================================================
GENERAL.NAME: VLAN12
GENERAL.UUID: 4129a37d-4feb-4be5-ac17-14a193821755
GENERAL.DEVICES: enp1s0.12
GENERAL.STATE: activating
[output truncated]
nmcli(1)
man page. In the man pages the device on which the VLAN is created is referred to as the parent device. In the example above the device was specified by its interface name, enp1s0
, it can also be specified by the connection UUID or MAC address.
13
, issue a command as follows:
~]$ nmcli con add type vlan con-name VLAN1 dev enp2s0 id 13 ingress "2:3,3:5"
~]$ nmcli connection show vlan-VLAN10
~]$ nmcli connection modify vlan-VLAN10 802.mtu 1496
The MTU setting determines the maximum size of the network layer packet. The maximum size of the payload the link-layer frame can carry in turn limits the network layer MTU. For standard Ethernet frames this means an MTU of 1500 bytes. It should not be necessary to change the MTU when setting up a VLAN as the link-layer header is increased in size by 4 bytes to accommodate the 802.1Q tag.
connection.interface-name
and vlan.interface-name
have to be the same (if they are set). They must therefore be changed simultaneously using nmcli's interactive mode. To change a VLAN connections name, issue commands as follows:
~]$nmcli con edit vlan-VLAN10
nmcli>set vlan.interface-name superVLAN
nmcli>set connection.interface-name superVLAN
nmcli>save
nmcli>quit
ioctl
flags which change the way the 802.1Q code functions. The following VLAN flags are supported by NetworkManager:
- 0x01 - reordering of output packet headers
- 0x02 - use GVRP protocol
- 0x04 - loose binding of the interface and its master
0x04
enables a loose binding mode, in which only the operational state is passed from the parent to the associated VLANs, but the VLAN device state is not changed.
~]$ nmcli connection modify vlan-VLAN10 vlan.flags 1
10.4. Configure 802.1Q VLAN Tagging Using the Command Line
8021q
module is loaded by default. If necessary, you can make sure that the module is loaded by issuing the following command as root
:
~]# modprobe --first-time 8021q
modprobe: ERROR: could not insert '8021q': Module already in kernel
To display information about the module, issue the following command:
~]$ modinfo 8021q
See the modprobe(8)
man page for more command options.
10.4.1. Setting Up 802.1Q VLAN Tagging Using ifcfg Files
- Configure the parent interface in
/etc/sysconfig/network-scripts/ifcfg-device_name
, where device_name is the name of the interface:DEVICE=interface_name TYPE=Ethernet BOOTPROTO=none ONBOOT=yes
- Configure the VLAN interface configuration in the
/etc/sysconfig/network-scripts/
directory. The configuration file name should be the parent interface plus a.
character plus the VLAN ID number. For example, if the VLAN ID is 192, and the parent interface is enp1s0, then the configuration file name should beifcfg-enp1s0.192
:DEVICE=enp1s0.192 BOOTPROTO=none ONBOOT=yes IPADDR=192.168.1.1 PREFIX=24 NETWORK=192.168.1.0 VLAN=yes
If there is a need to configure a second VLAN, with for example, VLAN ID 193, on the same interface, enp1s0, add a new file with the nameenp1s0.193
with the VLAN configuration details. - Restart the networking service in order for the changes to take effect. As
root
issue the following command:~]#
systemctl restart network
10.4.2. Configure 802.1Q VLAN Tagging Using ip Commands
8
, issue a command as root
as follows:
~]# ip link add link enp1s0 name enp1s0.8 type vlan id 8
To view the VLAN, issue the following command:
~]$ ip -d link show enp1s0.8
4: enp1s0.8@enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 52:54:00:ce:5f:6c brd ff:ff:ff:ff:ff:ff promiscuity 0
vlan protocol 802.1Q id 8 <REORDER_HDR>
0x
and as an octal value if it has a leading 0
. This means that in order to assign a VLAN ID with a decimal value of 22
, you must not add any leading zeros.
root
as follows:
~]# ip link delete enp1s0.8
~]#ip link add link enp1s0 name enp1s0.1 type vlan id 1
ip link set dev enp1s0.1 up
~]#ip link add link enp1s0 name enp1s0.2 type vlan id 2
ip link set dev enp1s0.2 up
tcpdump -nnei enp1s0 -vvv
Note
ifcfg
files. See Section 10.4.1, “Setting Up 802.1Q VLAN Tagging Using ifcfg Files”
10.5. Configure 802.1Q VLAN Tagging Using a GUI
10.5.1. Establishing a VLAN Connection
Procedure 10.1. Adding a New VLAN Connection Using nm-connection-editor
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Click the Add button. The Choose a Connection Type window appears. Select VLAN and click Create. The Editing VLAN connection 1 window appears.
- On the VLAN tab, select the parent interface from the drop-down list you want to use for the VLAN connection.
- Enter the VLAN ID
- Enter a VLAN interface name. This is the name of the VLAN interface that will be created. For example,
enp1s0.1
orvlan2
. (Normally this is either the parent interface name plus “.
” and the VLAN ID, or “vlan
” plus the VLAN ID.) - Review and confirm the settings and then click the Save button.
- To edit the VLAN-specific settings see Section 10.5.1.1, “Configuring the VLAN Tab”.
Figure 10.3. Adding a New VLAN Connection Using nm-connection-editor
Procedure 10.2. Editing an Existing VLAN Connection
- Enter nm-connection-editor in a terminal:
~]$ nm-connection-editor
- Select the connection you want to edit and click the Edit button.
- Select the General tab.
- Configure the connection name, auto-connect behavior, and availability settings.These settings in the Editing dialog are common to all connection types:
- Connection name — Enter a descriptive name for your network connection. This name will be used to list this connection in the VLAN section of the Network window.
- Automatically connect to this network when it is available — Select this box if you want NetworkManager to auto-connect to this connection when it is available. Refer to the section called “Editing an Existing Connection with control-center” for more information.
- Available to all users — Select this box to create a connection available to all users on the system. Changing this setting may require root privileges. Refer to Section 3.4.5, “Managing System-wide and Private Connection Profiles with a GUI” for details.
- To edit the VLAN-specific settings see Section 10.5.1.1, “Configuring the VLAN Tab”.
Saving Your New (or Modified) Connection and Making Further Configurations
IPv4
settings for the connection, click the IPv4 Settings tab and proceed to Section 5.4, “Configuring IPv4 Settings”.orIPv6
settings for the connection, click the IPv6 Settings tab and proceed to Section 5.5, “Configuring IPv6 Settings”.
10.5.1.1. Configuring the VLAN Tab
- Parent Interface
- A previously configured interface can be selected in the drop-down list.
- VLAN ID
- The identification number to be used to tag the VLAN network traffic.
- VLAN interface name
- The name of the VLAN interface that will be created. For example,
enp1s0.1
orvlan2
. - Cloned MAC address
- Optionally sets an alternate MAC address to use for identifying the VLAN interface. This can be used to change the source MAC address for packets sent on this VLAN.
- MTU
- Optionally sets a Maximum Transmission Unit (MTU) size to be used for packets to be sent over the VLAN connection.
10.6. VLAN on Bond and Bridge Using ip Commands
- Add a bond device as
root
:# ip link add bond0 type bond # ip link set bond0 type bond miimon 100 mode active-backup # ip link set em1 down # ip link set em1 master bond0 # ip link set em2 down # ip link set em2 master bond0 # ip link set bond0 up
- Set VLAN on the bond device:
# ip link add link bond0 name bond0.2 type vlan id 2 # ip link set bond0.2 up
- Add the bridge device and attach VLAN to it:
# ip link add br0 type bridge # ip link set bond0.2 master br0 # ip link set br0 up
10.7. VLAN on Bond and Bridge Using the NetworkManager Command Line Tool, nmcli
- Add a bond device:
~]$
Note that in this case a bond connection serves only as a "lower interface" for VLAN, and does not get any IP address. Therefore, thenmcli connection add type bond con-name Bond0 ifname bond0 bond.options "mode=active-backup,miimon=100" ipv4.method disabled ipv6.method ignore
ipv4.method disabled
andipv6.method ignore
parameters have been added on the command line. - Add ports to the bond device:
~]$
nmcli connection add type ethernet con-name Slave1 ifname em1 master bond0 slave-type bond
~]$nmcli connection add type ethernet con-name Slave2 ifname em2 master bond0 slave-type bond
- Add a bridge device:
~]$
nmcli connection add type bridge con-name Bridge0 ifname br0 ipv4.method manual ipv4.addresses 192.0.2.1/24
- Add a VLAN interface on top of bond, assigned to the bridge device:
~]$
nmcli connection add type vlan con-name Vlan2 ifname bond0.2 dev bond0 id 2 master br0 slave-type bridge
- View the created connections:
~]$
nmcli connection show
NAME UUID TYPE DEVICE Bond0 f05806fa-72c3-4803-8743-2377f0c10bed bond bond0 Bridge0 22d3c0de-d79a-4779-80eb-10718c2bed61 bridge br0 Slave1 e59e13cb-d749-4df2-aee6-de3bfaec698c 802-3-ethernet em1 Slave2 25361a76-6b3c-4ae5-9073-005be5ab8b1c 802-3-ethernet em2 Vlan2 e2333426-eea4-4f5d-a589-336f032ec822 vlan bond0.2
10.8. Configuring VLAN switchport mode
switchport mode
when the Ethernet interface is connected to a switch and there are VLANs running over the physical interface. A Red Hat Enterprise Linux server or workstation is usually connected to only one VLAN, which makes switchport mode access
suitable, and the default setting.
switchport mode trunk
to be configured on both ends.
switchport mode trunk
. If you only set an IP address on an interface, use Cisco switchport mode access
.
10.9. Additional Resources
ip-link(8)
man page — Describes the ip utility's network device configuration commands.nmcli(1)
man page — Describes NetworkManager's command‐line tool.nmcli-examples(5)
man page — Gives examples of nmcli commands.nm-settings(5)
man page — Description of settings and parameters of NetworkManager connections.nm-settings-ifcfg-rh(5)
man page — Description of ifcfg-rh settings in the/etc/sysconfig/network-scripts/ifcfg-*
files.
Chapter 11. Consistent Network Device Naming
eth[0123…]s0
, but these names do not necessarily correspond to actual labels on the chassis. Modern server platforms with multiple network adapters can encounter non-deterministic and counter-intuitive naming of these interfaces. This affects both network adapters embedded on the motherboard (Lan-on-Motherboard, or LOM) and add-in (single and multiport) adapters.
Warning
11.1. Naming Schemes Hierarchy
systemd
will name interfaces using the following policy to apply the supported naming schemes:
- Scheme 1: Names incorporating Firmware or BIOS provided index numbers for on-board devices (example:
eno1
), are applied if that information from the firmware or BIOS is applicable and available, else falling back to scheme 2. - Scheme 2: Names incorporating Firmware or BIOS provided PCI Express hotplug slot index numbers (example:
ens1
) are applied if that information from the firmware or BIOS is applicable and available, else falling back to scheme 3. - Scheme 3: Names incorporating physical location of the connector of the hardware (example:
enp2s0
), are applied if applicable, else falling directly back to scheme 5 in all other cases. - Scheme 4: Names incorporating interface's MAC address (example:
enx78e7d1ea46da
), is not used by default, but is available if the user chooses. - Scheme 5: The traditional unpredictable kernel naming scheme, is used if all other methods fail (example:
eth0
).
biosdevname=1
as a kernel command-line parameter, except in the case of a Dell system, where biosdevname will be used by default as long as it is installed. If the user has added udev rules which change the name of the kernel devices, those rules will take precedence.
11.2. Understanding the Device Renaming Procedure
- A rule in
/usr/lib/udev/rules.d/60-net.rules
instructs the udev helper utility, /lib/udev/rename_device, to look into all/etc/sysconfig/network-scripts/ifcfg-suffix
files. If it finds anifcfg
file with aHWADDR
entry matching the MAC address of an interface it renames the interface to the name given in theifcfg
file by theDEVICE
directive. - A rule in
/usr/lib/udev/rules.d/71-biosdevname.rules
instructs biosdevname to rename the interface according to its naming policy, provided that it was not renamed in a previous step, biosdevname is installed, andbiosdevname=0
was not given as a kernel command on the boot command line. - A rule in
/lib/udev/rules.d/75-net-description.rules
instructs udev to fill in the internal udev device property values ID_NET_NAME_ONBOARD, ID_NET_NAME_SLOT, ID_NET_NAME_PATH, ID_NET_NAME_MAC by examining the network interface device. Note, that some device properties might be undefined. - A rule in
/usr/lib/udev/rules.d/80-net-name-slot.rules
instructs udev to rename the interface, provided that it was not renamed in step 1 or 2, and the kernel parameternet.ifnames=0
was not given, according to the following priority: ID_NET_NAME_ONBOARD, ID_NET_NAME_SLOT, ID_NET_NAME_PATH. It falls through to the next in the list, if one is unset. If none of these are set, then the interface will not be renamed.
11.3. Understanding the Predictable Network Interface Device Names
en
for Ethernet,wl
for wireless LAN (WLAN),ww
for wireless wide area network (WWAN).
- o<index>
- on-board device index number
- s<slot>[f<function>][d<dev_id>]
- hotplug slot index number. All multi-function PCI devices will carry the
[f<function>]
number in the device name, including the function0
device. - x<MAC>
- MAC address
- [P<domain>]p<bus>s<slot>[f<function>][d<dev_id>]
- PCI geographical location. In PCI geographical location, the
[P<domain>]
number is only mentioned if the value is not0
. For example:ID_NET_NAME_PATH=P1enp5s0
- [P<domain>]p<bus>s<slot>[f<function>][u<port>][..][c<config>][i<interface>]
- USB port number chain. For USB devices, the full chain of port numbers of hubs is composed. If the name gets longer than the maximum number of 15 characters, the name is not exported. If there are multiple USB devices in the chain, the default values for USB configuration descriptors
(c1)
and USB interface descriptors(i0)
are suppressed.
11.4. Naming Scheme for Network Devices Available for Linux on System z
0.n
, where n
is the subchannel set ID. For example, 0.1.0ab1
.
enccw0.0.1234
slccw0.0.1234
znetconf -c
command or the lscss -a
command to display available network devices and their bus-IDs.
Format | Description |
---|---|
enccwbus-ID | device type Ethernet |
slccwbus-ID | CTC network devices of device type SLIP |
11.5. Naming Scheme for VLAN Interfaces
VLAN-ID
ranges from 0
to 4096
, which is a maximum of four characters and the total interface name has a limit of 15 characters. The maximum interface name length is defined by the kernel headers and is a global limit, affecting all applications.
- VLAN plus VLAN ID
- The word
vlan
plus the VLAN ID. For example: vlan0005 - VLAN plus VLAN ID without padding
- The word
vlan
plus the VLAN ID without padding by means of additional leading zeros. For example: vlan5 - Device name plus VLAN ID
- The name of the parent interface plus the VLAN ID. For example: enp1s0.0005
- Device name plus VLAN ID without padding
- The name of the parent interface plus the VLAN ID without padding by means of additional leading zeros. For example: enp1s0.5
11.6. Consistent Network Device Naming Using biosdevname
eth[0123…]
to the new naming convention as shown in Table 11.2, “The biosdevname Naming Convention”. Note that unless the system is a Dell system, or biosdevname is explicitly enabled as described in Section 11.6.2, “Enabling and Disabling the Feature”, the systemd
naming scheme will take precedence.
Device | Old Name | New Name |
---|---|---|
Embedded network interface (LOM) | eth[0123…] | em[1234…] [a] |
PCI card network interface | eth[0123…] | p<slot>p<ethernet port> [b] |
Virtual function | eth[0123…] | p<slot>p<ethernet port>_<virtual interface> [c] |
[a]
New enumeration starts at 1 .
[b]
For example: p3p4
[c]
For example: p3p4_1
|
11.6.1. System Requirements
root
:
~]# yum install biosdevname
11.6.2. Enabling and Disabling the Feature
biosdevname=0
biosdevname=1
systemd
naming scheme as described in the beginning of the chapter.
biosdevname
install option is specified, it must remain as a boot option for the lifetime of the system.
11.7. Notes for Administrators
irqbalance
, and other similar configuration files. Also, enabling this change for installation will require modification to existing kickstart files that use device names through the ksdevice
parameter; these kickstart files will need to be updated to use the network device's MAC address or the network device's new name.
Note
11.8. Controlling the Selection of Network Device Names
- By identifying the network interface device
- Setting the MAC address in an
ifcfg
file using theHWADDR
directive enables it to be identified by udev. The name will be taken from the string given by theDEVICE
directive, which by convention is the same as theifcfg
suffix. For example,ifcfg
-enp1s0. - By turning on or off biosdevname
- The name provided by biosdevname will be used (if biosdevname can determine one).
- By turning on or off the
systemd-udev
naming scheme - The name provided by
systemd-udev
will be used (ifsystemd-udev
can determine one).
11.9. Troubleshooting Network Device Naming
root
:
~]# udevadm info /sys/class/net/ifname | grep ID_NET_NAME
where ifname is one of the interfaces listed by the following command:
~]$ ls /sys/class/net/
/usr/lib/udev/rules.d/60-net.rules
- from initscripts,/usr/lib/udev/rules.d/71-biosdevname.rules
- from biosdevname,/usr/lib/udev/rules.d/80-net-name-slot.rules
- fromsystemd
80-net-name-slot.rules
from /usr
to /etc
and edit the file appropriately. In other words, comment out or arrange schemes to be used in a certain order.
Example 11.1. Some Interfaces Have Names from the Kernel Namespace (eth[0,1,2...]) While Others Are Successfully Renamed by udev
Example 11.2. In /var/log/messages or the systemd Journal, Renaming Is Seen to Be Done Twice for Each Interface
initrd
image are likely to encounter this issue. The interface name is initially assigned (through biosdevname or udev or dracut parameters on the kernel command line) during early-boot while still in initrd
. Then after switching to real rootfs
, renaming is done a second time and a new interface name is determined by the /usr/lib/udev/rename_device
binary spawned by udev because of processing 60-net.rules. You can safely ignore such messages.
Example 11.3. Using Naming Scheme in ifcfg Files with ethX Names Does Not Work
/usr/lib/udev/rules.d/60-net.rules
file.
- The system has only one network interface.
- When used for virtio NICs in Red Hat Enterprise Linux 7 virtual machine guests. See the KVM Paravirtualized (virtio) Drivers and Network Configuration chapters in the Virtualization Deployment and Administration Guide
Example 11.4. Setting net.ifnames=0 Results in Inconsistent enpXxX Names
systemd
predictable interface naming (net.ifnames
) and biosdevname naming schemes are disabled, network interfaces continue to use the unpredictable and potentially inconsistent ethX name originally given by the kernel.
systemd
predictable interface naming scheme or the biosdevname naming scheme to rename the kernel unpredictable ethX interfaces in a predictable way to a name which is always consistent across reboots.
Example 11.5. Limitations for Prefixes of Ethernet Interfaces
- It consists of ASCII characters.
- It is an alpha-numeric string.
- It is shorter than 16 characters.
- It does not conflict with any other well-known prefix used for network interface naming, such as
eth
,eno
,ens
, andem
.
11.10. Additional Resources
Installed Documentation
udev(7)
man page — Describes the Linux dynamic device management daemon,udevd
.systemd(1)
man page — Describessystemd
system and service manager.biosdevname(1)
man page — Describes the utility for obtaining the BIOS-given name of a device.
Online Documentation
- The IBM Knowledge Center Publication SC34-2710-00 Device Drivers, Features, and Commands on Red Hat Enterprise Linux 7 includes information on “Predictable network device names” for IBM System z devices and attachments.
Chapter 12. Configuring Policy-based Routing to Define Alternative Routes
Note
nmcli
utility supports setting routing rules and assigning routes to specific tables.
12.1. Routing Traffic from a Specific Subnet to a Different Default Gateway
Figure 12.1. Activate a Connection
Prerequisites
- The RHEL router you want to set up in the procedure has four network interfaces:
- The
enp7s0
interface is connected to the network of provider A. The gateway IP in the provider’s network is198.51.100.2
, and the network uses a/30
network mask. - The
enp1s0
interface is connected to the network of provider B. The gateway IP in the provider’s network is192.0.2.2
, and the network uses a/30
network mask. - The
enp8s0
interface is connected to the10.0.0.0/24
subnet with internal workstations. - The
enp9s0
interface is connected to the203.0.113.0/24
subnet with the company’s servers.
- Hosts in the internal workstations subnet use
10.0.0.1
as default gateway. In the procedure, you assign this IP address to theenp8s0
network interface of the router. - Hosts in the server subnet use
203.0.113.1
as default gateway. In the procedure, you assign this IP address to theenp9s0
network interface of the router. - The
firewalld
service is enabled and active, which is the default.
Procedure
- Configure the network interface to provider A:
# nmcli connection add type ethernet con-name Provider-A ifname enp7s0 ipv4.method manual ipv4.addresses 198.51.100.1/30 ipv4.gateway 198.51.100.2 ipv4.dns 198.51.100.200 connection.zone external
Thenmcli connection add
command creates a NetworkManager connection profile. The following list describes the options of the command:type ethernet
: Defines that the connection type is Ethernet.con-name connection_name
: Sets the name of the profile. Use a meaningful name to avoid confusion.ifname network_device
: Sets the network interface.ipv4.method manual
: Enables to configure a static IP address.ipv4.addresses IP_address/subnet_mask
: Sets the IPv4 addresses and subnet mask.ipv4.gateway IP_address
: Sets the default gateway address.ipv4.dns IP_of_DNS_server
: Sets the IPv4 address of the DNS server.connection.zone firewalld_zone
: Assigns the network interface to the definedfirewalld
zone. Note thatfirewalld
automatically enables masquerading interfaces assigned to theexternal
zone.
- Configure the network interface to provider B:
# nmcli connection add type ethernet con-name Provider-B ifname enp1s0 ipv4.method manual ipv4.addresses 192.0.2.1/30 ipv4.routes "0.0.0.0/1 192.0.2.2 table=5000, 128.0.0.0/1 192.0.2.2 table=5000" connection.zone external
This command uses theipv4.routes
parameter instead ofipv4.gateway
to set the default gateway. This is required to assign the default gateway for this connection to a different routing table (5000
) than the default. NetworkManager automatically creates this new routing table when the connection is activated.Note
Thenmcli
utility does not support using0.0.0.0/0
for the default gateway inipv4.gateway
. To work around this problem, the command creates separate routes for both the0.0.0.0/1
and128.0.0.0/1
subnets, which covers also the full IPv4 address space. - Configure the network interface to the internal workstations subnet:
# nmcli connection add type ethernet con-name Internal-Workstations ifname enp8s0 ipv4.method manual ipv4.addresses 10.0.0.1/24 ipv4.routes "10.0.0.0/24 src=192.0.2.1 table=5000" ipv4.routing-rules "priority 5 from 10.0.0.0/24 table 5000" connection.zone trusted
This command uses theipv4.routes
parameter to add a static route to the routing table with ID5000
. This static route for the10.0.0.0/24
subnet uses the IP of the local network interface to provider B (192.0.2.1
) as next hop.Additionally, the command uses theipv4.routing-rules
parameter to add a routing rule with priority5
that routes traffic from the10.0.0.0/24
subnet to table5000
. Low values have a high priority.Note that the syntax in theipv4.routing-rules
parameter is the same as in anip route add
command, except thatipv4.routing-rules
always requires specifying a priority. - Configure the network interface to the server subnet:
# nmcli connection add type ethernet con-name Servers ifname enp9s0 ipv4.method manual ipv4.addresses 203.0.113.1/24 connection.zone trusted
Verification Steps
- On a RHEL host in the internal workstation subnet:
- Install the traceroute package:
# yum install traceroute
- Use the
traceroute
utility to display the route to a host on the internet:# traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 10.0.0.1 (10.0.0.1) 0.337 ms 0.260 ms 0.223 ms 2 192.0.2.1 (192.0.2.1) 0.884 ms 1.066 ms 1.248 ms ...
The output of the command displays that the router sends packets over192.0.2.1
, which is the network of provider B.
- On a RHEL host in the server subnet:
- Install the traceroute package:
# yum install traceroute
- Use the
traceroute
utility to display the route to a host on the internet:# traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 203.0.113.1 (203.0.113.1) 2.179 ms 2.073 ms 1.944 ms 2 198.51.100.2 (198.51.100.2) 1.868 ms 1.798 ms 1.549 ms ...
The output of the command displays that the router sends packets over198.51.100.2
, which is the network of provider A.
Troubleshooting Steps
- Display the rule list:
# ip rule list 0: from all lookup local 5: from 10.0.0.0/24 lookup 5000 32766: from all lookup main 32767: from all lookup default
- Display the routes in table
5000
:# ip route list table 5000 0.0.0.0/1 via 192.0.2.2 dev enp1s0 proto static metric 100 10.0.0.0/24 dev enp8s0 proto static scope link src 192.0.2.1 metric 102 128.0.0.0/1 via 192.0.2.2 dev enp1s0 proto static metric 100
- Display which interfaces are assigned to which firewall zones:
# firewall-cmd --get-active-zones external interfaces: enp1s0 enp7s0 trusted interfaces: enp8s0 enp9s0
- Verify that the
external
zone has masquerading enabled:# firewall-cmd --info-zone=external external (active) target: default icmp-block-inversion: no interfaces: enp1s0 enp7s0 sources: services: ssh ports: protocols: masquerade: yes ...
Additional Resources
- For further details about the
ipv4.*
parameters you can set in thenmcli connection add
command, see the IPv4 settings section in the nm-settings(5) man page. - For further details about the
connection.*
parameters you can set in thenmcli connection add
command, see the Connection settings section in the nm-settings(5) man page. - For further details about managing NetworkManager connections using
nmcli
, see the Connection management commands section in the nmcli(1) man page.
Part III. InfiniBand and RDMA Networking
Chapter 13. Configure InfiniBand and RDMA Networks
13.1. Understanding InfiniBand and RDMA technologies
IP
data transfer, application X on machine A sends some data to application Y on machine B. As part of the transfer, the kernel on machine B must first receive the data, decode the packet headers, determine that the data belongs to application Y, wake up application Y, wait for application Y to perform a read syscall into the kernel, then it must manually copy the data from the kernel's own internal memory space into the buffer provided by application Y. This process means that most network traffic must be copied across the system's main memory bus at least twice (once when the host adapter uses DMA to put the data into the kernel-provided memory buffer, and again when the kernel moves the data to the application's memory buffer) and it also means the computer must execute a number of context switches to switch between kernel context and application Y context. Both of these things impose extremely high CPU loads on the system when network traffic is flowing at very high rates and can make other tasks to slow down.
IP
communications because they bypass kernel intervention in the communication process, and in the process greatly reduce the CPU overhead normally needed to process network communications. The RDMA protocol allows the host adapter in the machine to know when a packet comes in from the network, which application should receive that packet, and where in the application's memory space it should go. Instead of sending the packet to the kernel to be processed and then copied into the user application's memory, it places the contents of the packet directly in the application's buffer without any further intervention necessary. However, it cannot be accomplished using the standard Berkeley Sockets API that most IP
networking applications are built upon, so it must provide its own API, the InfiniBand Verbs API, and applications must be ported to this API before they can use RDMA technology directly.
- The Internet Wide Area RDMA Protocol (iWARP)iWARP is a computer networking protocol that implements remote direct memory access (RDMA) for efficient data transfer over Internet Protocol (IP) networks.
- The RDMA over Converged Ethernet (RoCE) protocol, which later renamed to InfiniBand over Ethernet (IBoE).RoCE is a network protocol that allows remote direct memory access (RDMA) over an Ethernet network.
Prerequisites
IP
network link layer as their underlying technology, and so the majority of their configuration is actually covered in Chapter 3, Configuring IP Networking. For the most part, once their IP
networking features are properly configured, their RDMA features are all automatic and will show up as long as the proper drivers for the hardware are installed. The kernel drivers are always included with each kernel Red Hat provides, however the user-space drivers must be installed manually if the InfiniBand package group was not selected at machine install time.
root
:
~]# yum install libibverbs
/etc/modprobe.d/mlx4.conf
to instruct the driver which packet priority is configured for the “no-drop” service on the Ethernet switches the cards are plugged into and rebuild the initramfs
to include the modified file. Newer mlx5-based cards auto-negotiate PFC settings with the switch and do not need any module option to inform them of the “no-drop” priority or priorities.
13.2. Transferring Data Using RoCE
- RoCE v1
- The
RoCE v1
protocol is an Ethernet link layer protocol with ethertype0x8915
that enables communication between any two hosts in the same Ethernet broadcast domain. RoCE v1 is the default version for RDMA Connection Manager (RDMA_CM) when using the ConnectX-3 network adapter. - RoCE v2
- The
RoCE v2
protocol exists on top of either the UDP over IPv4 or the UDP over IPv6 protocol. The UDP destination port number4791
has been reserved for RoCE v2. Since Red Hat Enterprise Linux 7.5, RoCE v2 is the default version for RDMA_CM when using the ConnectX-3 Pro, ConnectX-4, ConnectX-4 Lx and ConnectX-5 network adapters. Hardware supports bothRoCE v1
andRoCE v2
.
Prerequisites
- Both client and server support the same RoCE mode.
- A client supports RoCE v1 and a server RoCE v2.
- A successful connection:
- If a client is in RoCE v1 or in RoCE v2 mode depending on the network card and the driver used, the corresponding server must have the same version to create a connection. Also, the connection is successful if a client is in RoCE v1 and a server in RoCE v2 mode.
- A failed connection:
- If a client is in RoCE v2 and the corresponding server is in RoCE v1, no connection can be established. In this case, update the driver or the network adapter of the corresponding server, see Section 13.2, “Transferring Data Using RoCE”
Client | Server | Default setting |
---|---|---|
RoCE v1 | RoCE v1 | Connection |
RoCE v1 | RoCE v2 | Connection |
RoCE v2 | RoCE v2 | Connection |
RoCE v2 | RoCE v1 | No connection |
Procedure 13.1. Changing the Default RoCE Mode When the Hardware Is Already Running in Roce v2
- Change into the
/sys/kernel/config/rdma_cm
directory to et the RoCE mode:~]#
cd /sys/kernel/config/rdma_cm
- Enter the
ibstat
command with an Ethernet network device to display the status. For example, for mlx5_0:~]$
ibstat
mlx5_0 CA 'mlx5_0' CA type: MT4115 Number of ports: 1 Firmware version: 12.17.1010 Hardware version: 0 Node GUID: 0x248a0703004bf0a4 System image GUID: 0x248a0703004bf0a4 Port 1: State: Active Physical state: LinkUp Rate: 40 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 0x04010000 Port GUID: 0x268a07fffe4bf0a4 Link layer: Ethernet - Create a directory for the mlx5_0 device:
~]#
mkdir
mlx5_0 - Display the RoCE mode in the
default_roce_mode
file in the tree format:~]#
cd
mlx5_0~]$ tree └── ports └── 1 ├── default_roce_mode └── default_roce_tos
~]$
cat /sys/kernel/config/rdma_cm/mlx5_0/ports/1/default_roce_mode
RoCE v2 - Change the default RoCE mode:
~]#
echo "RoCE v1" > /sys/kernel/config/rdma_cm/mlx5_0/ports/1/default_roce_mode
- View the changes:
~]$
cat /sys/kernel/config/rdma_cm/mlx5_0/ports/1/default_roce_mode
RoCE v1
13.3. Configuring Soft-RoCE
Prerequisites
rxe_cfg
script. To view options for rxe_cfg
, enter rxe_cfg help
.
Procedure 13.2. Configuring Soft-RoCE
- As the
root
user, enter the following command to display the current configuration status of RXE:~]#
rxe_cfg
rdma_rxe module not loaded Name Link Driver Speed NMTU IPv4_addr RDEV RMTU igb_1yes
igb mlx4_1 no mlx4_en mlx4_2 no mlx4_en - To load the RXE kernel module and start RXE, enter as
root
:~]#
Optionally, to verify that the RXE kernel module is loaded, enter:rxe_cfg start
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU igb_1yes
igb mlx4_1 no mlx4_en mlx4_2 no mlx4_en~]#
lsmod |grep rdma_rxe
rdma_rxe 111129 0 ip6_udp_tunnel 12755 1 rdma_rxe udp_tunnel 14423 1 rdma_rxe ib_core 236827 15 rdma_cm,ib_cm,iw_cm,rpcrdma,mlx4_ib,ib_srp,ib_ucm,ib_iser,ib_srpt,ib_umad,ib_uverbs,rdma_rxe,rdma_ucm,ib_ipoib,ib_isert - Before adding a new RXE device over an Ethernet interface, the corresponding interface should be opened and has a valid IP address assigned. To add a new RXE device, for example igb_1:
~]#
rxe_cfg add igb_1
~]#
The rxe0 in the RDEV column indicates that rxe is enabled for the igb_1 device.rxe_cfg status
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU igb_1 yes igb rxe0 1024 (3) mlx4_1 no mlx4_en mlx4_2 no mlx4_en - To verify the status of an RXE device, use the
ibv_devices
command:~]#
Alternatively, enter theibv_devices
device node GUID ------ ---------------- mlx4_0 0002c90300b3cff0 rxe0 a2369ffffe018294ibstat
for a detailed status:~]#
ibstat
rxe0 CA 'rxe0' CA type: Number of ports: 1 Firmware version: Hardware version: Node GUID: 0xa2369ffffe018294 System image GUID: 0x0000000000000000 Port 1: State: Active Physical state: LinkUp Rate: 2.5 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 0x00890000 Port GUID: 0xa2369ffffe018294 Link layer: Ethernet
Removing an RXE device
~]# rxe_cfg remove igb_1
Verifying Connectivity of an RXE device
Example 13.1. Verifying Connectivity of an RXE device on the Server Side
~]$ ibv_rc_pingpong -d rxe0 -g 0
local address: LID 0x0000, QPN 0x000012, PSN 0xe2965f, GID fe80::290:faff:fe29:486a
remote address: LID 0x0000, QPN 0x000011, PSN 0x4bf206, GID fe80::290:faff:fe29:470a
8192000 bytes in 0.05 seconds = 1244.06 Mbit/sec
1000 iters in 0.05 seconds = 52.68 usec/iter
Example 13.2. Verifying Connectivity of an RXE device on the Client Side
~]$ ibv_rc_pingpong -d rxe0 -g 0 172.31.40.4
local address: LID 0x0000, QPN 0x000011, PSN 0x4bf206, GID fe80::290:faff:fe29:470a
remote address: LID 0x0000, QPN 0x000012, PSN 0xe2965f, GID fe80::290:faff:fe29:486a
8192000 bytes in 0.05 seconds = 1245.72 Mbit/sec
1000 iters in 0.05 seconds = 52.61 usec/iter
13.5. Configuring the Base RDMA Subsystem
rdma
service is automatic. When RDMA capable hardware, whether InfiniBand or iWARP or RoCE/IBoE is detected, udev instructs systemd
to start the rdma
service.
~]# systemctl status rdma
● rdma.service - Initialize the iWARP/InfiniBand/RDMA stack in the kernel
Loaded: loaded (/usr/lib/systemd/system/rdma.service; disabled; vendor preset: disabled)
Active: inactive (dead)
Docs: file:/etc/rdma/rdma.conf
rdma
service, but they can if they want to force it on all the time. To do that, enter the following command as root:
~]# systemctl enable rdma
13.5.1. Configuration of the rdma.conf file
rdma
service reads /etc/rdma/rdma.conf
to find out which kernel-level and user-level RDMA protocols the administrator wants to be loaded by default. Users should edit this file to turn various drivers on or off.
IPoIB
— This is anIP
network emulation layer that allowsIP
applications to run over InfiniBand networks.SRP
— This is the SCSI Request Protocol. It allows a machine to mount a remote drive or drive array that is exported through theSRP
protocol on the machine as though it were a local hard disk.SRPT
— This is the target mode, or server mode, of theSRP
protocol. This loads the kernel support necessary for exporting a drive or drive array for other machines to mount as though it were local on their machine. Further configuration of the target mode support is required before any devices will actually be exported. See the documentation in the targetd and targetcli packages for further information.ISER
— This is a low-level driver for the general iSCSI layer of the Linux kernel that provides transport over InfiniBand networks for iSCSI devices.RDS
— This is the Reliable Datagram Service in the Linux kernel. It is not enabled in Red Hat Enterprise Linux 7 kernels and so cannot be loaded.
13.5.2. Usage of 70-persistent-ipoib.rules
/etc/udev.d/rules.d/70-persistent-ipoib.rules
. This udev rules file is used to rename IPoIB devices from their default names (such as ib0
and ib1
) to more descriptive names. Users must edit this file to change how their devices are named. First, find out the GUID address for the device to be renamed:
~]$ ip link show ib0
8: ib0: >BROADCAST,MULTICAST,UP,LOWER_UP< mtu 65520 qdisc pfifo_fast state UP mode DEFAULT qlen 256
link/infiniband 80:00:02:00:fe:80:00:00:00:00:00:00:f4:52:14:03:00:7b:cb:a1 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
Immediately after link/infiniband
is the 20 byte hardware address for the IPoIB interface. The final 8 bytes of the address, marked in bold above, is all that is required to make a new name. Users can make up whatever naming scheme suits them. For example, use a device_fabric naming convention such as mlx4_ib0
if a mlx4
device is connected to an ib0
subnet fabric. The only thing that is not recommended is to use the standard names, like ib0
or ib1
, as these can conflict with the kernel assigned automatic names. Next, add an entry in the rules file. Copy the existing example in the rules file, replace the 8 bytes in the ATTR{address}
entry with the highlighted 8 bytes from the device to be renamed, and enter the new name to be used in the NAME
field.
13.5.3. Relaxing memlock restrictions for users
root
to run large RDMA applications, it will likely be necessary to increase the amount of memory that non-root
users are allowed to pin in the system. This is done by adding a file in the /etc/security/limits.d/
directory with contents such as the following:
~]$ more /etc/security/limits.d/rdma.conf
# configuration for rdma tuning
* soft memlock unlimited
* hard memlock unlimited
# rdma tuning end
13.5.4. Configuring Mellanox cards for Ethernet operation
/etc/rdma/mlx4.conf
file and follow the instructions in that file to set the port types properly for RoCE/IBoE usage. In this case is also necessary to rebuild the initramfs
to make sure the updated port settings are copied into the initramfs
.
mlx4_core 0000:05:00.0: Requested port type for port 1 is not supported on this HCAThis is normal and does not affect operation. The script responsible for setting the port type has no way of knowing when the driver has finished switching port 2 to the requested type internally, and from the time that the script issues a request for port 2 to switch until that switch is complete, the attempts to set port 1 to a different type get rejected. The script retries until the command succeeds or until a timeout has passed indicating that the port switch never completed.
13.5.5. Connecting to a Remote Linux SRP Target
ib_qib
, can have different format of GIDs. The ACL ID also depends on how you initiate the connection request.
Connecting to a Remote Linux SRP Target: High-Level Overview
- Prepare the target side:
- Create storage back end. For example get the /dev/sdc1 partition:
/> /backstores/block create vol1 /dev/sdc1
- Create an SRP target:
/> /srpt create 0xfe80000000000000001175000077dd7e
- Create a LUN based on the back end created in step a:
/> /srpt/ib.fe80000000000000001175000077dd7e/luns create /backstores/block/vol1
- Create a Node ACL for the remote SRP client:
/> /srpt/ib.fe80000000000000001175000077dd7e/acls create 0x7edd770000751100001175000077d708
Note that the Node ACL is different forsrp_daemon
andibsrpdm
.
- Initiate an SRP connection with
srp_daemon
oribsrpdm
for the client side:[root@initiator]# srp_daemon -e -n -i qib0 -p 1 -R 60 &
[root@initiator]# ibsrpdm -c -d /dev/infiniband/umad0 > /sys/class/infiniband_srp/srp-qib0-1/add_target
- Optional. It is recommended to verify the SRP connection with different tools, such as
lsscsi
ordmesg
.
Procedure 13.3. Connecting to a Remote Linux SRP Target with srp_daemon or ibsrpdm
- Use the
ibstat
command on the target to determine theState
andPort GUID
values. The HCA must be inActive
state. The ACL ID is based on thePort GUID
:[root@target]#
ibstat
CA 'qib0' CA type: InfiniPath_QLE7342 Number of ports: 1 Firmware version: Hardware version: 2 Node GUID: 0x001175000077dd7e System image GUID: 0x001175000077dd7e Port 1: State:Active
Physical state: LinkUp Rate: 40 Base lid: 1 LMC: 0 SM lid: 1 Capability mask: 0x0769086a Port GUID:0x001175000077dd7e
Link layer: InfiniBand - Get the SRP target ID, which is based on the GUID of the HCA port. Note that you need a dedicated disk partition as a back end for a SRP target, for example
/dev/sdc1
. The following command replaces the default prefix of fe80, removes the colon, and adds the new prefix to the remainder of the string:[root@target]#
ibstatus | grep '<default-gid>' | sed -e 's/<default-gid>://' -e 's/://g' | grep 001175000077dd7e
fe80000000000000001175000077dd7e - Use the
targetcli
tool to create the LUN vol1 on the block device, create an SRP target, and export the LUN:[root@target]#
targetcli
/>/backstores/block create vol1 /dev/sdc1
Created block storage object vol1 using /dev/sdc1. />/srpt create 0xfe80000000000000001175000077dd7e
Created target ib.fe80000000000000001175000077dd7e. />/srpt/ib.fe80000000000000001175000077dd7e/luns create /backstores/block/vol1
Created LUN 0. />ls /
o- / ............................................................................. [...] o- backstores .................................................................. [...] | o- block ...................................................... [Storage Objects: 1] | | o- vol1 ............................... [/dev/sdc1 (77.8GiB) write-thru activated] | o- fileio ..................................................... [Storage Objects: 0] | o- pscsi ...................................................... [Storage Objects: 0] | o- ramdisk .................................................... [Storage Objects: 0] o- iscsi ................................................................ [Targets: 0] o- loopback ............................................................. [Targets: 0] o- srpt ................................................................. [Targets: 1] o- ib.fe80000000000000001175000077dd7e ............................... [no-gen-acls] o- acls ................................................................ [ACLs: 0] o- luns ................................................................ [LUNs: 1] o- lun0 ............................................... [block/vol1 (/dev/sdc1)] /> - Use the
ibstat
command on the initiator to check if the state isActive
and determine thePort GUID
:[root@initiator]#
ibstat
CA 'qib0' CA type: InfiniPath_QLE7342 Number of ports: 1 Firmware version: Hardware version: 2 Node GUID: 0x001175000077d708 System image GUID: 0x001175000077d708 Port 1: State:Active
Physical state: LinkUp Rate: 40 Base lid: 2 LMC: 0 SM lid: 1 Capability mask: 0x07690868 Port GUID:0x001175000077d708
Link layer: InfiniBand - Use the following command to scan without connecting to a remote SRP target. The target GUID shows that the initiator had found remote target. The ID string shows that the remote target is a Linux software target (
ib_srpt.ko
).[root@initiator]#
srp_daemon -a -o
IO Unit Info: port LID: 0001 port GID:fe80000000000000001175000077dd7e
change ID: 0001 max controllers: 0x10 controller[ 1] GUID:001175000077dd7e
vendor ID: 000011 device ID: 007322 IO class : 0100 ID:Linux SRP target
service entries: 1 service[ 0]: 001175000077dd7e / SRP.T10:001175000077dd7e - To verify the SRP connection, use the
lsscsi
command to list SCSI devices and compare thelsscsi
output before and after the initiator connects to target.[root@initiator]#
lsscsi
[0:0:10:0] disk IBM-ESXS ST9146803SS B53C /dev/sda - To connect to a remote target without configuring a valid ACL for the initiator port, which is expected to fail, use the following commands for
srp_daemon
oribsrpdm
:[root@initiator]#
srp_daemon -e -n -i qib0 -p 1 -R 60 &
[1] 4184[root@initiator]#
ibsrpdm -c -d /dev/infiniband/umad0 > /sys/class/infiniband_srp/srp-qib0-1/add_target
- The output of the
dmesg
shows why the SRP connection operation failed. In a later step, thedmesg
command on the target side is used to make the situation clear.[root@initiator]#
dmesg -c
[ 1230.059652] scsi host5: ib_srp: REJ received [ 1230.059659] scsi host5: ib_srp: SRP LOGIN fromfe80:0000:0000:0000:0011:7500:0077:d708
to fe80:0000:0000:0000:0011:7500:0077:dd7e REJECTED, reason0x00010006
[ 1230.073792] scsi host5: ib_srp: Connection 0/2 failed [ 1230.078848] scsi host5: ib_srp: Sending CM DREQ failed - Because of failed LOGIN, the output of the
lsscsi
command is the same as in the earlier step.[root@initiator]#
lsscsi
[0:0:10:0] disk IBM-ESXS ST9146803SS B53C /dev/sda - Using the
dmesg
on the target side (ib_srpt.ko
) provides an explanation of why LOGIN failed. Also, the output contains the valid ACL ID provided bysrp_daemon
:0x7edd770000751100001175000077d708
.[root@target]#
dmesg
[ 1200.303001] ib_srpt Received SRP_LOGIN_REQ with i_port_id 0x7edd770000751100:0x1175000077d708, t_port_id 0x1175000077dd7e:0x1175000077dd7e and it_iu_len 260 on port 1 (guid=0xfe80000000000000:0x1175000077dd7e)[ 1200.322207] ib_srpt Rejected login because no ACL has been configured yet for initiator 0x7edd770000751100001175000077d708.
- Use the
targetcli
tool to add a valid ACL:[root@target]#
targetcli
targetcli shell version 2.1.fb41 Copyright 2011-2013 by Datera, Inc and others. For help on commands, type 'help'. /> /srpt/ib.fe80000000000000001175000077dd7e/acls create0x7edd770000751100001175000077d708
Created Node ACL for ib.7edd770000751100001175000077d708 Created mapped LUN 0. - Verify the SRP LOGIN operation:
- Wait for 60 seconds to allow
srp_daemon
to re-try logging in:[root@initiator]#
sleep 60
- Verify the SRP LOGIN operation:
[root@initiator]#
lsscsi
[0:0:10:0] disk IBM-ESXS ST9146803SS B53C /dev/sda[7:0:0:0] disk LIO-ORG vol1 4.0 /dev/sdb
- For a kernel log of SRP target discovery, use:
[root@initiator]#
dmesg -c
[ 1354.182072] scsi host7: SRP.T10:001175000077DD7E [ 1354.187258] scsi 7:0:0:0: Direct-Access LIO-ORG vol1 4.0 PQ: 0 ANSI: 5 [ 1354.208688] scsi 7:0:0:0: alua: supports implicit and explicit TPGS [ 1354.215698] scsi 7:0:0:0: alua: port group 00 rel port 01 [ 1354.221409] scsi 7:0:0:0: alua: port group 00 state A non-preferred supports TOlUSNA [ 1354.229147] scsi 7:0:0:0: alua: Attached [ 1354.233402] sd 7:0:0:0: Attached scsi generic sg1 type 0 [ 1354.233694] sd 7:0:0:0: [sdb] 163258368 512-byte logical blocks: (83.5 GB/77.8 GiB) [ 1354.235127] sd 7:0:0:0: [sdb] Write Protect is off [ 1354.235128] sd 7:0:0:0: [sdb] Mode Sense: 43 00 00 08 [ 1354.235550] sd 7:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 1354.255491] sd 7:0:0:0: [sdb] Attached SCSI disk [ 1354.265233] scsi host7: ib_srp: new target: id_ext 001175000077dd7e ioc_guid 001175000077dd7e pkey ffff service_id 001175000077dd7e sgid fe80:0000:0000:0000:0011:7500:0077:d708 dgid fe80:0000:0000:0000:0011:7500:0077:dd7e xyx
13.6. Configuring the Subnet Manager
13.6.1. Determining Necessity
13.6.2. Configuring the opensm main configuration file
/etc/rdma/opensm.conf
. Users may edit this file at any time and edits will be kept on upgrade. There is extensive documentation of the options in the file itself. However, for the two most common edits needed, setting the GUID to bind to and the PRIORITY to run with, it is highly recommended that the opensm.conf
file is not edited but instead edit /etc/sysconfig/opensm
. If there are no edits to the base /etc/rdma/opensm.conf
file, it will get upgraded whenever the opensm package is upgraded. As new options are added to this file regularly, this makes it easier to keep the current configuration up to date. If the opensm.conf
file has been changed, then on upgrade, it might be necessary to merge new options into the edited file.
13.6.3. Configuring the opensm startup options
/etc/sysconfig/opensm
file control how the subnet manager is actually started, as well as how many copies of the subnet manager are started. For example, a dual port InfiniBand card, with each port plugged into physically separate networks, will need a copy of the subnet manager running on each port. The opensm subnet manager will only manage one subnet per instance of the application and must be started once for each subnet that needs to be managed. In addition, if there is more than one opensm server, then set the priorities on each server to control which are to be ports and which are to be controller.
/etc/sysconfig/opensm
is used to provide a simple means to set the priority of the subnet manager and to control which GUID the subnet manager binds to. There is an extensive explanation of the options in the /etc/sysconfig/opensm
file itself. Users need only read and follow the directions in the file itself to enable failover and multifabric operation of opensm.
13.6.4. Creating a P_Key definition
opensm.conf
looks for the file /etc/rdma/partitions.conf
to get a list of partitions to create on the fabric. All fabrics must contain the 0x7fff
subnet, and all switches and all hosts must belong to that fabric. Any other partition can be created in addition to that, and all hosts and all switches do not have to be members of these additional partitions. This allows an administrator to create subnets akin to Ethernet’s VLANs on InfiniBand fabrics. If a partition is defined with a given speed, such as 40 Gbps, and there is a host on the network unable to do 40 Gbps, then that host will be unable to join the partition even if it has permission to do so as it will be unable to match the speed requirements, therefore it is recommended that the speed of a partition be set to the slowest speed of any host with permission to join the partition. If a faster partition for some subset of hosts is required, create a different partition with the higher speed.
0x7fff
partition at a reduced speed of 10 Gbps, and a partition of 0x0002
with a speed of 40 Gbps:
~]$ more /etc/rdma/partitions.conf
# For reference:
# IPv4 IANA reserved multicast addresses:
# http://www.iana.org/assignments/multicast-addresses/multicast-addresses.txt
# IPv6 IANA reserved multicast addresses:
# http://www.iana.org/assignments/ipv6-multicast-addresses/ipv6-multicast-addresses.xml
#
# mtu =
# 1 = 256
# 2 = 512
# 3 = 1024
# 4 = 2048
# 5 = 4096
#
# rate =
# 2 = 2.5 GBit/s
# 3 = 10 GBit/s
# 4 = 30 GBit/s
# 5 = 5 GBit/s
# 6 = 20 GBit/s
# 7 = 40 GBit/s
# 8 = 60 GBit/s
# 9 = 80 GBit/s
# 10 = 120 GBit/s
Default=0x7fff, rate=3, mtu=4, scope=2, defmember=full:
ALL, ALL_SWITCHES=full;
Default=0x7fff, ipoib, rate=3, mtu=4, scope=2:
mgid=ff12:401b::ffff:ffff # IPv4 Broadcast address
mgid=ff12:401b::1 # IPv4 All Hosts group
mgid=ff12:401b::2 # IPv4 All Routers group
mgid=ff12:401b::16 # IPv4 IGMP group
mgid=ff12:401b::fb # IPv4 mDNS group
mgid=ff12:401b::fc # IPv4 Multicast Link Local Name Resolution group
mgid=ff12:401b::101 # IPv4 NTP group
mgid=ff12:401b::202 # IPv4 Sun RPC
mgid=ff12:601b::1 # IPv6 All Hosts group
mgid=ff12:601b::2 # IPv6 All Routers group
mgid=ff12:601b::16 # IPv6 MLDv2-capable Routers group
mgid=ff12:601b::fb # IPv6 mDNS group
mgid=ff12:601b::101 # IPv6 NTP group
mgid=ff12:601b::202 # IPv6 Sun RPC group
mgid=ff12:601b::1:3 # IPv6 Multicast Link Local Name Resolution group
ALL=full, ALL_SWITCHES=full;
ib0_2=0x0002, rate=7, mtu=4, scope=2, defmember=full:
ALL, ALL_SWITCHES=full;
ib0_2=0x0002, ipoib, rate=7, mtu=4, scope=2:
mgid=ff12:401b::ffff:ffff # IPv4 Broadcast address
mgid=ff12:401b::1 # IPv4 All Hosts group
mgid=ff12:401b::2 # IPv4 All Routers group
mgid=ff12:401b::16 # IPv4 IGMP group
mgid=ff12:401b::fb # IPv4 mDNS group
mgid=ff12:401b::fc # IPv4 Multicast Link Local Name Resolution group
mgid=ff12:401b::101 # IPv4 NTP group
mgid=ff12:401b::202 # IPv4 Sun RPC
mgid=ff12:601b::1 # IPv6 All Hosts group
mgid=ff12:601b::2 # IPv6 All Routers group
mgid=ff12:601b::16 # IPv6 MLDv2-capable Routers group
mgid=ff12:601b::fb # IPv6 mDNS group
mgid=ff12:601b::101 # IPv6 NTP group
mgid=ff12:601b::202 # IPv6 Sun RPC group
mgid=ff12:601b::1:3 # IPv6 Multicast Link Local Name Resolution group
ALL=full, ALL_SWITCHES=full;
13.6.5. Enabling opensm
root
:
~]# systemctl enable opensm
13.7. Testing Early InfiniBand RDMA operation
Note
IP
based devices, users should proceed to the section on testing RDMA operations once IPoIB has been configured and the devices have IP
addresses.
rdma
service is enabled, and the opensm service (if needed) is enabled, and the proper user-space library for the specific hardware has been installed, user space rdma
operation should be possible. Simple test programs from the libibverbs-utils package are helpful in determining that RDMA operations are working properly. The ibv_devices program will show which devices are present in the system and the ibv_devinfo
command will give detailed information about each device. For example:
~]$ibv_devices
device node GUID ------ ---------------- mlx4_0 0002c903003178f0 mlx4_1 f4521403007bcba0 ~]$ibv_devinfo -d mlx4_1
hca_id: mlx4_1 transport: InfiniBand (0) fw_ver: 2.30.8000 node_guid: f452:1403:007b:cba0 sys_image_guid: f452:1403:007b:cba3 vendor_id: 0x02c9 vendor_part_id: 4099 hw_ver: 0x0 board_id: MT_1090120019 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 2048 (4) sm_lid: 2 port_lid: 2 port_lmc: 0x01 link_layer: InfiniBand port: 2 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet~]$ ibstat mlx4_1
CA 'mlx4_1' CA type: MT4099 Number of ports: 2 Firmware version: 2.30.8000 Hardware version: 0 Node GUID: 0xf4521403007bcba0 System image GUID: 0xf4521403007bcba3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 2 LMC: 1 SM lid: 2 Capability mask: 0x0251486a Port GUID: 0xf4521403007bcba1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 0x04010000 Port GUID: 0xf65214fffe7bcba2 Link layer: Ethernet
ibv_devinfo
and ibstat
commands output slightly different information (such as port MTU exists in ibv_devinfo
but not in ibstat
output, and the Port GUID exists in ibstat
output but not in ibv_devinfo
output), and a few things are named differently (for example, the Base local identifier (LID) in ibstat
output is the same as the port_lid
output of ibv_devinfo
)
IP
addresses for specifying the server.
ibv_devinfo
and ibstat
commands to print out the port_lid
(or Base lid) and the Port GUID of the port they want to test (assuming port 1 of the above interface, the port_lid
/Base LID
is 2
and Port GUID is 0xf4521403007bcba1
)). Then start ibping with the necessary options to bind specifically to the card and port to be tested, and also specifying ibping should run in server mode. You can see the available options to ibping by passing -?
or --help
, but in this instance we will need either the -S
or --Server
option and for binding to the specific card and port we will need either -C
or --Ca
and -P
or --Port
. Note: port in this instance does not denote a network port number, but denotes the physical port number on the card when using a multi-port card. To test connectivity to the RDMA fabric using, for example, the second port of a multi-port card, requires telling ibping to bind to port 2
on the card. When using a single port card, or testing the first port on a card, this option is not needed. For example:
~]$ ibping -S -C mlx4_1 -P 1
~]$ ibping -c 10000 -f -C mlx4_0 -P 1 -L 2
--- rdma-host.example.com.(none) (Lid 2) ibping statistics ---
10000 packets transmitted, 10000 received, 0% packet loss, time 816 ms
rtt min/avg/max = 0.032/0.081/0.446 ms
or
~]$ ibping -c 10000 -f -C mlx4_0 -P 1 -G 0xf4521403007bcba1 \
--- rdma-host.example.com.(none) (Lid 2) ibping statistics ---
10000 packets transmitted, 10000 received, 0% packet loss, time 769 ms
rtt min/avg/max = 0.027/0.076/0.278 ms
This outcome verifies that end to end RDMA communications are working for user space applications.
~]$ ibv_devinfo
libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0
No IB devices found
This error indicates that the necessary user-space library is not installed. The administrator will need to install one of the user-space libraries (as appropriate for their hardware) listed in section Section 13.4, “InfiniBand and RDMA related software packages”. On rare occasions, this can happen if a user installs the wrong arch type for the driver or for libibverbs. For example, if libibverbs is of arch x86_64
, and libmlx4 is installed but is of type i686
, then this error can result.
Note
13.8. Configuring IPoIB
13.8.1. Understanding the role of IPoIB
IP
networks. InfiniBand is not. The role of IPoIB is to provide an IP
network emulation layer on top of InfiniBand RDMA networks. This allows existing applications to run over InfiniBand networks unmodified. However, the performance of those applications is considerably lower than if the application were written to use RDMA communication natively. Since most InfiniBand networks have some set of applications that really must get all of the performance they can out of the network, and then some other applications for which a degraded rate of performance is acceptable if it means that the application does not need to be modified to use RDMA communications, IPoIB is there to allow those less critical applications to run on the network as they are.
IP
networks with RDMA layered on top of their IP
link layer, they have no need of IPoIB. As a result, the kernel will refuse to create any IPoIB devices on top of iWARP or RoCE/IBoE RDMA devices.
13.8.2. Understanding IPoIB communication modes
IP
packet being transmitted. As a result, the IPoIB MTU must be 4 bytes less than the InfiniBand link-layer MTU. As 2048 is a common InfiniBand link-layer MTU, the common IPoIB device MTU in datagram mode is 2044.
IP
packet only has a 16 bit size field, and is therefore limited to 65535
as the maximum byte count. The maximum allowed MTU is actually smaller than that because we have to account for various TCP/IP headers that must also fit in that size. As a result, the IPoIB MTU in connected mode is capped at 65520
in order to make sure there is sufficient room for all needed TCP
headers.
13.8.3. Understanding IPoIB hardware addresses
0xfe:80:00:00:00:00:00:00
. The device will use the default subnet prefix (0xfe80000000000000) until it makes contact with the subnet manager, at which point it will reset the subnet prefix to match what the subnet manager has configured it to be. The final 8 bytes are the GUID address of the InfiniBand port that the IPoIB device is attached to. Because both the first 4 bytes and the next 8 bytes can change from time to time, they are not used or matched against when specifying the hardware address for an IPoIB interface. Section Section 13.5.2, “Usage of 70-persistent-ipoib.rules” explains how to derive the address by leaving the first 12 bytes out of the ATTR{address}
field in the udev rules file so that device matching will happen reliably. When configuring IPoIB interfaces, the HWADDR field of the configuration file can contain all 20 bytes, but only the last 8 bytes are actually used to match against and find the hardware specified by a configuration file. However, if the TYPE=InfiniBand
entry is not spelled correctly in the device configuration file, and ifup-ib is not the actual script used to open the IPoIB interface, then an error about the system being unable to find the hardware specified by the configuration will be issued. For IPoIB interfaces, the TYPE=
field of the configuration file must be either InfiniBand
or infiniband
(the entry is case sensitive, but the scripts will accept these two specific spellings).
13.8.4. Understanding InfiniBand P_Key subnets
P_Key
subnets. This is highly analogous to using VLANs on Ethernet interfaces. All switches and hosts must be a member of the default P_Key
subnet, but administrators can create additional subnets and limit members of those subnets to subsets of the hosts or switches in the fabric. A P_Key
subnet must be defined by the subnet manager before a host can use it. See section Section 13.6.4, “Creating a P_Key definition” for information on how to define a P_Key
subnet using the opensm subnet manager. For IPoIB interfaces, once a P_Key
subnet has been created, we can create additional IPoIB configuration files specifically for those P_Key
subnets. Just like VLAN interfaces on Ethernet devices, each IPoIB interface will behave as though it were on a completely different fabric from other IPoIB interfaces that share the same link but have different P_Key
values.
P_Key
interfaces. All IPoIB P_Key
s range from 0x0000
to 0x7fff
, and the high bit, 0x8000
, denotes that membership in a P_Key
is full membership instead of partial membership. The Linux kernel’s IPoIB driver only supports full membership in P_Key
subnets, so for any subnet that Linux can connect to, the high bit of the P_Key
number will always be set. That means that if a Linux computer joins P_Key 0x0002
, its actual P_Key
number once joined will be 0x8002
, denoting that we are full members of P_Key 0x0002
. For this reason, when creating a P_Key
definition in an opensm partitions.conf
file as depicted in section Section 13.6.4, “Creating a P_Key definition”, it is required to specify a P_Key
value without 0x8000
, but when defining the P_Key
IPoIB interfaces on the Linux clients, add the 0x8000
value to the base P_Key
value.
13.8.5. Configure InfiniBand Using the Text User Interface, nmtui
~]$ nmtui
The text user interface appears. Any invalid command prints a usage message.
Figure 13.1. The NetworkManager Text User Interface Add an InfiniBand Connection menu
Figure 13.2. The NetworkManager Text User Interface Configuring a InfiniBand Connection menu
13.8.6. Configure IPoIB using the command-line tool, nmcli
ib_ipoib
kernel module and then reloading it as follows:
~]$rmmod ib_ipoib
~]$modprobe ib_ipoib
Example 13.3. Creating and modifying IPoIB in two separate commands.
~]$nmcli con add type infiniband con-name mlx4_ib0 ifname mlx4_ib0 transport-mode connected mtu 65520
Connection 'mlx4_ib0' (8029a0d7-8b05-49ff-a826-2a6d722025cc) successfully added. ~]$nmcli con edit mlx4_ib0
===| nmcli interactive connection editor |=== Editing existing 'infiniband' connection: 'mlx4_ib0' Type 'help' or '?' for available commands. Type 'describe [>setting<.>prop<]' for detailed property description. You may edit the following settings: connection, infiniband, ipv4, ipv6 nmcli> set infiniband.mac-address 80:00:02:00:fe:80:00:00:00:00:00:00:f4:52:14:03:00:7b:cb:a3 nmcli> save Connection 'mlx4_ib3' (8029a0d7-8b05-49ff-a826-2a6d722025cc) successfully updated. nmcli> quit
nmcli c add
and nmcli c modify
in one command, as follows:
Example 13.4. Creating and modifying IPoIB in one command.
nmcli con add type infiniband con-name mlx4_ib0 ifname mlx4_ib0 transport-mode connected mtu 65520
infiniband.mac-address 80:00:02:00:fe:80:00:00:00:00:00:00:f4:52:14:03:00:7b:cb:a3
mlx4_ib0
has been created and set to use connected mode, with the maximum connected mode MTU, DHCP
for IPv4
and IPv6
. If using IPoIB interfaces for cluster traffic and an Ethernet interface for out-of-cluster communications, it is likely that disabling default routes and any default name server on the IPoIB interfaces will be required. This can be done as follows:
~]$ nmcli con edit mlx4_ib0
===| nmcli interactive connection editor |===
Editing existing 'infiniband' connection: 'mlx4_ib0'
Type 'help' or '?' for available commands.
Type 'describe [>setting<.>prop<]' for detailed property description.
You may edit the following settings: connection, infiniband, ipv4, ipv6
nmcli> set ipv4.ignore-auto-dns yes
nmcli> set ipv4.ignore-auto-routes yes
nmcli> set ipv4.never-default true
nmcli> set ipv6.ignore-auto-dns yes
nmcli> set ipv6.ignore-auto-routes yes
nmcli> set ipv6.never-default true
nmcli> save
Connection 'mlx4_ib0' (8029a0d7-8b05-49ff-a826-2a6d722025cc) successfully updated.
nmcli> quit
P_Key
interface is required, create one using nmcli as follows:
~]$nmcli con add type infiniband con-name mlx4_ib0.8002 ifname mlx4_ib0.8002 parent mlx4_ib0 p-key 0x8002
Connection 'mlx4_ib0.8002' (4a9f5509-7bd9-4e89-87e9-77751a1c54b4) successfully added. ~]$nmcli con modify mlx4_ib0.8002 infiniband.mtu 65520 infiniband.transport-mode connected ipv4.ignore-auto-dns yes ipv4.ignore-auto-routes yes ipv4.never-default true ipv6.ignore-auto-dns yes ipv6.ignore-auto-routes yes ipv6.never-default true
13.8.7. Configure IPoIB Using the command line
ib_ipoib
kernel module and then reloading it as follows:
~]$rmmod ib_ipoib
~]$modprobe ib_ipoib
ifcfg
files with their preferred editor to control the devices. A typical IPoIB configuration file with static IPv4
addressing looks as follows:
~]$ more ifcfg-mlx4_ib0
DEVICE=mlx4_ib0
TYPE=InfiniBand
ONBOOT=yes
HWADDR=80:00:00:4c:fe:80:00:00:00:00:00:00:f4:52:14:03:00:7b:cb:a1
BOOTPROTO=none
IPADDR=172.31.0.254
PREFIX=24
NETWORK=172.31.0.0
BROADCAST=172.31.0.255
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
MTU=65520
CONNECTED_MODE=yes
NAME=mlx4_ib0
The DEVICE field must match the custom name created in any udev renaming rules. The NAME entry need not match the device name. If the GUI connection editor is started, the NAME field is what is used to present a name for this connection to the user. The TYPE field must be InfiniBand in order for InfiniBand options to be processed properly. CONNECTED_MODE is either yes
or no
, where yes
will use connected mode and no
will use datagram mode for communications (see section Section 13.8.2, “Understanding IPoIB communication modes”).
P_Key
interfaces, this is a typical configuration file:
~]$ more ifcfg-mlx4_ib0.8002
DEVICE=mlx4_ib0.8002
PHYSDEV=mlx4_ib0
PKEY=yes
PKEY_ID=2
TYPE=InfiniBand
ONBOOT=yes
HWADDR=80:00:00:4c:fe:80:00:00:00:00:00:00:f4:52:14:03:00:7b:cb:a1
BOOTPROTO=none
IPADDR=172.31.2.254
PREFIX=24
NETWORK=172.31.2.0
BROADCAST=172.31.2.255
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
MTU=65520
CONNECTED_MODE=yes
NAME=mlx4_ib0.8002
For