Chapter 24. Getting started with Multipath TCP
Transmission Control Protocol (TCP) ensures reliable delivery of the data through the internet and automatically adjusts its bandwidth in response to network load. Multipath TCP (MPTCP) is an extension to the original TCP protocol (single-path). MPTCP enables a transport connection to operate across multiple paths simultaneously, and brings network connection redundancy to user endpoint devices.
24.1. Understanding MPTCP
The Multipath TCP (MPTCP) protocol allows for simultaneous usage of multiple paths between connection endpoints. The protocol design improves connection stability and also brings other benefits compared to the single-path TCP.
In MPTCP terminology, links are considered as paths.
The following are some of the advantages of using MPTCP:
- It allows a connection to simultaneously use multiple network interfaces.
- In case a connection is bound to a link speed, the usage of multiple links can increase the connection throughput. Note, that in case of the connection is bound to a CPU, the usage of multiple links causes the connection slowdown.
- It increases the resilience to link failures.
For more details about MPTCP, review the Additional resources.
24.2. Permanently configuring multiple paths for MPTCP applications
You can configure MultiPath TCP (MPTCP) by using the nmcli
command to permanently establish multiple subflows between a source and destination system. The subflows can use different resources, different routes to the destination, and even different networks. Such as Ethernet, cellular, wifi, and so on. As a result, you achieve combined connections, which increase network resilience and throughput.
The server uses the following network interfaces in our example:
-
enp4s0:
192.0.2.1/24
-
enp1s0:
198.51.100.1/24
-
enp7s0:
192.0.2.3/24
The client uses the following network interfaces in our example:
-
enp4s0f0:
192.0.2.2/24
-
enp4s0f1:
198.51.100.2/24
-
enp6s0:
192.0.2.5/24
Prerequisites
- You configured the default gateway on the relevant interfaces.
Procedure
Enable MPTCP sockets in the kernel:
# echo "net.mptcp.enabled=1" > /etc/sysctl.d/90-enable-MPTCP.conf # sysctl -p /etc/sysctl.d/90-enable-MPTCP.conf
Optional: The RHEL kernel default for subflow limit is 2. If you require more:
Create the
/etc/systemd/system/set_mptcp_limit.service
file with the following content:[Unit] Description=Set MPTCP subflow limit to 3 After=network.target [Service] ExecStart=ip mptcp limits set subflows 3 Type=oneshot [Install] WantedBy=multi-user.target
The oneshot unit executes the
ip mptcp limits set subflows 3
command after your network (network.target
) is operational during every boot process.The
ip mptcp limits set subflows 3
command sets the maximum number of additional subflows for each connection, so 4 in total. It is possible to add maximally 3 additional subflows.Enable the
set_mptcp_limit
service:# systemctl enable --now set_mptcp_limit
Enable MPTCP on all connection profiles that you want to use for connection aggregation:
# nmcli connection modify <profile_name> connection.mptcp-flags signal,subflow,also-without-default-route
The
connection.mptcp-flags
parameter configures MPTCP endpoints and the IP address flags. If MPTCP is enabled in a NetworkManager connection profile, the setting will configure the IP addresses of the relevant network interface as MPTCP endpoints.By default, NetworkManager does not add MPTCP flags to IP addresses if there is no default gateway. If you want to bypass that check, you need to use the
also-without-default-route
flag.
Verification
Verify that you enabled the MPTCP kernel parameter:
# sysctl net.mptcp.enabled net.mptcp.enabled = 1
Verify that you set the subflow limit correctly, in case the default was not enough:
# ip mptcp limit show add_addr_accepted 2 subflows 3
Verify that you configured the per-address MPTCP setting correctly:
# ip mptcp endpoint show 192.0.2.1 id 1 subflow dev enp4s0 198.51.100.1 id 2 subflow dev enp1s0 192.0.2.3 id 3 subflow dev enp7s0 192.0.2.4 id 4 subflow dev enp3s0 ...
Additional resources
24.3. Configuring mptcpd
The mptcpd
service is a component of the mptcp
protocol which provides an instrument to configure mptcp
endpoints. The mptcpd
service creates a subflow endpoint for each address by default. The endpoint list is updated dynamically according to IP addresses modification on the running host. The mptcpd
service creates the list of endpoints automatically. It enables multiple paths as an alternative to using the ip
utility.
Prerequisites
-
The
mptcpd
package installed
Procedure
Enable
mptcp.enabled
option in the kernel with the following command:# echo "net.mptcp.enabled=1" > /etc/sysctl.d/90-enable-MPTCP.conf # sysctl -p /etc/sysctl.d/90-enable-MPTCP.conf
Enable and start the
mptcpd
service:# systemctl enable --now mptcp.service
To configure
mptcpd
service manually, modify the/etc/mptcpd/mptcpd.conf
configuration file.Note, that the endpoint, which mptcpd service creates, lasts till the host shutdown.
Verification
Verify endpoint creation:
# ip mptcp endpoint
Additional resources
-
mptcpd(8)
man page on your system.