2.9. Network Determinism Tips
Transmission Control Protocol (TCP)
TCP can have a large effect on latency. TCP adds latency in order to obtain efficiency, control congestion, and to ensure reliable delivery. When tuning, consider the following points:
- Do you need ordered delivery?
- Do you need to guard against packet loss?Transmitting packets more than once can cause delays.
- If you must use TCP, consider disabling the Nagle buffering algorithm by using
TCP_NODELAY
on your socket. The Nagle algorithm collects small outgoing packets to send all at once, and can have a detrimental effect on latency.
Network Tuning
There are numerous tools for tuning the network. Here are some of the more useful:
- Interrupt Coalescing
- To reduce the amount of interrupts, packets can be collected and a single interrupt generated for a collection of packets.In systems that transfer large amounts of data where throughput is a priority, using the default value or increasing coalesce can increase throughput and lower the number of interrupts hitting CPUs. For systems requiring a rapid network response, reducing or disabling coalesce is advised.Use the
-C
(--coalesce
) option with theethtool
command to enable. - Congestion
- Often, I/O switches can be subject to back-pressure, where network data builds up as a result of full buffers.Use the
-A
(--pause
) option with theethtool
command to change pause parameters and avoid network congestion. - Infiniband (IB)
- Infiniband is a type of communications architecture often used to increase bandwidth and provide quality of service and failover. It can also be used to improve latency through Remote Direct Memory Access (RDMA) capabilities.
- Network Protocol Statistics
- Use the
-s
(--statistics
) option with thenetstat
command to monitor network traffic.
See also Section 2.12, “Reduce TCP Performance Spikes” and Section 3.7, “Reducing the TCP Delayed ACK Timeout”.
Related Manual Pages
For more information, or for further reading, the following man pages are related to the information given in this section.
- ethtool(8)
- netstat(8)