Chapter 8. Tuning applications with a large number of incoming requests
If you run an application that handles a large number of incoming requests, such as web servers, it can be necessary to tune Red Hat Enterprise Linux to optimize the performance.
8.1. Tuning the TCP listen backlog to process a high number of TCP connection attempts
When an application opens a TCP socket in LISTEN
state, the kernel limits the number of accepted client connections this socket can handle. If clients try to establish more connections than the application can process, the new connections get lost or the kernel sends SYN cookies to the client.
If the system is under normal workload and too many connections from legitimate clients cause the kernel to send SYN cookies, tune Red Hat Enterprise Linux (RHEL) to avoid them.
Prerequisites
-
RHEL logs
possible SYN flooding on port <ip_address>:<port_number>
error messages in the Systemd journal. - The high number of connection attempts are from valid sources and not caused by an attack.
Procedure
To verify whether tuning is required, display the statistics for the affected port:
ss -ntl '( sport = :443 )'
# ss -ntl '( sport = :443 )' State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 650 500 192.0.2.1:443 0.0.0.0:*
Copy to Clipboard Copied! If the current number of connections in the backlog (
Recv-Q
) is larger than the socket backlog (Send-Q
), the listen backlog is still not large enough and tuning is required.Optional: Display the current TCP listen backlog limit:
sysctl net.core.somaxconn
# sysctl net.core.somaxconn net.core.somaxconn = 4096
Copy to Clipboard Copied! Create the
/etc/sysctl.d/10-socket-backlog-limit.conf
file, and set a larger listen backlog limit:net.core.somaxconn = 8192
net.core.somaxconn = 8192
Copy to Clipboard Copied! Note that applications can request a larger listen backlog than specified in the
net.core.somaxconn
kernel parameter but the kernel limits the application to the number you set in this parameter.Load the setting from the
/etc/sysctl.d/10-socket-backlog-limit.conf
file:sysctl -p /etc/sysctl.d/10-socket-backlog-limit.conf
# sysctl -p /etc/sysctl.d/10-socket-backlog-limit.conf
Copy to Clipboard Copied! Reconfigure the application to use the new listen backlog limit:
-
If the application provides a config option for the limit, update it. For example, the Apache HTTP Server provides the
ListenBacklog
configuration option to set the listen backlog limit for this service. - If you cannot configure the limit, recompile the application.
ImportantYou must always update both the
net.core.somaxconn
kernel setting and the application’s settings.-
If the application provides a config option for the limit, update it. For example, the Apache HTTP Server provides the
- Restart the application.
Verification
-
Monitor the Systemd journal for further occurrences of
possible SYN flooding on port <port_number>
error messages. Monitor the current number of connections in the backlog and compare it with the socket backlog:
ss -ntl '( sport = :443 )'
# ss -ntl '( sport = :443 )' State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 500 192.0.2.1:443 0.0.0.0:*
Copy to Clipboard Copied! If the current number of connections in the backlog (
Recv-Q
) is larger than the socket backlog (Send-Q
), the listen backlog is not large enough and further tuning is required.