25.6.2.3. Improving UDP Performance by Configuring OS UDP Buffer Limits
By default, the JGroups channels in JBoss Enterprise Application Platform use the UDP transport protocol to take advantage of IP multicast. However, one disadvantage of UDP is it does not come with the reliable delivery guarantees provided by TCP. The protocols discussed in Section 25.1.5, “Reliable Delivery Protocols” allow JGroups to guarantee delivery of UDP messages, but those protocols are implemented in Java, not at the operating system network layer. For peak performance from a UDP-based JGroups channel it is important to limit the need for JGroups to retransmit messages by limiting UDP datagram loss.
One of the most common causes of lost UDP datagrams is an undersized receive buffer on the socket. The UDP protocol's
mcast_recv_buf_size
and ucast_recv_buf_size
configuration attributes are used to specify the amount of receive buffer JGroups requests from the operating system, but the actual size of the buffer the operating system provides is limited by operating system-level maximums. These maximums are often very low:
Operating System | Default Max UDP Buffer (in bytes) |
---|---|
Linux | 131071 |
Windows | No known limit |
Solaris | 262144 |
FreeBSD, Darwin | 262144 |
AIX | 1048576 |
The command used to increase the above limits is operating system-specific. The table below shows the command required to increase the maximum buffer to 25 megabytes. In all cases, root privileges are required:
Operating System | Command |
---|---|
Linux | sysctl -w net.core.rmem_max=26214400 |
Solaris | ndd -set /dev/udp udp_max_buf 26214400 |
FreeBSD, Darwin | sysctl -w kern.ipc.maxsockbuf=26214400 |
AIX | no -o sb_max=8388608 (AIX will only allow 1 megabyte, 4 megabytes or 8 megabytes). |