26.2. Configure JGroups (Library Mode)
Red Hat JBoss Data Grid must have an appropriate JGroups configuration in order to operate in clustered mode.
Example 26.5. JGroups Programmatic Configuration
GlobalConfiguration gc = new GlobalConfigurationBuilder()
.transport()
.defaultTransport()
.addProperty("configurationFile","jgroups.xml")
.build();Example 26.6. JGroups XML Configuration
<infinispan>
<global>
<transport>
<properties>
<property name="configurationFile" value="jgroups.xml" />
</properties>
</transport>
</global>
<!-- Additional configuration elements here -->
</infinispan>
In either programmatic or XML configuration methods, JBoss Data Grid searches for
jgroups.xml in the classpath before searching for an absolute path name if it is not found in the classpath.
26.2.1. JGroups Transport Protocols
A transport protocol is the protocol at the bottom of a protocol stack. Transport Protocols are responsible for sending and receiving messages from the network.
Red Hat JBoss Data Grid ships with both UDP and TCP transport protocols.
26.2.1.1. The UDP Transport Protocol
UDP is a transport protocol that uses:
- IP multicasting to send messages to all members of a cluster.
- UDP datagrams for unicast messages, which are sent to a single member.
When the UDP transport is started, it opens a unicast socket and a multicast socket. The unicast socket is used to send and receive unicast messages, the multicast socket sends and receives multicast sockets. The physical address of the channel will be the same as the address and port number of the unicast socket.
26.2.1.2. The TCP Transport Protocol
TCP/IP is a replacement transport for UDP in situations where IP multicast cannot be used, such as operations over a WAN where routers may discard IP multicast packets.
TCP is a transport protocol used to send unicast and multicast messages.
- When sending multicast messages, TCP sends multiple unicast messages.
- When using TCP, each message to all cluster members is sent as multiple unicast messages, or one to each member.
As IP multicasting cannot be used to discover initial members, another mechanism must be used to find initial membership.
Red Hat JBoss Data Grid's Hot Rod is a custom TCP client/server protocol.
26.2.1.3. Using the TCPPing Protocol
Some networks only allow TCP to be used. The pre-configured
default-configs/default-jgroups-tcp.xml includes the MPING protocol, which uses UDP multicast for discovery. When UDP multicast is not available, the MPING protocol, has to be replaced by a different mechanism. The recommended alternative is the TCPPING protocol. The TCPPING configuration contains a static list of IP addresses which are contacted for node discovery.
Example 26.7. Configure the JGroups Subsystem to Use TCPPING
<TCP bind_port="7800" />
<TCPPING initial_hosts="${jgroups.tcpping.initial_hosts:HostA[7800],HostB[7801]}"
port_range="1" />26.2.2. Pre-Configured JGroups Files
Red Hat JBoss Data Grid ships with a number of pre-configured JGroups files packaged in
infinispan-embedded.jar, and are available on the classpath by default. In order to use one of these files, specify one of these file names instead of using jgroups.xml.
The JGroups configuration files shipped with JBoss Data Grid are intended to be used as a starting point for a working project. JGroups will usually require fine-tuning for optimal network performance.
The available configurations are:
default-configs/default-jgroups-udp.xmldefault-configs/default-jgroups-tcp.xmldefault-configs/default-jgroups-ec2.xml
26.2.2.1. default-jgroups-udp.xml
The
default-configs/default-jgroups-udp.xml file is a pre-configured JGroups configuration in Red Hat JBoss Data Grid. The default-jgroups-udp.xml configuration
- uses UDP as a transport and UDP multicast for discovery.
- is suitable for large clusters (over 8 nodes).
- is suitable if using Invalidation or Replication modes.
The behavior of some of these settings can be altered by adding certain system properties to the JVM at startup. The settings that can be configured are shown in the following table.
| System Property | Description | Default | Required? |
|---|---|---|---|
| jgroups.udp.mcast_addr | IP address to use for multicast (both for communications and discovery). Must be a valid Class D IP address, suitable for IP multicast. | 228.6.7.8 | No |
| jgroups.udp.mcast_port | Port to use for multicast socket | 46655 | No |
| jgroups.udp.ip_ttl | Specifies the time-to-live (TTL) for IP multicast packets. The value here refers to the number of network hops a packet is allowed to make before it is dropped | 2 | No |
26.2.2.2. default-jgroups-tcp.xml
The
default-configs/default-jgroups-tcp.xml file is a pre-configured JGroups configuration in Red Hat JBoss Data Grid. The default-jgroups-tcp.xml configuration
- uses TCP as a transport and UDP multicast for discovery.
- is generally only used where multicast UDP is not an option.
- TCP does not perform as well as UDP for clusters of eight or more nodes. Clusters of four nodes or fewer result in roughly the same level of performance for both UDP and TCP.
As with other pre-configured JGroups files, the behavior of some of these settings can be altered by adding certain system properties to the JVM at startup. The settings that can be configured are shown in the following table.
| System Property | Description | Default | Required? |
|---|---|---|---|
| jgroups.tcp.address | IP address to use for the TCP transport. | 127.0.0.1 | No |
| jgroups.tcp.port | Port to use for TCP socket | 7800 | No |
| jgroups.udp.mcast_addr | IP address to use for multicast (for discovery). Must be a valid Class D IP address, suitable for IP multicast. | 228.6.7.8 | No |
| jgroups.udp.mcast_port | Port to use for multicast socket | 46655 | No |
| jgroups.udp.ip_ttl | Specifies the time-to-live (TTL) for IP multicast packets. The value here refers to the number of network hops a packet is allowed to make before it is dropped | 2 | No |
23149%2C+Administration+and+Configuration+Guide-6.628-06-2017+13%3A51%3A02JBoss+Data+Grid+6Documentation6.6.1Report a bug
26.2.2.3. default-jgroups-ec2.xml
The
default-configs/default-jgroups-ec2.xml file is a pre-configured JGroups configuration in Red Hat JBoss Data Grid. The default-jgroups-ec2.xml configuration
- uses TCP as a transport and S3_PING for discovery.
- is suitable on Amazon EC2 nodes where UDP multicast isn't available.
As with other pre-configured JGroups files, the behavior of some of these settings can be altered by adding certain system properties to the JVM at startup. The settings that can be configured are shown in the following table.
| System Property | Description | Default | Required? |
|---|---|---|---|
| jgroups.tcp.address | IP address to use for the TCP transport. | 127.0.0.1 | No |
| jgroups.tcp.port | Port to use for TCP socket | 7800 | No |
| jgroups.s3.access_key | The Amazon S3 access key used to access an S3 bucket | Yes | |
| jgroups.s3.secret_access_key | The Amazon S3 secret key used to access an S3 bucket | Yes | |
| jgroups.s3.bucket | Name of the Amazon S3 bucket to use. Must be unique and must already exist | Yes | |
| jgroups.s3.pre_signed_delete_url | The pre-signed URL to be used for the DELETE operation. | Yes | |
| jgroups.s3.pre_signed_put_url | The pre-signed URL to be used for the PUT operation. | Yes | |
| jgroups.s3.prefix | If set, S3_PING searches for a bucket with a name that starts with the prefix value. | No |
