Search
SupportConsoleDevelopersStart a trialContact

Chapter 2. Configuring Data Grid for Cross-Site Replication

download PDF

Configuring Data Grid to replicate data across sites, you first set up cluster transport so Data Grid clusters can discover each other and site masters can communicate. You then add backup locations to cache definitions in your Data Grid configuration.

2.1. Configuring Cluster Transport for Cross-Site Replication

Add JGroups RELAY2 to your transport layer so that Data Grid clusters can communicate with backup locations.

Procedure

  1. Open infinispan.xml for editing.
  2. Add the RELAY2 protocol to a JGroups stack.

  3. Configure Data Grid cluster transport to use the stack. 

    <infinispan>
  <jgroups>
    <!-- Extends the default UDP stack. -->
    <stack name="xsite" extends="udp">
      <!-- Adds RELAY2 for cross-site replication. -->
      <!-- Names the local site as LON. -->
      <!-- Specifies 1000 nodes as the maximum number of site masters. -->
      <relay.RELAY2 site="LON" xmlns="urn:org:jgroups" max_site_masters="1000"/>
      <!-- Uses the default TCP stack for inter-cluster communication. -->
      <!-- Names all sites that act as backup locations. -->
      <remote-sites default-stack="tcp">
         <remote-site name="LON"/>
         <remote-site name="NYC"/>
      </remote-sites>
    </stack>
  </jgroups>
  <cache-container name="default" statistics="true">
    <!-- Use the "xsite" stack for cluster transport. -->
    <transport cluster="${cluster.name}" stack="xsite"/>
  </cache-container>
</infinispan>
  4. Save and close infinispan.xml.

Reference

2.1.1. JGroups RELAY2 Stacks

Data Grid clusters use JGroups RELAY2 for inter-cluster discovery and communication. 

<jgroups>
   <!-- Uses the default JGroups UDP stack for intra-cluster traffic. -->
   <stack name="xsite" extends="udp">
      <!-- Adds RELAY2 to the stack for inter-cluster transport. -->
      <!-- Names the local site. Data in caches from the local site is replicated to backup locations. -->
      <!-- Configures a maximum of 1000 site masters for the local cluster. -->
      <relay.RELAY2 xmlns="urn:org:jgroups"
                    site="LON"
                    max_site_masters="1000"/>
      <!-- Specifies all site names and uses the default JGroups TCP stack for inter-cluster transport. -->
      <remote-sites default-stack="tcp">
         <!-- Names all sites that participate in cross-site replication. -->
         <remote-site name="LON"/>
         <remote-site name="NYC"/>
      </remote-sites>
   </stack>
</jgroups>
Tip

Set max_site_masters >= the number of nodes in the Data Grid cluster for optimal performance with backup requests.

2.1.2. Custom JGroups RELAY2 Stacks

The following configuration adds a custom RELAY2 stack that extends the default TCP stack: 

<jgroups>
  <!-- Uses TCPPING instead of MPING for discovery. -->
   <stack name="relay-global" extends="tcp">
     <MPING stack.combine="REMOVE"/>
     <TCPPING initial_hosts="192.0.2.0[7800]"
              stack.combine="INSERT_AFTER"
              stack.position="TCP"/>
   </stack>
   <!-- Extends the default UDP stack with RELAY2. -->
   <!-- Specifies RELAY2 properties. -->
   <stack name="xsite" extends="udp">
      <relay.RELAY2 site="LON" xmlns="urn:org:jgroups"
                    max_site_masters="10"
                    can_become_site_master="true"/>
      <remote-sites default-stack="relay-global">
         <remote-site name="LON"/>
         <remote-site name="NYC"/>
      </remote-sites>
   </stack>
</jgroups>

You can also reference externally defined JGroups stack files, for example: 

<infinispan>
  <jgroups>
     <stack-file name="relay-global" path="jgroups-relay.xml"/>
  </jgroups>
  <cache-container default-cache="replicatedCache">
    <transport stack="relay-global" />
    <replicated-cache name="replicatedCache"/>
  </cache-container>
</infinispan>

Where jgroups-relay.xml references a JGroups stack file such as: 

<config xmlns="urn:org:jgroups"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups-4.1.xsd">

    <!-- Use TCP for inter-cluster transport. -->
    <TCP bind_addr="127.0.0.1"
         bind_port="7200"
         port_range="30"

         thread_pool.min_threads="0"
         thread_pool.max_threads="8"
         thread_pool.keep_alive_time="5000"
    />

    <!-- Use TCPPING for inter-cluster discovery. -->
    <TCPPING timeout="3000"
             initial_hosts="127.0.0.1[7200]"
             port_range="3"
             ergonomics="false"/>

    <!-- Provide other JGroups stack configuration as required. -->
</config>

Reference

2.2. Adding Backup Locations to Caches

Specify the names of remote sites so Data Grid can back up data to those locations.

Procedure

  1. Add the backups element to your cache definition.

  2. Specify the name of each remote site with the backup element.

    As an example, in the LON configuration, specify NYC as the remote site.

  3. Repeat the preceding steps so that each site is a backup for all other sites. For example, you cannot add LON as a backup for NYC without adding NYC as a backup for LON.
Note

Cache configurations can be different across sites and use different backup strategies. Data Grid replicates data based on cache names.

Example "customers" configuration in LON

<replicated-cache name="customers">
  <backups>
    <backup site="NYC" strategy="ASYNC" />
  </backups>
</replicated-cache>

Example "customers" configuration in NYC

<distributed-cache name="customers">
  <backups>
    <backup site="LON" strategy="SYNC" />
  </backups>
</distributed-cache>

Reference

2.3. Backing Up to Caches with Different Names

By default, Data Grid replicates data between caches that have the same name.

Procedure

  • Use backup-for to replicate data from a remote site into a cache with a different name on the local site.

For example, the following configuration backs up the "customers" cache on LON to the "eu-customers" cache on NYC. 

<distributed-cache name="eu-customers">
  <backups>
    <backup site="LON" strategy="SYNC" />
  </backups>
  <backup-for remote-cache="customers" remote-site="LON" />
</distributed-cache>

2.4. Configuring Cross-Site State Transfer

Change cross-site state transfer settings to optimize performance and specify whether operations happen manually or automatically.

Procedure

  • Use <state-transfer> to configure state transfer operations.

For example, the following configuration automatically performs cross-site state transfer with custom settings: 

<distributed-cache name="eu-customers">
   <backups>
      <backup site="LON" strategy="ASYNC">
         <state-transfer chunk-size="64"
                         timeout="30000"
                         max-retries="30"
                         wait-time="2000"
                         mode="AUTO"/>
      </backup>
   </backups>
</distributed-cache>

2.5. Customizing the Conflict Resolution Algorithm

Configure Data Grid to use a different algorithm to resolve conflicting entries between backup locations.

Procedure

  1. Open your Data Grid configuration for editing.
  2. Specify one of the Data Grid algorithms or a custom implementation with the merge-policy attribute for the backups element.

Using Data Grid algorithms

Find all Data Grid algorithms and their descriptions in the org.infinispan.xsite.spi.XSiteMergePolicy enum.

The following example configuration uses the ALWAYS_REMOVE algorithm that deletes conflicting entries from both sites: 

<distributed-cache name="eu-customers">
   <backups merge-policy="ALWAYS_REMOVE">
      <backup site="LON" strategy="ASYNC"/>
   </backups>
</distributed-cache>

Using custom implementations

  1. Create a custom XSiteEntryMergePolicy implementation. 

    public interface XSiteEntryMergePolicy<K, V> {
   CompletionStage<SiteEntry<V>> merge(K key, SiteEntry<V> localEntry, SiteEntry<V> remoteEntry);
}

  2. Specify the fully qualified class name as the value of the merge-policy attribute, as in the following example: 

    <distributed-cache name="eu-customers">
   <backups merge-policy="org.mycompany.MyCustomXSiteEntryMergePolicy">
      <backup site="LON" strategy="ASYNC"/>
   </backups>
</distributed-cache>

Additional resources

2.6. Verifying Cross-Site Views

After you configure Data Grid for cross-site replication, you should verify that Data Grid clusters successfully form cross-site views.

Procedure

  • Check log messages for ISPN000439: Received new x-site view messages.

For example, if the Data Grid cluster in LON has formed a cross-site view with the Data Grid cluster in NYC, it provides the following messages: 

INFO  [org.infinispan.XSITE] (jgroups-5,${server.hostname}) ISPN000439: Received new x-site view: [NYC]
INFO  [org.infinispan.XSITE] (jgroups-7,${server.hostname}) ISPN000439: Received new x-site view: [NYC, LON]

2.7. Configuring Hot Rod Clients for Cross-Site Replication

Configure Hot Rod clients to use Data Grid clusters at different sites.

hotrod-client.properties

# Servers at the active site
infinispan.client.hotrod.server_list = LON_host1:11222,LON_host2:11222,LON_host3:11222

# Servers at the backup site
infinispan.client.hotrod.cluster.NYC = NYC_hostA:11222,NYC_hostB:11222,NYC_hostC:11222,NYC_hostD:11222

ConfigurationBuilder

ConfigurationBuilder builder = new ConfigurationBuilder();
builder.addServers("LON_host1:11222;LON_host2:11222;LON_host3:11222")
       .addCluster("NYC")
       .addClusterNodes("NYC_hostA:11222;NYC_hostB:11222;NYC_hostC:11222;NYC_hostD:11222")

Tip

Use the following methods to switch Hot Rod clients to the default cluster or to a cluster at a different site:

  • RemoteCacheManager.switchToDefaultCluster()
  • RemoteCacheManager.switchToCluster(${site.name})

Reference

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.