Search

Chapter 2. Migrating Data Grid Server deployments

download PDF

Review the details in this section to plan and prepare a successful migration of Data Grid Server.

2.1. Data Grid Server 8

Data Grid Server 8 is:

  • Designed for modern system architectures.
  • Built for containerized platforms.
  • Optimized for native image compilation with Quarkus.

The transition to a cloud-native architecture means that Data Grid Server 8 is no longer based on Red Hat JBoss Enterprise Application Platform (EAP). Instead Data Grid Server 8 is based on the Netty project’s client/server framework.

This change affects migration from previous versions because many of the facilities that integration with EAP provided are no longer relevant to Data Grid 8 or have changed.

For instance, while complexity of server configuration is greatly reduced in comparison to previous releases, you do need to adapt your existing configuration to a new schema. Data Grid 8 also provides more of a convention for server configuration than in previous versions where it was possible to achieve much more granular configuration. Additionally Data Grid Server no longer leverages Domain Mode to centrally manage configuration.

The Data Grid team acknowledge that these configuration changes place additional effort on our customers to migrate their existing clusters to Data Grid 8.

We believe that it is better to use container orchestration platforms, such as Red Hat OpenShift, to provision and administer Data Grid clusters along with automation engines, such as Red Hat Ansible, to manage Data Grid configuration. These technologies offer greater flexibility in that they are more generic and suitable for multiple disparate systems, rather than solutions that are more specific to Data Grid.

In terms of migration to Data Grid 8, it is worth noting that solutions like Red Hat Ansible are helpful with large-scale configuration deployment. However, that tooling might not necessarily aid the actual migration of your existing Data Grid configuration.

2.2. Data Grid Server configuration

Data Grid provides a scalable data layer that lets you intelligently and efficiently utilize available computing resources. To achieve this with Data Grid Server deployments, configuration is separated into two layers: dynamic and static.

Dynamic configuration

Dynamic configuration is mutable, changing at runtime as you create caches and add and remove nodes to and from the cluster.

After you deploy your Data Grid Server cluster, you create caches through the Data Grid CLI, Data Grid Console, or Hot Rod and REST endpoints. Data Grid Server permanently stores those caches as part of the cluster state that is distributed across nodes. Each joining node receives the complete cluster state that Data Grid Server automatically synchronizes across all nodes as changes occur.

Static configuration

Static configuration is immutable, remaining unchanged at runtime.

You define static configuration when setting up underlying mechanisms such as cluster transport, authentication and encryption, shared datasources, and so on.

By default Data Grid Server uses $RHDG_HOME/server/conf/infinispan.xml for static configuration.

The root element of the configuration is infinispan and declares two base schema:

<infinispan
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:infinispan:config:11.0 https://infinispan.org/schemas/infinispan-config-11.0.xsd
                            urn:infinispan:server:11.0 https://infinispan.org/schemas/infinispan-server-11.0.xsd"
        xmlns="urn:infinispan:config:11.0"
        xmlns:server="urn:infinispan:server:11.0">
  • The urn:infinispan:config schema validates configuration for core Infinispan capabilities such as the cache container.
  • The urn:infinispan:server schema validates configuration for Data Grid Server.
Cache container configuration

You use the cache-container element to configure the CacheManager interface that provides mechanisms to manage cache lifecycles:

<!-- Creates a cache manager named "default" that exports statistics. -->
<cache-container name="default"
                 statistics="true">
  <!-- Defines cluster transport properties, including the cluster name. -->
  <!-- Uses the default TCP stack for inter-cluster communication. -->
  <transport cluster="${infinispan.cluster.name}"
             stack="${infinispan.cluster.stack:tcp}"
             node-name="${infinispan.node.name:}"/>
</cache-container>

The cache-container element can also hold the following configuration elements:

  • security for the cache manager.
  • metrics for MicroProfile compatible metrics.
  • jmx for JMX monitoring and administration.
Important

In previous versions, you could define multiple cache-container elements in your Data Grid configuration to expose cache containers on different endpoints.

In Data Grid 8 you must not configure multiple cache containers because the Data Grid CLI and Console can handle only one cache manager per cluster. However you can change the name of the cache container to something more meaningful to your environment than "default", if necessary.

You should use separate Data Grid clusters to achieve multitenancy to ensure that cache managers do not interfere with each other.

Server configuration

You use the server element to configure underlying Data Grid Server mechanisms:

<server>
  <interfaces>
    <interface name="public"> 1
      <inet-address value="${infinispan.bind.address:127.0.0.1}"/> 2
    </interface>
  </interfaces>

  <socket-bindings default-interface="public" 3
                   port-offset="${infinispan.socket.binding.port-offset:0}"> 4
    <socket-binding name="default" 5
                    port="${infinispan.bind.port:11222}"/> 6
    <socket-binding name="memcached" port="11221"/> 7
  </socket-bindings>

  <security>
     <security-realms> 8
        <security-realm name="default"> 9
           <server-identities> 10
              <ssl>
                <keystore path="application.keystore" 11
                          keystore-password="password"
                          alias="server"
                          key-password="password"
                          generate-self-signed-certificate-host="localhost"/>
              </ssl>
           </server-identities>
           <properties-realm groups-attribute="Roles"> 12
              <user-properties path="users.properties" 13
                               relative-to="infinispan.server.config.path"
                               plain-text="true"/> 14
              <group-properties path="groups.properties" 15
                                relative-to="infinispan.server.config.path" />
           </properties-realm>
        </security-realm>
     </security-realms>
  </security>

  <endpoints socket-binding="default" security-realm="default" /> 16

</server>
1
Creates an interface named "public" that makes the server available on your network.
2
Uses the 127.0.0.1 loopback address for the public interface.
3
Binds the public interface to the network ports where Data Grid Server endpoints listen for incoming client connections.
4
Specifies an offset of 0 for network ports.
5
Creates a socket binding named "default".
6
Specifies port 11222 for the socket binding.
7
Creates a socket binding for the Memcached connector at port 11221.
8
Defines security realms that protect endpoints from network intrusion.
9
Creates a security realm named "default".
10
Configures SSL/TLS keystores for identity verification.
11
Specifies the keystore that contains server certificates.
12
Configures the "default" security realm to use properties files to define users and groups that map users to roles.
13
Names the properties file that contains Data Grid users.
14
Specifies that contents of the users.properties file are stored as plain text.
15
Names the properties file that maps Data Grid users to roles.
16
Configures endpoints with Hot Rod and REST connectors.

This example shows implicit hotrod-connector and rest-connector elements, which is the default from Data Grid 8.2.
Data Grid Server configuration in 8.0 and 8.1 use explicitly declared Hot Rod and REST connectors.

2.3. Data Grid Server endpoint and network configuration

This section describes Data Grid Server endpoint and network configuration when migrating from previous versions.

Data Grid 8 simplifies server endpoint configuration by using a single network interface and port to expose endpoints on the network.

2.3.1. Interfaces

Interfaces bind expose endpoints to network locations.

Data Grid Server 7.x network interface configuration

In Data Grid 7.x, the server configuration used different interfaces to separate administrative and management access from cache access.

<interfaces>
  <interface name="management">
    <inet-address value="${jboss.bind.address.management:127.0.0.1}"/>
  </interface>
  <interface name="public">
    <inet-address value="${jboss.bind.address:127.0.0.1}"/>
  </interface>
</interfaces>
Data Grid Server 8 network interface configuration

In Data Grid 8, there is one network interface for all client connections for administrative and management access as well as cache access.

<interfaces>
  <interface name="public">
    <inet-address value="${infinispan.bind.address:127.0.0.1}"/>
  </interface>
</interfaces>

2.3.2. Socket bindings

Socket bindings map network interfaces to ports where endpoints listen for client connections.

Data Grid Server 7.x socket binding configuration

In Data Grid 7.x, the server configuration used unique ports for management and administration, such as 9990 for the Management Console and port 9999 for the native management protocol. Older versions also used unique ports for each endpoint, such as 11222 for external Hot Rod access and 8080 for REST.

<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
  <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
  <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>
  <socket-binding name="hotrod" port="11222"/>
  <socket-binding name="hotrod-internal" port="11223"/>
  <socket-binding name="hotrod-multi-tenancy" port="11224"/>
  <socket-binding name="memcached" port="11211"/>
  <socket-binding name="rest" port="8080"/>
  ...
</socket-binding-group>
Data Grid Server 8 single port configuration

Data Grid 8 uses a single port to handle all connections to the server. Hot Rod clients, REST clients, Data Grid CLI, and Data Grid Console all use port 11222.

<socket-bindings default-interface="public"
                 port-offset="${infinispan.socket.binding.port-offset:0}">
  <socket-binding name="default" port="${infinispan.bind.port:11222}"/>
  <socket-binding name="memcached" port="11221"/>
</socket-bindings>

2.3.3. Endpoints

Endpoints listen for remote client connections and handle requests over protocols such as Hot Rod and HTTP (REST).

Note

Data Grid CLI uses the REST endpoint for all cache and administrative operations.

Data Grid Server 7.x endpoint subsystem

In Data Grid 7.x, the endpoint subsystem let you configure connectors for Hot Rod and REST endpoints.

<subsystem xmlns="urn:infinispan:server:endpoint:9.4">
  <hotrod-connector socket-binding="hotrod" cache-container="local">
    <topology-state-transfer lazy-retrieval="false" lock-timeout="1000" replication-timeout="5000"/>
  </hotrod-connector>
  <rest-connector socket-binding="rest" cache-container="local">
    <authentication security-realm="ApplicationRealm" auth-method="BASIC"/>
  </rest-connector>
</subsystem>
Data Grid Server 8 endpoint configuration

Data Grid 8 replaces the endpoint subsystem with an endpoints element. The hotrod-connector and rest-connector configuration elements and attributes are the same as previous versions.

<endpoints socket-binding="default" security-realm="default">
  <hotrod-connector name="hotrod"/>
  <rest-connector name="rest"/>
</endpoints>

Additional resources

2.4. Data Grid Server security

Data Grid Server security configures authentication and encryption to prevent network attack and safeguard data.

2.4.1. Security realms

In Data Grid 8 security realms provide implicit configuration options that mean you do not need to provide as many settings as in previous versions. For example, if you define a Kerberos realm, you get Kerberos features. If you add a truststore, you get certificate authentication.

In Data Grid 7.x, there were two default security realms:

  • ManagementRealm secures the Management API.
  • ApplicationRealm secures endpoints and remote client connections.

Data Grid 8, on the other hand, provides a security element that lets you define multiple different security realms that you can use for Hot Rod and REST endpoints:

<security>
  <security-realms>
    ...
  </security-realms>
</security>
Supported security realms
  • Property realms use property files, users.properties and groups.properties, to define users and groups that can access Data Grid.
  • LDAP realms connect to LDAP servers, such as OpenLDAP, Red Hat Directory Server, Apache Directory Server, or Microsoft Active Directory, to authenticate users and obtain membership information.
  • Trust store realms use keystores that contain the public certificates of all clients that are allowed to access Data Grid.
  • Token realms use external services to validate tokens and require providers that are compatible with RFC-7662 (OAuth2 Token Introspection) such as Red Hat SSO.

2.4.2. Server identities

Server identities use certificate chains to prove Data Grid Server identities to remote clients.

Data Grid 8 uses the same configuration to define SSL identities as in previous versions with some usability improvements.

  • If a security realm contains an SSL identity, Data Grid automatically enables encryption for endpoints that use that security realm.
  • For test and development environments, Data Grid includes a generate-self-signed-certificate-host attribute that automatically generates a keystore at startup.
<security-realm name="default">
  <server-identities>
    <ssl>
      <keystore path="..."
                relative-to="..."
                keystore-password="..."
                alias="..."
                key-password="..."
                generate-self-signed-certificate-host="..."/>
    </ssl>
  </server-identities>
  ...
<security-realm>

2.4.3. Endpoint authentication mechanisms

Hot Rod and REST endpoints use SASL or HTTP mechanisms to authenticate client connections.

Data Grid 8 uses the same authentication element for hotrod-connector and rest-connector configuration as in Data Grid 7.x and earlier.

<hotrod-connector name="hotrod">
  <authentication>
    <sasl mechanisms="..." server-name="..."/>
  </authentication>
</hotrod-connector>
<rest-connector name="rest">
  <authentication>
    <mechanisms="..." server-principal="..."/>
  </authentication>
</rest-connector>

One key difference with previous versions is that Data Grid 8 supports additional authentication mechanisms for endpoints.

Hot Rod SASL authentication mechanisms

Hot Rod clients now use SCRAM-SHA-512 as the default authentication mechanism instead of DIGEST-MD5.

Note

If you use property security realms, you must use the PLAIN authentication mechanism.

Authentication mechanismDescriptionRelated details

PLAIN

Uses credentials in plain-text format. You should use PLAIN authentication with encrypted connections only.

Similar to the Basic HTTP mechanism.

DIGEST-*

Uses hashing algorithms and nonce values. Hot Rod connectors support DIGEST-MD5, DIGEST-SHA, DIGEST-SHA-256, DIGEST-SHA-384, and DIGEST-SHA-512 hashing algorithms, in order of strength.

Similar to the Digest HTTP mechanism.

SCRAM-*

Uses salt values in addition to hashing algorithms and nonce values. Hot Rod connectors support SCRAM-SHA, SCRAM-SHA-256, SCRAM-SHA-384, and SCRAM-SHA-512 hashing algorithms, in order of strength.

Similar to the Digest HTTP mechanism.

GSSAPI

Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding kerberos server identity in the realm configuration. In most cases, you also specify an ldap-realm to provide user membership information.

Similar to the SPNEGO HTTP mechanism.

GS2-KRB5

Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding kerberos server identity in the realm configuration. In most cases, you also specify an ldap-realm to provide user membership information.

Similar to the SPNEGO HTTP mechanism.

EXTERNAL

Uses client certificates.

Similar to the CLIENT_CERT HTTP mechanism.

OAUTHBEARER

Uses OAuth tokens and requires a token-realm configuration.

Similar to the BEARER_TOKEN HTTP mechanism.

HTTP (REST) authentication mechanisms
Authentication mechanismDescriptionRelated details

BASIC

Uses credentials in plain-text format. You should use BASIC authentication with encrypted connections only.

Corresponds to the Basic HTTP authentication scheme and is similar to the PLAIN SASL mechanism.

DIGEST

Uses hashing algorithms and nonce values. REST connectors support SHA-512, SHA-256 and MD5 hashing algorithms.

Corresponds to the Digest HTTP authentication scheme and is similar to DIGEST-* SASL mechanisms.

SPNEGO

Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding kerberos server identity in the realm configuration. In most cases, you also specify an ldap-realm to provide user membership information.

Corresponds to the Negotiate HTTP authentication scheme and is similar to the GSSAPI and GS2-KRB5 SASL mechanisms.

BEARER_TOKEN

Uses OAuth tokens and requires a token-realm configuration.

Corresponds to the Bearer HTTP authentication scheme and is similar to OAUTHBEARER SASL mechanism.

CLIENT_CERT

Uses client certificates.

Similar to the EXTERNAL SASL mechanism.

2.4.4. Authenticating EAP applications

You can now add credentials to hotrod-client.properties on your EAP application classpath to authenticate with Data Grid through:

  • Remote cache containers (remote-cache-container)
  • Remote stores (remote-store)
  • EAP modules

2.4.5. Logging

Data Grid uses Apache Log4j2 instead of the logging subsystem in previous versions that was based on JBossLogManager.

By default, Data Grid writes log messages to the following directory:
$RHDG_HOME/${infinispan.server.root}/log

server.log is the default log file.

Access logs

In previous versions Data Grid included a logger to audit security logs for the caches:

<authorization audit-logger="org.infinispan.security.impl.DefaultAuditLogger">

Data Grid 8 no longer provides this audit logger.

However you can use the logging categories for the Hot Rod and REST endpoints:

  • org.infinispan.HOTROD_ACCESS_LOG
  • org.infinispan.REST_ACCESS_LOG

Additional resources

2.5. Separating Data Grid Server endpoints

When migrating from previous versions, you can create different network locations for Data Grid endpoints to match your existing configuration. However, because Data Grid architecture has changed and now uses a single port for all client connections, not all options in previous versions are available.

Important

Administration tools such as the Data Grid CLI and Console use the REST API. You cannot remove the REST API from your endpoint configuration without disabling the Data Grid CLI and Console. Likewise you cannot separate the REST endpoint to use different ports or socket bindings for cache access and administrative access.

Procedure

  1. Define separate network interfaces for REST and Hot Rod endpoints.

    For example, define a "public" interface to expose the Hot Rod endpoint externally and a "private" interface to expose the REST endpoint on an network location that has restricted access.

    <interfaces>
      <interface name="public">
        <inet-address value="${infinispan.bind.address:198.51.100.0}"/>
      </interface>
      <interface name="private">
        <inet-address value="${infinispan.bind.address:192.0.2.0}"/>
      </interface>
    </interfaces>

    This configuration creates:

    • A "public" interface with the 198.51.100.0 IP address.
    • A "private" interface with the 192.0.2.0 IP address.
  2. Configure separate socket bindings for the endpoints, as in the following example:

    <socket-bindings default-interface="private"
                     port-offset="${infinispan.socket.binding.port-offset:0}">
      <socket-binding name="default"
                      port="${infinispan.bind.port:8080}"/>
      <socket-binding name="hotrod"
                      interface="public"
                      port="11222"/>
    </socket-bindings>

    This example:

    • Sets the "private" interface as the default for socket bindings.
    • Creates a "default" socket binding that uses port 8080.
    • Creates a "hotrod" socket binding that uses the "public" interface and port 11222.
  3. Create separate security realms for the endpoints, for example:

    <security>
      <security-realms>
        <security-realm name="truststore">
          <server-identities>
            <ssl>
              <keystore path="server.p12"
                        relative-to="infinispan.server.config.path"
                        keystore-password="secret"
                        alias="server"/>
            </ssl>
          </server-identities>
          <truststore-realm path="trust.p12"
                            relative-to="infinispan.server.config.path"
                            keystore-password="secret"/>
        </security-realm>
        <security-realm name="kerberos">
          <server-identities>
            <kerberos keytab-path="http.keytab"
                      principal="HTTP/localhost@INFINISPAN.ORG"
                      required="true"/>
          </server-identities>
        </security-realm>
      </security-realms>
    </security>

    This example:

    • Configures a trust store security realm.
    • Configures a Kerberos security realm.
  4. Configure endpoints as follows:

    <endpoints socket-binding="default"
               security-realm="kerberos">
      <hotrod-connector name="hotrod"
                        socket-binding="hotrod"
                        security-realm="truststore"/>
      <rest-connector name="rest"/>
    </endpoints>
  5. Start Data Grid Server.

    Logs contain the following messages that indicate the network locations where endpoints accept client connections:

    [org.infinispan.SERVER] ISPN080004: Protocol HotRod listening on 198.51.100.0:11222
    [org.infinispan.SERVER] ISPN080004: Protocol SINGLE_PORT listening on 192.0.2.0:8080
    [org.infinispan.SERVER] ISPN080034: Server '<hostname>' listening on http://192.0.2.0:8080

Next steps

  1. Access Data Grid Console from any browser at http://192.0.2.0:8080
  2. Configure the Data Grid CLI to connect at the custom location, for example:

    $ bin/cli.sh -c http://192.0.2.0:8080

2.6. Data Grid Server shared datasources

Data Grid 7.x JDBC cache stores can use a PooledConnectionFactory to obtain database connections.

Data Grid 8 lets you create managed datasources in the server configuration to optimize connection pooling and performance for database connections with JDBC cache stores.

Datasource configurations are composed of two sections:

  • connection factory that defines how to connect to the database.
  • connection pool that defines how to pool and reuse connections and is based on Agroal.

You first define the datasource connection factory and connection pool in the server configuration and then add it to your JDBC cache store configuration.

For more information on migrating JDBC cache stores, see the Migrating Cache Stores section in this document.

Additional resources

2.7. Data Grid Server JMX and metrics

Data Grid 8 exposes metrics via both JMX and a /metrics endpoint for integration with metrics tooling such as Prometheus.

The /metrics endpoint provides:

  • Gauges that return values, such as JVM uptime or average number of seconds for cache operations.
  • Histograms that show how long read, write, and remove operations take, in percentiles.

In previous versions, Prometheus metrics were collected by an agent that mapped JMX metrics instead of being supported natively.

Previous versions of Data Grid also used the JBoss Operations Network (JON) plug-in to obtain metrics and perform operations. Data Grid 8 no longer uses the JON plug-in.

Data Grid 8 separates JMX and Prometheus metrics into cache manager and cache level configurations.

<cache-container name="default"
                 statistics="true"> 1
  <jmx enabled="true" /> 2
</cache-container>
1
Enables statistics for the cache manager. This is the default.
2
Exports JMX MBeans, which includes all statistics and operations.
<distributed-cache name="mycache" statistics="true" /> 1
1
Enables statistics for the cache.

Additional resources

2.8. Data Grid Server cheatsheet

Use the following commands and examples as a quick reference for working with Data Grid Server.

Starting server instances

  • Linux

    $ bin/server.sh
  • Microsoft Windows

    $ bin\server.bat

Starting the CLI

  • Linux

    $ bin/cli.sh
  • Microsoft Windows

    $ bin\cli.bat

Creating users

  • Linux

    $ bin/cli.sh user create myuser -p "qwer1234!"
  • Microsoft Windows

    $ bin\cli.bat user create myuser -p "qwer1234!"

Stopping server instances

  • Single server instances

    [//containers/default]> shutdown server $hostname
  • Entire clusters

    [//containers/default]> shutdown cluster

Listing available command options

Use the -h flag to list available command options for running servers.

  • Linux

    $ bin/server.sh -h
  • Microsoft Windows

    $ bin\server.bat -h

7.x to 8 reference

7.x8.x

./standalone.sh -c clustered.xml

./server.sh

./standalone.sh

./server.sh -c infinispan-local.xml

-Djboss.default.multicast.address=234.99.54.20

-Djgroups.mcast_addr=234.99.54.20

-Djboss.bind.address=172.18.1.13

-Djgroups.bind.address=172.18.1.13

-Djboss.default.jgroups.stack=udp

-j udp

Additional resources

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.