Chapter 2. Migrating Data Grid Server deployments
Review the details in this section to plan and prepare a successful migration of Data Grid Server.
2.1. Data Grid Server 8
Data Grid Server 8 is:
- Designed for modern system architectures.
- Built for containerized platforms.
- Optimized for native image compilation with Quarkus.
The transition to a cloud-native architecture means that Data Grid Server 8 is no longer based on Red Hat JBoss Enterprise Application Platform (EAP). Instead Data Grid Server 8 is based on the Netty project’s client/server framework.
This change affects migration from previous versions because many of the facilities that integration with EAP provided are no longer relevant to Data Grid 8 or have changed.
For instance, while complexity of server configuration is greatly reduced in comparison to previous releases, you do need to adapt your existing configuration to a new schema. Data Grid 8 also provides more of a convention for server configuration than in previous versions where it was possible to achieve much more granular configuration. Additionally Data Grid Server no longer leverages Domain Mode to centrally manage configuration.
The Data Grid team acknowledge that these configuration changes place additional effort on our customers to migrate their existing clusters to Data Grid 8.
We believe that it is better to use container orchestration platforms, such as Red Hat OpenShift, to provision and administer Data Grid clusters along with automation engines, such as Red Hat Ansible, to manage Data Grid configuration. These technologies offer greater flexibility in that they are more generic and suitable for multiple disparate systems, rather than solutions that are more specific to Data Grid.
In terms of migration to Data Grid 8, it is worth noting that solutions like Red Hat Ansible are helpful with large-scale configuration deployment. However, that tooling might not necessarily aid the actual migration of your existing Data Grid configuration.
2.2. Data Grid Server configuration
Data Grid provides a scalable data layer that lets you intelligently and efficiently utilize available computing resources. To achieve this with Data Grid Server deployments, configuration is separated into two layers: dynamic and static.
Dynamic configuration
Dynamic configuration is mutable, changing at runtime as you create caches and add and remove nodes to and from the cluster.
After you deploy your Data Grid Server cluster, you create caches through the Data Grid CLI, Data Grid Console, or Hot Rod and REST endpoints. Data Grid Server permanently stores those caches as part of the cluster state that is distributed across nodes. Each joining node receives the complete cluster state that Data Grid Server automatically synchronizes across all nodes as changes occur.
Static configuration
Static configuration is immutable, remaining unchanged at runtime.
You define static configuration when setting up underlying mechanisms such as cluster transport, authentication and encryption, shared datasources, and so on.
By default Data Grid Server uses $RHDG_HOME/server/conf/infinispan.xml
for static configuration.
The root element of the configuration is infinispan
and declares two base schema:
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:infinispan:config:11.0 https://infinispan.org/schemas/infinispan-config-11.0.xsd urn:infinispan:server:11.0 https://infinispan.org/schemas/infinispan-server-11.0.xsd" xmlns="urn:infinispan:config:11.0" xmlns:server="urn:infinispan:server:11.0">
-
The
urn:infinispan:config
schema validates configuration for core Infinispan capabilities such as the cache container. -
The
urn:infinispan:server
schema validates configuration for Data Grid Server.
Cache container configuration
You use the cache-container
element to configure the CacheManager
interface that provides mechanisms to manage cache lifecycles:
<!-- Creates a cache manager named "default" that exports statistics. --> <cache-container name="default" statistics="true"> <!-- Defines cluster transport properties, including the cluster name. --> <!-- Uses the default TCP stack for inter-cluster communication. --> <transport cluster="${infinispan.cluster.name}" stack="${infinispan.cluster.stack:tcp}" node-name="${infinispan.node.name:}"/> </cache-container>
The cache-container
element can also hold the following configuration elements:
-
security
for the cache manager. -
metrics
for MicroProfile compatible metrics. -
jmx
for JMX monitoring and administration.
In previous versions, you could define multiple cache-container
elements in your Data Grid configuration to expose cache containers on different endpoints.
In Data Grid 8 you must not configure multiple cache containers because the Data Grid CLI and Console can handle only one cache manager per cluster. However you can change the name of the cache container to something more meaningful to your environment than "default", if necessary.
You should use separate Data Grid clusters to achieve multitenancy to ensure that cache managers do not interfere with each other.
Server configuration
You use the server
element to configure underlying Data Grid Server mechanisms:
<server> <interfaces> <interface name="public"> 1 <inet-address value="${infinispan.bind.address:127.0.0.1}"/> 2 </interface> </interfaces> <socket-bindings default-interface="public" 3 port-offset="${infinispan.socket.binding.port-offset:0}"> 4 <socket-binding name="default" 5 port="${infinispan.bind.port:11222}"/> 6 <socket-binding name="memcached" port="11221"/> 7 </socket-bindings> <security> <security-realms> 8 <security-realm name="default"> 9 <server-identities> 10 <ssl> <keystore path="application.keystore" 11 keystore-password="password" alias="server" key-password="password" generate-self-signed-certificate-host="localhost"/> </ssl> </server-identities> <properties-realm groups-attribute="Roles"> 12 <user-properties path="users.properties" 13 relative-to="infinispan.server.config.path" plain-text="true"/> 14 <group-properties path="groups.properties" 15 relative-to="infinispan.server.config.path" /> </properties-realm> </security-realm> </security-realms> </security> <endpoints socket-binding="default" security-realm="default" /> 16 </server>
- 1
- Creates an interface named "public" that makes the server available on your network.
- 2
- Uses the
127.0.0.1
loopback address for the public interface. - 3
- Binds the public interface to the network ports where Data Grid Server endpoints listen for incoming client connections.
- 4
- Specifies an offset of
0
for network ports. - 5
- Creates a socket binding named "default".
- 6
- Specifies port
11222
for the socket binding. - 7
- Creates a socket binding for the Memcached connector at port
11221
. - 8
- Defines security realms that protect endpoints from network intrusion.
- 9
- Creates a security realm named "default".
- 10
- Configures SSL/TLS keystores for identity verification.
- 11
- Specifies the keystore that contains server certificates.
- 12
- Configures the "default" security realm to use properties files to define users and groups that map users to roles.
- 13
- Names the properties file that contains Data Grid users.
- 14
- Specifies that contents of the
users.properties
file are stored as plain text. - 15
- Names the properties file that maps Data Grid users to roles.
- 16
- Configures endpoints with Hot Rod and REST connectors.
This example shows implicit
hotrod-connector
andrest-connector
elements, which is the default from Data Grid 8.2.
Data Grid Server configuration in 8.0 and 8.1 use explicitly declared Hot Rod and REST connectors.
Additional resources
2.3. Data Grid Server endpoint and network configuration
This section describes Data Grid Server endpoint and network configuration when migrating from previous versions.
Data Grid 8 simplifies server endpoint configuration by using a single network interface and port to expose endpoints on the network.
2.3.1. Interfaces
Interfaces bind expose endpoints to network locations.
Data Grid Server 7.x network interface configuration
In Data Grid 7.x, the server configuration used different interfaces to separate administrative and management access from cache access.
<interfaces> <interface name="management"> <inet-address value="${jboss.bind.address.management:127.0.0.1}"/> </interface> <interface name="public"> <inet-address value="${jboss.bind.address:127.0.0.1}"/> </interface> </interfaces>
Data Grid Server 8 network interface configuration
In Data Grid 8, there is one network interface for all client connections for administrative and management access as well as cache access.
<interfaces> <interface name="public"> <inet-address value="${infinispan.bind.address:127.0.0.1}"/> </interface> </interfaces>
2.3.2. Socket bindings
Socket bindings map network interfaces to ports where endpoints listen for client connections.
Data Grid Server 7.x socket binding configuration
In Data Grid 7.x, the server configuration used unique ports for management and administration, such as 9990
for the Management Console and port 9999
for the native management protocol. Older versions also used unique ports for each endpoint, such as 11222
for external Hot Rod access and 8080
for REST.
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}"> <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/> <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/> <socket-binding name="hotrod" port="11222"/> <socket-binding name="hotrod-internal" port="11223"/> <socket-binding name="hotrod-multi-tenancy" port="11224"/> <socket-binding name="memcached" port="11211"/> <socket-binding name="rest" port="8080"/> ... </socket-binding-group>
Data Grid Server 8 single port configuration
Data Grid 8 uses a single port to handle all connections to the server. Hot Rod clients, REST clients, Data Grid CLI, and Data Grid Console all use port 11222
.
<socket-bindings default-interface="public" port-offset="${infinispan.socket.binding.port-offset:0}"> <socket-binding name="default" port="${infinispan.bind.port:11222}"/> <socket-binding name="memcached" port="11221"/> </socket-bindings>
2.3.3. Endpoints
Endpoints listen for remote client connections and handle requests over protocols such as Hot Rod and HTTP (REST).
Data Grid CLI uses the REST endpoint for all cache and administrative operations.
Data Grid Server 7.x endpoint subsystem
In Data Grid 7.x, the endpoint
subsystem let you configure connectors for Hot Rod and REST endpoints.
<subsystem xmlns="urn:infinispan:server:endpoint:9.4"> <hotrod-connector socket-binding="hotrod" cache-container="local"> <topology-state-transfer lazy-retrieval="false" lock-timeout="1000" replication-timeout="5000"/> </hotrod-connector> <rest-connector socket-binding="rest" cache-container="local"> <authentication security-realm="ApplicationRealm" auth-method="BASIC"/> </rest-connector> </subsystem>
Data Grid Server 8 endpoint configuration
Data Grid 8 replaces the endpoint
subsystem with an endpoints
element. The hotrod-connector
and rest-connector
configuration elements and attributes are the same as previous versions.
<endpoints socket-binding="default" security-realm="default"> <hotrod-connector name="hotrod"/> <rest-connector name="rest"/> </endpoints>
Additional resources
2.4. Data Grid Server security
Data Grid Server security configures authentication and encryption to prevent network attack and safeguard data.
2.4.1. Security realms
In Data Grid 8 security realms provide implicit configuration options that mean you do not need to provide as many settings as in previous versions. For example, if you define a Kerberos realm, you get Kerberos features. If you add a truststore, you get certificate authentication.
In Data Grid 7.x, there were two default security realms:
-
ManagementRealm
secures the Management API. -
ApplicationRealm
secures endpoints and remote client connections.
Data Grid 8, on the other hand, provides a security
element that lets you define multiple different security realms that you can use for Hot Rod and REST endpoints:
<security> <security-realms> ... </security-realms> </security>
Supported security realms
-
Property realms use property files,
users.properties
andgroups.properties
, to define users and groups that can access Data Grid. - LDAP realms connect to LDAP servers, such as OpenLDAP, Red Hat Directory Server, Apache Directory Server, or Microsoft Active Directory, to authenticate users and obtain membership information.
- Trust store realms use keystores that contain the public certificates of all clients that are allowed to access Data Grid.
- Token realms use external services to validate tokens and require providers that are compatible with RFC-7662 (OAuth2 Token Introspection) such as Red Hat SSO.
2.4.2. Server identities
Server identities use certificate chains to prove Data Grid Server identities to remote clients.
Data Grid 8 uses the same configuration to define SSL identities as in previous versions with some usability improvements.
- If a security realm contains an SSL identity, Data Grid automatically enables encryption for endpoints that use that security realm.
-
For test and development environments, Data Grid includes a
generate-self-signed-certificate-host
attribute that automatically generates a keystore at startup.
<security-realm name="default"> <server-identities> <ssl> <keystore path="..." relative-to="..." keystore-password="..." alias="..." key-password="..." generate-self-signed-certificate-host="..."/> </ssl> </server-identities> ... <security-realm>
2.4.3. Endpoint authentication mechanisms
Hot Rod and REST endpoints use SASL or HTTP mechanisms to authenticate client connections.
Data Grid 8 uses the same authentication
element for hotrod-connector
and rest-connector
configuration as in Data Grid 7.x and earlier.
<hotrod-connector name="hotrod"> <authentication> <sasl mechanisms="..." server-name="..."/> </authentication> </hotrod-connector> <rest-connector name="rest"> <authentication> <mechanisms="..." server-principal="..."/> </authentication> </rest-connector>
One key difference with previous versions is that Data Grid 8 supports additional authentication mechanisms for endpoints.
Hot Rod SASL authentication mechanisms
Hot Rod clients now use SCRAM-SHA-512
as the default authentication mechanism instead of DIGEST-MD5
.
If you use property security realms, you must use the PLAIN
authentication mechanism.
Authentication mechanism | Description | Related details |
---|---|---|
|
Uses credentials in plain-text format. You should use |
Similar to the |
|
Uses hashing algorithms and nonce values. Hot Rod connectors support |
Similar to the |
|
Uses salt values in addition to hashing algorithms and nonce values. Hot Rod connectors support |
Similar to the |
|
Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding |
Similar to the |
|
Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding |
Similar to the |
| Uses client certificates. |
Similar to the |
|
Uses OAuth tokens and requires a |
Similar to the |
HTTP (REST) authentication mechanisms
Authentication mechanism | Description | Related details |
---|---|---|
|
Uses credentials in plain-text format. You should use |
Corresponds to the |
|
Uses hashing algorithms and nonce values. REST connectors support |
Corresponds to the |
|
Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding |
Corresponds to the |
|
Uses OAuth tokens and requires a |
Corresponds to the |
| Uses client certificates. |
Similar to the |
2.4.4. Authenticating EAP applications
You can now add credentials to hotrod-client.properties
on your EAP application classpath to authenticate with Data Grid through:
-
Remote cache containers (
remote-cache-container
) -
Remote stores (
remote-store
) - EAP modules
2.4.5. Logging
Data Grid uses Apache Log4j2 instead of the logging subsystem in previous versions that was based on JBossLogManager.
By default, Data Grid writes log messages to the following directory:$RHDG_HOME/${infinispan.server.root}/log
server.log
is the default log file.
Access logs
In previous versions Data Grid included a logger to audit security logs for the caches:
<authorization audit-logger="org.infinispan.security.impl.DefaultAuditLogger">
Data Grid 8 no longer provides this audit logger.
However you can use the logging categories for the Hot Rod and REST endpoints:
-
org.infinispan.HOTROD_ACCESS_LOG
-
org.infinispan.REST_ACCESS_LOG
Additional resources
2.5. Separating Data Grid Server endpoints
When migrating from previous versions, you can create different network locations for Data Grid endpoints to match your existing configuration. However, because Data Grid architecture has changed and now uses a single port for all client connections, not all options in previous versions are available.
Administration tools such as the Data Grid CLI and Console use the REST API. You cannot remove the REST API from your endpoint configuration without disabling the Data Grid CLI and Console. Likewise you cannot separate the REST endpoint to use different ports or socket bindings for cache access and administrative access.
Procedure
Define separate network interfaces for REST and Hot Rod endpoints.
For example, define a "public" interface to expose the Hot Rod endpoint externally and a "private" interface to expose the REST endpoint on an network location that has restricted access.
<interfaces> <interface name="public"> <inet-address value="${infinispan.bind.address:198.51.100.0}"/> </interface> <interface name="private"> <inet-address value="${infinispan.bind.address:192.0.2.0}"/> </interface> </interfaces>
This configuration creates:
-
A "public" interface with the
198.51.100.0
IP address. -
A "private" interface with the
192.0.2.0
IP address.
-
A "public" interface with the
Configure separate socket bindings for the endpoints, as in the following example:
<socket-bindings default-interface="private" port-offset="${infinispan.socket.binding.port-offset:0}"> <socket-binding name="default" port="${infinispan.bind.port:8080}"/> <socket-binding name="hotrod" interface="public" port="11222"/> </socket-bindings>
This example:
- Sets the "private" interface as the default for socket bindings.
-
Creates a "default" socket binding that uses port
8080
. -
Creates a "hotrod" socket binding that uses the "public" interface and port
11222
.
Create separate security realms for the endpoints, for example:
<security> <security-realms> <security-realm name="truststore"> <server-identities> <ssl> <keystore path="server.p12" relative-to="infinispan.server.config.path" keystore-password="secret" alias="server"/> </ssl> </server-identities> <truststore-realm path="trust.p12" relative-to="infinispan.server.config.path" keystore-password="secret"/> </security-realm> <security-realm name="kerberos"> <server-identities> <kerberos keytab-path="http.keytab" principal="HTTP/localhost@INFINISPAN.ORG" required="true"/> </server-identities> </security-realm> </security-realms> </security>
This example:
- Configures a trust store security realm.
- Configures a Kerberos security realm.
Configure endpoints as follows:
<endpoints socket-binding="default" security-realm="kerberos"> <hotrod-connector name="hotrod" socket-binding="hotrod" security-realm="truststore"/> <rest-connector name="rest"/> </endpoints>
Start Data Grid Server.
Logs contain the following messages that indicate the network locations where endpoints accept client connections:
[org.infinispan.SERVER] ISPN080004: Protocol HotRod listening on 198.51.100.0:11222 [org.infinispan.SERVER] ISPN080004: Protocol SINGLE_PORT listening on 192.0.2.0:8080 [org.infinispan.SERVER] ISPN080034: Server '<hostname>' listening on http://192.0.2.0:8080
Next steps
-
Access Data Grid Console from any browser at
http://192.0.2.0:8080
Configure the Data Grid CLI to connect at the custom location, for example:
$ bin/cli.sh -c http://192.0.2.0:8080
2.6. Data Grid Server shared datasources
Data Grid 7.x JDBC cache stores can use a PooledConnectionFactory
to obtain database connections.
Data Grid 8 lets you create managed datasources in the server configuration to optimize connection pooling and performance for database connections with JDBC cache stores.
Datasource configurations are composed of two sections:
-
connection factory
that defines how to connect to the database. -
connection pool
that defines how to pool and reuse connections and is based on Agroal.
You first define the datasource connection factory and connection pool in the server configuration and then add it to your JDBC cache store configuration.
For more information on migrating JDBC cache stores, see the Migrating Cache Stores section in this document.
Additional resources
2.7. Data Grid Server JMX and metrics
Data Grid 8 exposes metrics via both JMX and a /metrics
endpoint for integration with metrics tooling such as Prometheus.
The /metrics
endpoint provides:
- Gauges that return values, such as JVM uptime or average number of seconds for cache operations.
- Histograms that show how long read, write, and remove operations take, in percentiles.
In previous versions, Prometheus metrics were collected by an agent that mapped JMX metrics instead of being supported natively.
Previous versions of Data Grid also used the JBoss Operations Network (JON) plug-in to obtain metrics and perform operations. Data Grid 8 no longer uses the JON plug-in.
Data Grid 8 separates JMX and Prometheus metrics into cache manager and cache level configurations.
<cache-container name="default" statistics="true"> 1 <jmx enabled="true" /> 2 </cache-container>
<distributed-cache name="mycache" statistics="true" /> 1
- 1
- Enables statistics for the cache.
Additional resources
2.8. Data Grid Server cheatsheet
Use the following commands and examples as a quick reference for working with Data Grid Server.
Starting server instances
Linux
$ bin/server.sh
Microsoft Windows
$ bin\server.bat
Starting the CLI
Linux
$ bin/cli.sh
Microsoft Windows
$ bin\cli.bat
Creating users
Linux
$ bin/cli.sh user create myuser -p "qwer1234!"
Microsoft Windows
$ bin\cli.bat user create myuser -p "qwer1234!"
Stopping server instances
Single server instances
[//containers/default]> shutdown server $hostname
Entire clusters
[//containers/default]> shutdown cluster
Listing available command options
Use the -h
flag to list available command options for running servers.
Linux
$ bin/server.sh -h
Microsoft Windows
$ bin\server.bat -h
7.x to 8 reference
7.x | 8.x |
---|---|
|
|
|
|
|
|
|
|
|
|
Additional resources