Building and deploying Data Grid clusters with Helm
Create Data Grid clusters on OpenShift
Abstract
Red Hat Data Grid Copy linkLink copied to clipboard!
Data Grid is a high-performance, distributed in-memory data store.
- Schemaless data structure
- Flexibility to store different objects as key-value pairs.
- Grid-based data storage
- Designed to distribute and replicate data across clusters.
- Elastic scaling
- Dynamically adjust the number of nodes to meet demand without service disruption.
- Data interoperability
- Store, retrieve, and query data in the grid from different endpoints.
Data Grid documentation Copy linkLink copied to clipboard!
Documentation for Data Grid is available on the Red Hat customer portal.
Data Grid downloads Copy linkLink copied to clipboard!
Access the Data Grid Software Downloads on the Red Hat customer portal.
You must have a Red Hat account to access and download Data Grid software.
Making open source more inclusive Copy linkLink copied to clipboard!
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.
Chapter 1. Deploying Data Grid clusters as Helm chart releases Copy linkLink copied to clipboard!
Build, configure, and deploy Data Grid clusters with Helm. Data Grid provides a Helm chart that packages resources for running Data Grid clusters on OpenShift.
Install the Data Grid chart to create a Helm release, which instantiates a Data Grid cluster in your OpenShift project.
1.1. Installing the Data Grid chart through the OpenShift console Copy linkLink copied to clipboard!
Use the OpenShift Web Console to install the Data Grid chart from the Red Hat developer catalog. Installing the chart creates a Helm release that deploys a Data Grid cluster.
Prerequisites
- Have access to OpenShift.
Procedure
- Log in to the OpenShift Web Console.
- Select the Developer perspective.
- Open the Add view and then select Helm Chart to browse the Red Hat developer catalog.
- Locate and select the Data Grid chart.
- Specify a name for the chart and select a version.
Define values in the following sections of the Data Grid chart:
- Images configures the container images to use when creating pods for your Data Grid cluster.
Deploy configures your Data Grid cluster.
TipTo find descriptions for each value, select the YAML view option and access the schema. Edit the yaml configuration to customize your Data Grid chart.
- Select Install.
Verification
- Select the Helm view in the Developer perspective.
- Select the Helm release you created to view details, resources, and other information.
1.2. Installing the Data Grid chart on the command line Copy linkLink copied to clipboard!
Use the command line to install the Data Grid chart on OpenShift and instantiate a Data Grid cluster. Installing the chart creates a Helm release that deploys a Data Grid cluster.
Prerequisites
-
Install the
helmclient. - Add the OpenShift Helm Charts repository.
- Have access to an OpenShift cluster.
-
Have an
occlient.
Procedure
Create a values file that configures your Data Grid cluster.
For example, the following values file creates a cluster with two nodes:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Install the Data Grid chart and specify your values file.
helm install infinispan openshift-helm-charts/redhat-data-grid --values infinispan-values.yaml
$ helm install infinispan openshift-helm-charts/redhat-data-grid --values infinispan-values.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Use the --set flag to override configuration values for the deployment. For example, to create a cluster with three nodes:
--set deploy.replicas=3
--set deploy.replicas=3
Verification
Watch the pods to ensure all nodes in the Data Grid cluster are created successfully.
oc get pods -w
$ oc get pods -w
1.3. Upgrading Data Grid Helm releases Copy linkLink copied to clipboard!
Modify your Data Grid cluster configuration at runtime by upgrading Helm releases.
Prerequisites
- Deploy the Data Grid chart.
-
Have a
helmclient. -
Have an
occlient.
Procedure
- Modify the values file for your Data Grid deployment as appropriate.
Use the
helmclient to apply your changes, for example:helm upgrade infinispan openshift-helm-charts/redhat-data-grid --values infinispan-values.yaml
$ helm upgrade infinispan openshift-helm-charts/redhat-data-grid --values infinispan-values.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Watch the pods rebuild to ensure all changes are applied to your Data Grid cluster successfully.
oc get pods -w
$ oc get pods -w
1.4. Uninstalling Data Grid Helm releases Copy linkLink copied to clipboard!
Uninstall a release of the Data Grid chart to remove pods and other deployment artifacts.
This procedure shows you how to uninstall a Data Grid deployment on the command line but you can use the OpenShift Web Console instead. Refer to the OpenShift documentation for specific instructions.
Prerequisites
- Deploy the Data Grid chart.
-
Have a
helmclient. -
Have an
occlient.
Procedure
List the installed Data Grid Helm releases.
helm list
$ helm listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
helmclient to uninstall a release and remove the Data Grid cluster:helm uninstall <helm_release_name>
$ helm uninstall <helm_release_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
occlient to remove the generated secret.oc delete secret <helm_release_name>-generated-secret
$ oc delete secret <helm_release_name>-generated-secretCopy to Clipboard Copied! Toggle word wrap Toggle overflow
1.5. Deployment configuration values Copy linkLink copied to clipboard!
Deployment configuration values let you customize Data Grid clusters.
You can also find field and value descriptions in the Data Grid chart README.
| Field | Description | Default value |
|---|---|---|
|
| Specifies the internal Kubernetes cluster domain. |
|
|
| Specifies the number of nodes in your Data Grid cluster, with a pod created for each node. |
|
|
| Passes JVM options to Data Grid Server. | No default value. |
|
| Libraries to be downloaded before server startup. Specify multiple, space-separated artifacts represented as URLs or as Maven coordinates. Archive artifacts in .tar, .tar.gz or .zip formats will be extracted. | No default value. |
|
| Defines whether storage is ephemeral or permanent. |
The default value is |
|
| Defines how much storage is allocated to each Data Grid pod. | 1Gi |
|
|
Specifies the name of a |
No default value. By default, the persistent volume claim uses the storage class that has the |
|
| Defines the CPU limit, in CPU units, for each Data Grid pod. | 500m |
|
| Defines the maximum amount of memory, in bytes, for each Data Grid pod. | 512Mi |
|
| Specifies the maximum CPU requests, in CPU units, for each Data Grid pod. | 500m |
|
| Specifies the maximum memory requests, in bytes, for each Data Grid pod. | 512Mi |
|
| Specifies the name of a secret that creates credentials and configures security authorization. |
No default value. If you create a custom security secret then |
|
| Provides a batch file for the Data Grid command line interface (CLI) to create credentials and configure security authorization at startup. | No default value. |
|
| Specifies the service that exposes Hot Rod and REST endpoints on the network and provides access to your Data Grid cluster, including the Data Grid Console. |
|
|
| Specifies a network port for node port services within the default range of 30000 to 32767. | 0 If you do not specify a port, the platform selects an available one. |
|
| Optionally specifies the hostname where the Route is exposed. | No default value. |
|
| Adds annotations to the service that exposes Data Grid on the network. | No default value. |
|
| Configures Data Grid cluster log categories and levels. | No default value. |
|
| Adds labels to each Data Grid pod that you create. | No default value. |
|
| Adds labels to each service that you create. | No default value. |
|
| Adds labels to all Data Grid resources including pods and services. | No default value. |
|
|
Allows write access to the |
|
|
| Configures the securityContext used by the StatefulSet pods. |
|
|
|
Enable or disable monitoring using |
|
|
| Specifies a name for all Data Grid cluster resources. | Helm Chart release name. |
|
| Data Grid Server configuration. | Data Grid provides default server configuration. For more information about configuring server instances, see Data Grid Server configuration values. |
Chapter 2. Configuring Data Grid Servers Copy linkLink copied to clipboard!
Apply custom Data Grid Server configuration to your deployments.
2.1. Customizing Data Grid Server configuration Copy linkLink copied to clipboard!
Apply custom deploy.infinispan values to Data Grid clusters that configure the Cache Manager and underlying server mechanisms like security realms or Hot Rod and REST endpoints.
You must always provide a complete Data Grid Server configuration when you modify deploy.infinispan values.
Do not modify or remove the default "metrics" configuration if you want to use monitoring capabilities for your Data Grid cluster.
Procedure
Modify Data Grid Server configuration as required:
Specify configuration values for the Cache Manager with
deploy.infinispan.cacheContainerfields.For example, you can create caches at startup with any Data Grid configuration or add cache templates and use them to create caches on demand.
-
Configure security authorization to control user roles and permissions with the
deploy.infinispan.cacheContainer.security.authorizationfield. -
Select one of the default JGroups stacks or configure cluster transport with the
deploy.infinispan.cacheContainer.transportfields. -
Configure Data Grid Server endpoints with the
deploy.infinispan.server.endpointsfields. -
Configure Data Grid Server network interfaces and ports with the
deploy.infinispan.server.interfacesanddeploy.infinispan.server.socketBindingsfields. -
Configure Data Grid Server security mechanisms with the
deploy.infinispan.server.securityfields.
2.2. Data Grid Server configuration values Copy linkLink copied to clipboard!
Data Grid Server configuration values let you customize the Cache Manager and modify server instances that run in OpenShift pods.
Data Grid Server configuration
Data Grid cache configuration
Cache template
Cluster transport
Chapter 3. Configuring authentication and authorization Copy linkLink copied to clipboard!
Control access to Data Grid clusters by adding credentials and assigning roles with different permissions.
3.1. Default credentials Copy linkLink copied to clipboard!
Data Grid adds default credentials in a <helm_release_name>-generated-secret secret.
| Username | Description |
|---|---|
|
|
User that has the |
|
|
Internal user that has the |
3.1.1. Retrieving credentials Copy linkLink copied to clipboard!
Get Data Grid credentials from authentication secrets.
Prerequisites
- Install the Data Grid Helm chart.
-
Have an
occlient.
Procedure
Retrieve default credentials from the
<helm_release_name>-generated-secretor custom credentials from another secret with the following command:oc get secret <helm_release_name>-generated-secret \ -o jsonpath="{.data.identities-batch}" | base64 --decode$ oc get secret <helm_release_name>-generated-secret \ -o jsonpath="{.data.identities-batch}" | base64 --decodeCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2. Adding custom user credentials or credentials store Copy linkLink copied to clipboard!
Create Data Grid user credentials and assign roles that grant security authorization for cluster access.
Procedure
Create credentials by specifying the
user createcommand in thedeploy.security.batchfield.User with implicit authorization
deploy: security: batch: 'user create admin -p changeme'deploy: security: batch: 'user create admin -p changeme'Copy to Clipboard Copied! Toggle word wrap Toggle overflow User with a specific role
deploy: security: batch: 'user create personone -p changeme -g deployer'deploy: security: batch: 'user create personone -p changeme -g deployer'Copy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2.1. User roles and permissions Copy linkLink copied to clipboard!
Data Grid uses role-based access control to authorize users for access to cluster resources and data. For additional security, you should grant Data Grid users with appropriate roles when you add credentials.
| Role | Permissions | Description |
|---|---|---|
|
| ALL | Superuser with all permissions including control of the Cache Manager lifecycle. |
|
| ALL_READ, ALL_WRITE, LISTEN, EXEC, MONITOR, CREATE |
Can create and delete Data Grid resources in addition to |
|
| ALL_READ, ALL_WRITE, LISTEN, EXEC, MONITOR |
Has read and write access to Data Grid resources in addition to |
|
| ALL_READ, MONITOR |
Has read access to Data Grid resources in addition to |
|
| MONITOR | Can view statistics for Data Grid clusters. |
3.2.2. Adding credentials store Copy linkLink copied to clipboard!
Create Data Grid credentials store to avoid exposing passwords in clear text in the server configuration ConfigMap. See Section 4.1, “Enabling TLS encryption” for a use case.
Procedure
Create credentials store by specifying a
credentials addcommand in thedeploy.security.batchfield.Add a password to a store
deploy: security: batch: 'credentials add keystore -c password -p secret --path="credentials.pfx"'deploy: security: batch: 'credentials add keystore -c password -p secret --path="credentials.pfx"'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Credentials store needs then to be added to the server configuration.
Configure a credential store
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2.3. Adding multiple credentials with authentication secrets Copy linkLink copied to clipboard!
Add multiple credentials to Data Grid clusters with authentication secrets.
Prerequisites
-
Have an
occlient.
Procedure
Create an
identities-batchfile that contains the commands to add your credentials.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an authentication secret from your
identities-batchfile.oc apply -f identities-batch.yaml
$ oc apply -f identities-batch.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Specify the authentication secret in the
deploy.security.SecretNamefield.deploy: security: authentication: true secretName: 'connect-secret'deploy: security: authentication: true secretName: 'connect-secret'Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Install or upgrade your Data Grid Helm release.
3.3. Disabling authentication Copy linkLink copied to clipboard!
Allow users to access Data Grid clusters and manipulate data without providing credentials.
Do not disable authentication if endpoints are accessible from outside the OpenShift cluster. You should disable authentication for development environments only.
Procedure
-
Remove the
propertiesRealmfields from the "default" security realm. - Install or upgrade your Data Grid Helm release.
3.4. Disabling security authorization Copy linkLink copied to clipboard!
Allow Data Grid users to perform any operation regardless of their role.
Procedure
Set
nullas the value for thedeploy.infinispan.cacheContainer.securityfield.TipUse the
--set deploy.infinispan.cacheContainer.security=nullargument with thehelmclient.- Install or upgrade your Data Grid Helm release.
Chapter 4. Configuring encryption Copy linkLink copied to clipboard!
Configure encryption for your Data Grid.
4.1. Enabling TLS encryption Copy linkLink copied to clipboard!
Encryption can be independently enabled for endpoint and cluster transport.
Prerequisites
- A secret containing a certificate or a keystore. Endpoint and cluster should use different secrets.
- A credentials keystore containing any password needed to access the keystore. See Adding credentials keystore.
Procedure
Set the secret name in the deploy configuration.
Provide the name of the secret containing the keystore:
deploy: ssl: endpointSecretName: "tls-secret" transportSecretName: "tls-transport-secret"deploy: ssl: endpointSecretName: "tls-secret" transportSecretName: "tls-transport-secret"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enable cluster transport TLS.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Configures the transport stack to use the specified security-realm to provide cluster encryption.
- 2
- Configure the keystore path in the transport realm. The secret is mounted at
/etc/encrypt/transport. - 3 5
- Configures the truststore with the same keystore allowing the nodes to authenticate each other.
- 4
- Alias and password must be provided in case the secret contains a keystore.
Enable endpoint TLS.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 5. Configuring network access Copy linkLink copied to clipboard!
Configure network access for your Data Grid deployment and find out about internal network services.
5.1. Exposing Data Grid clusters on the network Copy linkLink copied to clipboard!
Make Data Grid clusters available on the network so you can access Data Grid Console as well as REST and Hot Rod endpoints. By default, the Data Grid chart exposes deployments through a Route but you can configure it to expose clusters via Load Balancer or Node Port. You can also configure the Data Grid chart so that deployments are not exposed on the network and only available internally to the OpenShift cluster.
Procedure
Specify one of the following for the
deploy.expose.typefield:Expand Option Description RouteExposes Data Grid through a route. This is the default value.
LoadBalancerExposes Data Grid through a load balancer service.
NodePortExposes Data Grid through a node port service.
""(empty value)Disables exposing Data Grid on the network.
-
Optionally specify a hostname with the
deploy.expose.hostfield if you expose Data Grid through a route. -
Optionally specify a port with the
deploy.expose.nodePortfield if you expose Data Grid through a node port service. - Install or upgrade your Data Grid Helm release.
5.2. Retrieving network service details Copy linkLink copied to clipboard!
Get network service details so you can connect to Data Grid clusters.
Prerequisites
- Expose your Data Grid cluster on the network.
-
Have an
occlient.
Procedure
Use one of the following commands to retrieve network service details:
If you expose Data Grid through a route:
oc get routes
$ oc get routesCopy to Clipboard Copied! Toggle word wrap Toggle overflow If you expose Data Grid through a load balancer or node port service:
oc get services
$ oc get servicesCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.3. Network services Copy linkLink copied to clipboard!
The Data Grid chart creates default network services for internal access.
| Service | Port | Protocol | Description |
|---|---|---|---|
|
|
| TCP | Provides access to Data Grid Hot Rod and REST endpoints. |
|
|
| TCP | Provides access to Data Grid metrics. |
|
|
| TCP | Allows Data Grid pods to discover each other and form clusters. |
You can retrieve details about internal network services as follows:
oc get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) infinispan ClusterIP 192.0.2.0 <none> 11222/TCP,11223/TCP infinispan-ping ClusterIP None <none> 8888/TCP
$ oc get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
infinispan ClusterIP 192.0.2.0 <none> 11222/TCP,11223/TCP
infinispan-ping ClusterIP None <none> 8888/TCP
Chapter 6. Connecting to Data Grid clusters Copy linkLink copied to clipboard!
After you configure and deploy Data Grid clusters you can establish remote connections through the Data Grid Console, command line interface (CLI), Hot Rod client, or REST API.
6.1. Accessing Data Grid Console Copy linkLink copied to clipboard!
Access the console to create caches, perform adminstrative operations, and monitor your Data Grid clusters.
Prerequisites
- Expose your Data Grid cluster on the network.
- Retrieve network service details.
Procedure
Access Data Grid Console from any browser at
$SERVICE_HOSTNAME:$PORT.Replace
$SERVICE_HOSTNAME:$PORTwith the hostname and port where Data Grid is available on the network.
6.2. Connecting with the command line interface (CLI) Copy linkLink copied to clipboard!
Use the Data Grid CLI to connect to clusters and create caches, manipulate data, and perform administrative operations.
Prerequisites
- Expose your Data Grid cluster on the network.
- Retrieve network service details.
- Download the native Data Grid CLI distribution from the Data Grid software downloads.
-
Extract the
.ziparchive for the native Data Grid CLI distribution to your host filesystem.
Procedure
Start the Data Grid CLI with the network service as the value for the
-cargument, for example:{native_cli} -c http://cluster-name-myroute.hostname.net/$ {native_cli} -c http://cluster-name-myroute.hostname.net/Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Enter your Data Grid credentials when prompted.
Perform CLI operations as required.
TipPress the tab key or use the
--helpargument to view available options and help text.-
Use the
quitcommand to exit the CLI.
6.3. Connecting Hot Rod clients running on OpenShift Copy linkLink copied to clipboard!
Access remote caches with Hot Rod clients running on the same OpenShift cluster as your Data Grid cluster.
Prerequisites
- Retrieve network service details.
Procedure
Specify the internal network service detail for your Data Grid cluster in the client configuration.
In the following configuration examples,
$SERVICE_HOSTNAME:$PORTdenotes the hostname and port that allows access to your Data Grid cluster.- Specify your credentials so the client can authenticate with Data Grid.
Configure client intelligence, if required.
Hot Rod clients running on OpenShift can use any client intelligence because they can access internal IP addresses for Data Grid pods.
The default intelligence,HASH_DISTRIBUTION_AWARE, is recommended because it allows clients to route requests to primary owners, which improves performance.
Programmatic configuration
Hot Rod client properties
6.3.1. Obtaining IP addresses for all Data Grid pods Copy linkLink copied to clipboard!
You can retrieve a list of all IP addresses for running Data Grid pods.
Connecting Hot Rod clients running on OpenShift is the recommended approach as it ensures the initial connection to one of the available pods.
Procedure
Obtain all the IP addresses for a running Data Grid pods in the following ways:
Using the OpenShift API:
-
Access
${APISERVER}/api/v1/namespaces/<chart-namespace>/endpoints/<helm-release-name>to retrieve theendpointsOpenShift resource associated with the<helm-release-name>service.
-
Access
Using the OpenShift DNS service:
-
Query the DNS service for the name
<helm-release-name>-pingto obtain IPs for all the nodes in a cluster.
-
Query the DNS service for the name
6.4. Connecting Hot Rod clients running outside OpenShift Copy linkLink copied to clipboard!
Access remote caches with Hot Rod clients running externally to the OpenShift cluster where you deploy your Data Grid cluster.
Prerequisites
- Expose your Data Grid cluster on the network.
- Retrieve network service details.
Procedure
Specify the internal network service detail for your Data Grid cluster in the client configuration.
In the following configuration examples,
$SERVICE_HOSTNAME:$PORTdenotes the hostname and port that allows access to your Data Grid cluster.- Specify your credentials so the client can authenticate with Data Grid.
-
Configure clients to use
BASICintelligence.
Programmatic configuration
Hot Rod client properties
6.5. Accessing the REST API Copy linkLink copied to clipboard!
Data Grid provides a RESTful interface that you can interact with using HTTP clients.
Prerequisites
- Expose your Data Grid cluster on the network.
- Retrieve network service details.
Procedure
Access the REST API with any HTTP client at
$SERVICE_HOSTNAME:$PORT/rest/v2.Replace
$SERVICE_HOSTNAME:$PORTwith the hostname and port where Data Grid is available on the network.