Chapter 3. Migrating data between cache stores
Data Grid provides a Java utility for migrating persistent data between cache stores.
In the case of upgrading Data Grid, functional differences between major versions do not allow backwards compatibility between cache stores. You can use StoreMigrator
to convert your data so that it is compatible with the target version.
For example, upgrading to Data Grid 8.0 changes the default marshaller to Protostream. In previous Data Grid versions, cache stores use a binary format that is not compatible with the changes to marshalling. This means that Data Grid 8.0 cannot read from cache stores with previous Data Grid versions.
In other cases Data Grid versions deprecate or remove cache store implementations, such as JDBC Mixed and Binary stores. You can use StoreMigrator
in these cases to convert to different cache store implementations.
3.1. Cache store migrator
Data Grid provides the StoreMigrator.java
utility that recreates data for the latest Data Grid cache store implementations.
StoreMigrator
takes a cache store from a previous version of Data Grid as source and uses a cache store implementation as target.
When you run StoreMigrator
, it creates the target cache with the cache store type that you define using the EmbeddedCacheManager
interface. StoreMigrator
then loads entries from the source store into memory and then puts them into the target cache.
StoreMigrator
also lets you migrate data from one type of cache store to another. For example, you can migrate from a JDBC string-based cache store to a RocksDB cache store.
StoreMigrator
cannot migrate data from segmented cache stores to:
- Non-segmented cache store.
- Segmented cache stores that have a different number of segments.
3.2. Configuring the cache store migrator
Use the migrator.properties
file to configure properties for source and target cache stores.
Procedure
-
Create a
migrator.properties
file. Configure properties for source and target cache store using the
migrator.properties
file.Add the
source.
prefix to all configuration properties for the source cache store.Example source cache store
source.type=SOFT_INDEX_FILE_STORE source.cache_name=myCache source.location=/path/to/source/sifs source.version=<version>
ImportantFor migrating data from segmented cache stores, you must also configure the number of segments using the
source.segment_count
property. The number of segments must matchclustering.hash.numSegments
in your Data Grid configuration. If the number of segments for a cache store does not match the number of segments for the corresponding cache, Data Grid cannot read data from the cache store.Add the
target.
prefix to all configuration properties for the target cache store.Example target cache store
target.type=SINGLE_FILE_STORE target.cache_name=myCache target.location=/path/to/target/sfs.dat
3.2.1. Configuration properties for the cache store migrator
Configure source and target cache stores in a StoreMigrator
properties.
Property | Description | Required/Optional |
---|---|---|
| Specifies the type of cache store for a source or target cache store.
| Required |
Property | Description | Example Value | Required/Optional |
---|---|---|---|
| The name of the cache that you want to back up. |
| Required |
| The number of segments for target cache stores that can use segmentation.
The number of segments must match |
| Optional |
Property | Description | Required/Optional |
---|---|---|
| Specifies the dialect of the underlying database. | Required |
|
Specifies the marshaller version for source cache stores.
*
*
*
*
*
* | Required for source stores only. |
| Specifies a custom marshaller class. | Required if using custom marshallers. |
|
Specifies a comma-separated list of custom | Optional |
| Specifies the JDBC connection URL. | Required |
| Specifies the class of the JDBC driver. | Required |
| Specifies a database username. | Required |
| Specifies a password for the database username. | Required |
| Disables database upsert. | Optional |
| Specifies if table indexes are created. | Optional |
| Specifies additional prefixes for the table name. | Optional |
| Specifies the column name. | Required |
| Specifies the column type. | Required |
|
Specifies the | Optional |
To migrate from Binary cache stores in older Data Grid versions, change table.string.*
to table.binary.\*
in the following properties:
-
source.table.binary.table_name_prefix
-
source.table.binary.<id\|data\|timestamp>.name
-
source.table.binary.<id\|data\|timestamp>.type
# Example configuration for migrating to a JDBC String-Based cache store target.type=STRING target.cache_name=myCache target.dialect=POSTGRES target.marshaller.class=org.example.CustomMarshaller target.marshaller.externalizers=25:Externalizer1,org.example.Externalizer2 target.connection_pool.connection_url=jdbc:postgresql:postgres target.connection_pool.driver_class=org.postrgesql.Driver target.connection_pool.username=postgres target.connection_pool.password=redhat target.db.disable_upsert=false target.db.disable_indexing=false target.table.string.table_name_prefix=tablePrefix target.table.string.id.name=id_column target.table.string.data.name=datum_column target.table.string.timestamp.name=timestamp_column target.table.string.id.type=VARCHAR target.table.string.data.type=bytea target.table.string.timestamp.type=BIGINT target.key_to_string_mapper=org.infinispan.persistence.keymappers. DefaultTwoWayKey2StringMapper
Property | Description | Required/Optional |
---|---|---|
| Sets the database directory. | Required |
| Specifies the compression type to use. | Optional |
# Example configuration for migrating from a RocksDB cache store. source.type=ROCKSDB source.cache_name=myCache source.location=/path/to/rocksdb/database source.compression=SNAPPY
Property | Description | Required/Optional |
---|---|---|
|
Sets the directory that contains the cache store | Required |
# Example configuration for migrating to a Single File cache store. target.type=SINGLE_FILE_STORE target.cache_name=myCache target.location=/path/to/sfs.dat
Property | Description | Value |
---|---|---|
Required/Optional |
| Sets the database directory. |
Required |
| Sets the database index directory. |
# Example configuration for migrating to a Soft-Index File cache store. target.type=SOFT_INDEX_FILE_STORE target.cache_name=myCache target.location=path/to/sifs/database target.location=path/to/sifs/index
3.3. Migrating Data Grid cache stores
You can use the StoreMigrator
to migrate data between cache stores with different Data Grid versions or to migrate data from one type of cache store to another.
Prerequisites
-
Have a
infinispan-tools.jar
. -
Have the source and target cache store configured in the
migrator.properties
file.
Procedure
If you built the
infinispan-tools.jar
from the source code, do the following:-
Add
infinispan-tools.jar
to your classpath. - Add dependencies for your source and target databases, such as JDBC drivers to your classpath.
-
Specify
migrator.properties
file as an argument forStoreMigrator
.
-
Add
If you pulled
infinispan-tools.jar
from the Maven repository, run the following command:mvn exec:java