Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.
Chapter 4. Configuring Cache Encoding
Data Grid saves your data in a specific format that can be converted on-the-fly when you read and write to and from caches. configure the storage format by specifying a MediaType for keys and values, which describes the format of the data.
Data Grid can also convert data between different storage formats to handle interoperability between different client protocols and when using custom code to process data.
4.1. Cache Encoding and Client Interoperability Link kopierenLink in die Zwischenablage kopiert!
The encoding that you use for your data affects client interoperability and capabilities such as Data Grid Search.
Store data in Protobuf format to use it with… | |
---|---|
Data Grid Console | Yes |
REST clients | Yes |
Java Hot Rod clients | Yes |
Non-Java Hot Rod clients | Yes |
Data Grid Search | Yes |
Custom Java objects | Yes |
Store data in a text-based format to use it with… | |
---|---|
Data Grid Console | Yes |
REST clients | Yes |
Java Hot Rod clients | Yes |
Non-Java Hot Rod clients | Yes |
Data Grid Search | No |
Custom Java objects | No |
Marshalled Java objects are compatible with… | |
---|---|
Data Grid Console | No |
REST clients | Yes |
Java Hot Rod clients | Yes |
Non-Java Hot Rod clients | No |
Data Grid Search | No |
Plain Old Java Objects (POJOs) are not recommended but compatible with… | |
---|---|
Data Grid Console | No |
REST clients | Yes |
Java Hot Rod clients | Yes |
Non-Java Hot Rod clients | No |
Data Grid Search | Yes. However, you must annotate entities to search with POJOs and make your classes available to Data Grid Server. |
Custom Java objects | Yes |
4.1.1. Configuring Cache Encoding for Memcached Clients Link kopierenLink in die Zwischenablage kopiert!
Data Grid Server disables the Memcached endpoint by default. If you enable the Memcached endpoint, you should configure caches with a suitable encoding for Memcached clients.
The Memcached endpoint does not support authentication. For security purposes you should use dedicated caches for Memcached clients. You should not use REST or Hot Rod clients to interact on the same data set as Memcached clients.
Procedure
-
Configure cache encoding to use
text/plain
for keys. Specify any appropriate MediaType, other than
application/x-java- object
, for values.Memcached clients can handle keys as
text/plain
only. Values can be any MediaType that Data Grid stores asbyte[]
, which can be Protobuf, marshalled Java objects, or a text-based format.<encoding> <key media-type="text/plain"/> <value media-type="application/x-protostream"/> </encoding>
<encoding> <key media-type="text/plain"/> <value media-type="application/x-protostream"/> </encoding>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
The Memcached endpoint includes a client-encoding
attribute that converts the encoding of values.
For example, as in the preceding configuration example, you store values encoded as Protobuf. If you want Memcached clients to read and write values as JSON, you can use the following configuration:
<memcached-connector cache="memcachedCache" client-encoding="application/json">
<memcached-connector cache="memcachedCache" client-encoding="application/json">
4.2. Configuring Encoding for Data Grid Caches Link kopierenLink in die Zwischenablage kopiert!
Define the MediaType that Data Grid uses to encode your data when writing and reading to and from the cache.
When you define a MediaType, you specify the format of your data to Data Grid.
If you want to use the Data Grid Console, Hot Rod clients, and REST clients interchangeably, specify the application/x-protostream
MediaType so Data Grid encodes data in Protobuf format.
Procedure
Specify a MediaType for key and values in your Data Grid cache configuration.
-
Declaratively: Set the
encoding
attribute. -
Programmatically: Use the
encoding()
method.
-
Declaratively: Set the
Declarative examples
- Use the same encoding for keys and values:
<local-cache> <encoding media-type="application/x-protostream"/> </local-cache>
<local-cache>
<encoding media-type="application/x-protostream"/>
</local-cache>
- Use a different encoding for keys and values:
Programmatic examples
- Use the same encoding for keys and values:
- Use a different encoding for keys and values:
ConfigurationBuilder cfg = new ConfigurationBuilder(); cfg.encoding().key().mediaType("text/plain"); cfg.encoding().value().mediaType("application/json");
ConfigurationBuilder cfg = new ConfigurationBuilder();
cfg.encoding().key().mediaType("text/plain");
cfg.encoding().value().mediaType("application/json");
4.3. Storing Data in Protobuf Format Link kopierenLink in die Zwischenablage kopiert!
Storing data in the cache as Protobuf encoded entries provides a platform independent configuration that enables you to perform cache operations from any client.
When you configure indexing for Data Grid Search, Data Grid automatically stores keys and values with the application/x-protostream
media type.
Procedure
Specify
application/x-protostream
as the MediaType for keys and values as follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure your clients.
Hot Rod clients must register Protocol Buffers schema definitions that describe entities and client marshallers.
Data Grid converts between application/x-protostream
and application/json
so REST clients only need to send the following headers to read and write JSON formatted data:
-
Accept: application/json
for read operations. -
Content-Type: application/json
for write operations.
4.4. Storing Data in Text-Based Formats Link kopierenLink in die Zwischenablage kopiert!
Configure Data Grid to store data in a text-based format such as text/ plain
, application/json
, or application/xml
.
Procedure
- Specify a text-based storage format as the MediaType for keys and values.
Optionally specify a character set such as
UTF-8
.The following example configures Data Grid to store entries with the
text/plain; charset=UTF-8
format:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure your clients.
Hot Rod clients can use org.infinispan.commons.marshall.StringMarshaller
to handle plain text, JSON, XML, or any other text-based format.
You can also use text-based formats with the ProtoStream marshaller. ProtoStream can handle String
and byte[]
types natively, without the need to create Serialization Contexts and register Protobuf schemas (.proto
files).
REST clients must send the correct headers with requests:
-
Accept: text/plain; charset=UTF-8
for read operations. -
Content-Type: text/plain; charset=UTF-8
for write operations.
4.5. Storing Marshalled Java Objects Link kopierenLink in die Zwischenablage kopiert!
Java Hot Rod clients can handle Java objects that represent entities and perform marshalling to serialize and deserialize objects into byte[]
arrays. C++, C#, and Javascript Hot Rod clients can also handle objects in the respective languages.
If you store entries in the cache as marshalled Java objects, you should configure the cache with the MediaType of the marshalled storage.
Procedure
Specify the MediaType that matches your marshaller implementation.
-
Protostream marshaller: Configure the MediaType as
application/x-protostream
. -
JBoss marshalling: Configure the MediaType as
application/x-jboss-marshalling
. -
Java serialization: Configure the MediaType as
application/x-java-serialized-object
.
-
Protostream marshaller: Configure the MediaType as
- Configure your clients.
Because REST clients are most suitable for handling text formats, you should use primitives such as java.lang.String
for keys. Otherwise, REST clients must handle keys as bytes[]
using a supported binary encoding.
REST clients can read values for cache entries in XML or JSON format.
Equality Considerations
When storing data in binary format, Data Grid uses the WrappedBytes
interface for keys and values. This wrapper class transparently takes care of serialization and deserialization on demand, and internally may have a reference to the object itself being wrapped, or the serialized, byte array representation of the object. This has an effect on the behavior of equality, which is important to note if you implement an equals()
methods on keys.
The equals()
method of the wrapper class either compares binary representations (byte arrays) or delegates to the wrapped object instance’s equals()
method, depending on whether both instances being compared are in serialized or deserialized form at the time of comparison. If one of the instances being compared is in one form and the other in another form, then one instance is either serialized or deserialized.
4.6. Storing Unmarshalled Java Objects Link kopierenLink in die Zwischenablage kopiert!
You can store data as deserialized Plain Old Java Objects (POJO) instead of storing data in a binary format.
Storing POJO instead of binary format is not recommended because it requires Data Grid to serialize data on client read operations and deserialize data on write operations. To handle client interoperability with custom code you should convert data on demand.
Procedure
Specify
application/x-java-object
as the MediaType for keys and values as follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Put class files for all custom objects on the Data Grid server classpath.
Add JAR files that contain custom classes and/or service providers for marshaller implementations in the
server/lib
directory.├── server │ ├── lib │ │ ├── UserObjects.jar │ └── README.txt
├── server │ ├── lib │ │ ├── UserObjects.jar │ └── README.txt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure your clients.
There are no changes required for Hot Rod clients. The only requirement is that the marshaller used in the client is available in the server/lib
directory so Data Grid can de-serialize the objects.
ProtoStream and Java Serialization marshallers are already available on the server.
REST clients must use either JSON or XML so Data Grid can convert to and from Java objects.
4.7. Data Encoding Link kopierenLink in die Zwischenablage kopiert!
Encoding is the data conversion operation done by Data Grid caches before storing data, and when reading back from storage.
4.7.1. Overview Link kopierenLink in die Zwischenablage kopiert!
Encoding allows dealing with a certain data format during API calls (map, listeners, stream, etc) while the format effectively stored is different.
The data conversions are handled by instances of org.infinispan.commons.dataconversion.Encoder :
4.7.2. Default encoders Link kopierenLink in die Zwischenablage kopiert!
Data Grid automatically picks the Encoder depending on the cache configuration. The table below shows which internal Encoder is used for several configurations:
Mode | Configuration | Encoder | Description |
---|---|---|---|
Embedded/Server | Default | IdentityEncoder | Passthrough encoder, no conversion done |
Embedded | StorageType.OFF_HEAP | GlobalMarshallerEncoder | Use the Data Grid internal marshaller to convert to byte[]. May delegate to the configured marshaller in the cache manager. |
Embedded | StorageType.BINARY | BinaryEncoder | Use the Data Grid internal marshaller to convert to byte[], except for primitives and String. |
Server | StorageType.OFF_HEAP | IdentityEncoder | Store byte[]s directly as received by remote clients |
4.7.3. Overriding programmatically Link kopierenLink in die Zwischenablage kopiert!
It is possible to override programmatically the encoding used for both keys and values, by calling the .withEncoding() method variants from AdvancedCache.
Example, consider the following cache configured as OFF_HEAP:
The override can be useful if any operation in the cache does not require decoding, such as counting number of entries, or calculating the size of byte[] of an OFF_HEAP cache.
4.7.4. Defining Custom Encoders Link kopierenLink in die Zwischenablage kopiert!
A custom encoder can be registered in the EncoderRegistry.
Ensure that the registration is done in every node of the cluster, before starting the caches.
Consider a custom encoder used to compress/decompress with gzip:
It can be registered by:
GlobalComponentRegistry registry = cacheManager.getGlobalComponentRegistry(); EncoderRegistry encoderRegistry = registry.getComponent(EncoderRegistry.class); encoderRegistry.registerEncoder(new GzipEncoder());
GlobalComponentRegistry registry = cacheManager.getGlobalComponentRegistry();
EncoderRegistry encoderRegistry = registry.getComponent(EncoderRegistry.class);
encoderRegistry.registerEncoder(new GzipEncoder());
And then be used to write and read data from a cache:
4.8. Transcoders and Data Conversion Link kopierenLink in die Zwischenablage kopiert!
Data Grid uses org.infinispan.commons.dataconversion.Transcoder
to convert data between MediaType formats.
4.8.1. Converting Data on Demand Link kopierenLink in die Zwischenablage kopiert!
You can deploy and run custom code on Data Grid, such as tasks, listeners, and merge policies. Custom code on Data Grid can directly access data but must also interoperate with clients that access the same data through different endpoints. For example, you can create tasks that handle custom objects while Hot Rod clients read and write data in binary format.
In this case, you can configure application/x-protostream
as the cache encoding to store data in binary format then configure your custom code to perform cache operations using a different MediaType.
For example:
Will return the value in JSON format:
{ "_type":"org.infinispan.sample.Person", "name":"John", "surname":"Doe" }
{
"_type":"org.infinispan.sample.Person",
"name":"John",
"surname":"Doe"
}
4.8.2. Installing Transcoders in Embedded Deloyments Link kopierenLink in die Zwischenablage kopiert!
Data Grid Server includes transcoders by default. However, when running Data Grid as a library, you must add the following to your project:
org.infinispan:infinispan-server-core
org.infinispan:infinispan-server-core
4.8.3. Transcoders and Encoders Link kopierenLink in die Zwischenablage kopiert!
Usually there will be none or only one data conversion involved in a cache operation:
- No conversion by default on caches using in embedded or server mode;
- Encoder based conversion for embedded caches without MediaType configured, but using OFF_HEAP or BINARY;
- Transcoder based conversion for caches used in server mode with multiple REST and Hot Rod clients sending and receiving data in different formats. Those caches will have MediaType configured describing the storage.
But it’s possible to have both encoders and transcoders being used simultaneously for advanced use cases.
Consider an example, a cache that stores marshalled objects (with jboss marshaller) content but for security reasons a transparent encryption layer should be added in order to avoid storing "plain" data to an external store. Clients should be able to read and write data in multiple formats.
This can be achieved by configuring the cache with the the MediaType that describes the storage regardless of the encoding layer:
ConfigurationBuilder cfg = new ConfigurationBuilder(); cfg.encoding().key().mediaType("application/x-jboss-marshalling"); cfg.encoding().key().mediaType("application/x-jboss-marshalling");
ConfigurationBuilder cfg = new ConfigurationBuilder();
cfg.encoding().key().mediaType("application/x-jboss-marshalling");
cfg.encoding().key().mediaType("application/x-jboss-marshalling");
The transparent encryption can be added by decorating the cache with a special Encoder that encrypts/decrypts with storing/retrieving, for example:
To make sure all data written to the cache will be stored encrypted, it’s necessary to decorate the cache with the Encoder above and perform all cache operations in this decorated cache:
Cache<?,?> secureStorageCache = cache.getAdvancedCache().withEncoding(Scrambler.class).put(k,v);
Cache<?,?> secureStorageCache = cache.getAdvancedCache().withEncoding(Scrambler.class).put(k,v);
The capability of reading data in multiple formats can be added by decorating the cache with the desired MediaType:
// Obtain a stream of values in XML format from the secure cache secureStorageCache.getAdvancedCache().withMediaType("application/xml","application/xml").values().stream();
// Obtain a stream of values in XML format from the secure cache
secureStorageCache.getAdvancedCache().withMediaType("application/xml","application/xml").values().stream();
Internally, Data Grid will first apply the encoder fromStorage operation to obtain the entries, that will be in "application/x-jboss-marshalling" format and then apply a successive conversion to "application/xml" by using the adequate Transcoder.