Chapter 5. Configuring Data Grid to Marshall Java Objects
Marshalling converts Java objects into binary format so they can be transferred over the wire or stored to disk. The reverse process, unmarshalling, transforms data from binary format into Java objects.
Data Grid performs marshalling and unmarshalling to:
- Send data to other Data Grid nodes in a cluster.
- Store data in persistent cache stores.
- Store data in binary format to provide deserialization capabilities.
5.1. Supported Types Copy linkLink copied to clipboard!
Data Grid uses a ProtoStream API to encode and decode Java objects into Protocol Buffers (Protobuf); a language-neutral, backwards compatible format.
ProtoStream can handle the following types for keys and values, as well as the unboxed equivalents in the case of primitive types:
-
byte[] -
Byte -
String -
Integer -
Long -
Double -
Float -
Boolean -
Short -
Character -
java.util.Date -
java.time.Instant
5.2. Marshalling User Types with ProtoStream Copy linkLink copied to clipboard!
User types are Java objects that Data Grid does not support out of the box. To marshall user types, you implement the SerializationContextInitializer interface to describe your Java objects so that the ProtoStream library can encode them to Protobuf format and Data Grid can transmit and store them.
5.2.1. Generating Serialization Context Initializers Copy linkLink copied to clipboard!
A ProtoStream SerializationContext contains Protobuf type definitions for custom Java objects, loaded from Protobuf schemas, and the accompanying marshallers for those objects.
Data Grid provides a protostream-processor artifact that processes Java annotations in your classes at compile time. The processor generates Protobuf schemas, marshallers, and a concrete implementation of the SerializationContextInitializer interface that you can use to initialize a ProtoStream SerializationContext.
By default, implementation names are the annotated class name with an "Impl" suffix.
Procedure
Add the
protostream-processordependency to yourpom.xml.<dependencyManagement> <dependencies> <dependency> <groupId>org.infinispan</groupId> <artifactId>infinispan-bom</artifactId> <version>${version.infinispan}</version> <type>pom</type> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>org.infinispan.protostream</groupId> <artifactId>protostream-processor</artifactId> <!-- This dependency should be declared in the "provided" scope or made "optional" because it is a compile-only dependency and is not required at runtime. Transitive propagation of this dependency should be also be avoided. --> <scope>provided</scope> </dependency> </dependencies>Annotate the Java objects that you want to marshall with
@ProtoFieldand@ProtoFactory.Author.java
import org.infinispan.protostream.annotations.ProtoFactory; import org.infinispan.protostream.annotations.ProtoField; public class Author { @ProtoField(number = 1) final String name; @ProtoField(number = 2) final String surname; @ProtoFactory Author(String name, String surname) { this.name = name; this.surname = surname; } // public Getter methods omitted for brevity }Book.java
import org.infinispan.protostream.annotations.ProtoFactory; import org.infinispan.protostream.annotations.ProtoField; ... public class Book { @ProtoField(number = 1) final String title; @ProtoField(number = 2) final String description; @ProtoField(number = 3, defaultValue = "0") final int publicationYear; @ProtoField(number = 4, collectionImplementation = ArrayList.class) final List<Author> authors; @ProtoFactory Book(String title, String description, int publicationYear, List<Author> authors) { this.title = title; this.description = description; this.publicationYear = publicationYear; this.authors = authors; } // public Getter methods omitted for brevity }Define an interface that extends
SerializationContextInitializerand is annotated with@AutoProtoSchemaBuilder.@AutoProtoSchemaBuilder( includeClasses = { Book.class, Author.class, }, schemaFileName = "library.proto",1 schemaFilePath = "proto/",2 schemaPackageName = "book_sample") interface LibraryInitializer extends SerializationContextInitializer { }
Next steps
Add the SerializationContextInitializer implementation to your Data Grid configuration to register it.
5.2.2. Manually Implementing Serialization Context Initializers Copy linkLink copied to clipboard!
In some cases you might need to manually define Protobuf schemas and implement ProtoStream marshallers. For example, if you cannot modify Java object classes to add annotations.
Procedure
Create a Protobuf schema,
.protofile, that provides a structured representations of the Java objects to marshall.package book_sample; message Book { optional string title = 1; optional string description = 2; optional int32 publicationYear = 3; // no native Date type available in Protobuf repeated Author authors = 4; } message Author { optional string name = 1; optional string surname = 2; }The preceding
.library.protofile defines an entity (Protobuf message type) named Book that is contained in the book_sample package. Book declares several fields of primitive types and an array (Protobuf repeatable field) named authors, which is the Author message type.- You can nest messages but the resulting structure is strictly a tree, never a graph.
- Type inheritance is not possible.
- Collections are not supported but you can emulate arrays with repeated fields.
Use the
org.infinispan.protostream.MessageMarshallerinterface to implement marshallers for your classes.NoteThe
MessageMarshallerinterface is now deprecated.The next version of Data Grid provides an alternative implementation that lets you create an adaptor class that uses the
@ProtoAdaptorannotation for any external, third-party Java object classes.BookMarshaller.java
import org.infinispan.protostream.MessageMarshaller; public class BookMarshaller implements MessageMarshaller<Book> { @Override public String getTypeName() { return "book_sample.Book"; } @Override public Class<? extends Book> getJavaClass() { return Book.class; } @Override public void writeTo(MessageMarshaller.ProtoStreamWriter writer, Book book) throws IOException { writer.writeString("title", book.getTitle()); writer.writeString("description", book.getDescription()); writer.writeInt("publicationYear", book.getPublicationYear()); writer.writeCollection("authors", book.getAuthors(), Author.class); } @Override public Book readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { String title = reader.readString("title"); String description = reader.readString("description"); int publicationYear = reader.readInt("publicationYear"); List<Author> authors = reader.readCollection("authors", new ArrayList<>(), Author.class); return new Book(title, description, publicationYear, authors); } }AuthorMarshaller.java
import org.infinispan.protostream.MessageMarshaller; public class AuthorMarshaller implements MessageMarshaller<Author> { @Override public String getTypeName() { return "book_sample.Author"; } @Override public Class<? extends Author> getJavaClass() { return Author.class; } @Override public void writeTo(MessageMarshaller.ProtoStreamWriter writer, Author author) throws IOException { writer.writeString("name", author.getName()); writer.writeString("surname", author.getSurname()); } @Override public Author readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { String name = reader.readString("name"); String surname = reader.readString("surname"); return new Author(name, surname); } }Create a
SerializationContextInitializerimplementation that registers the.protoschema and the ProtoStream marshaller implementations with aSerializationContext.ManualSerializationContextInitializer.java
import org.infinispan.protostream.FileDescriptorSource; import org.infinispan.protostream.SerializationContext; import org.infinispan.protostream.SerializationContextInitializer; ... public class ManualSerializationContextInitializer implements SerializationContextInitializer { @Override public String getProtoFileName() { return "library.proto"; } @Override public String getProtoFile() throws UncheckedIOException { // Assumes that the file is located in a Jar's resources, we must provide the path to the library.proto file return FileDescriptorSource.getResourceAsString(getClass(), "/" + getProtoFileName()); } @Override public void registerSchema(SerializationContext serCtx) { serCtx.registerProtoFiles(FileDescriptorSource.fromString(getProtoFileName(), getProtoFile())); } @Override public void registerMarshallers(SerializationContext serCtx) { serCtx.registerMarshaller(new AuthorMarshaller()); serCtx.registerMarshaller(new BookMarshaller()); } }
Next steps
Add the SerializationContextInitializer implementation to your Data Grid configuration to register it.
5.2.3. Registering Serialization Context Initializers Copy linkLink copied to clipboard!
Declare SerializationContextInitializer implementations in your Data Grid configuration to register them.
Procedure
-
Manually register
SerializationContextInitializerimplementations either programmatically or declaratively, as in the following examples:
Programmatic configuration
GlobalConfigurationBuilder builder = new GlobalConfigurationBuilder();
builder.serialization()
.addContextInitializers(new LibraryInitializerImpl(), new SCIImpl());
Declarative configuration
<serialization>
<context-initializer class="org.infinispan.example.LibraryInitializerImpl"/>
<context-initializer class="org.infinispan.example.another.SCIImpl"/>
</serialization>
5.3. Configuring Alternative Marshaller Implementations Copy linkLink copied to clipboard!
Data Grid provides Marshaller implementations that you can use instead of ProtoStream. You can also configure Data Grid to use custom marshaller implementations.
5.3.1. Using JBoss Marshalling Copy linkLink copied to clipboard!
JBoss Marshalling is a serialization-based marshalling library and was the default marshaller in previous Data Grid versions.
- You should not use serialization-based marshalling with Data Grid. Instead you should use Protostream, which is a high-performance binary wire format that ensures backwards compatibility.
-
JBoss Marshalling and the
AdvancedExternalizerinterface are deprecated and will be removed in a future release. However, Data Grid ignoresAdvancedExternalizerimplementations when persisting data unless you use JBoss Marshalling.
Procedure
-
Add the
infinispan-jboss-marshallingdependency to your classpath. -
Configure Data Grid to use the
GenericJBossMarshaller. Add your Java classes to the deserialization whitelist.
Programmatically:
GlobalConfigurationBuilder builder = new GlobalConfigurationBuilder(); builder.serialization() .marshaller(new GenericJBossMarshaller()) .whiteList() .addRegexps("org.infinispan.example.", "org.infinispan.concrete.SomeClass");Declaratively:
<serialization marshaller="org.infinispan.jboss.marshalling.commons.GenericJBossMarshaller"> <white-list> <class>org.infinispan.concrete.SomeClass</class> <regex>org.infinispan.example.*</regex> <white-list> </serialization>
5.3.2. Using Java Serialization Copy linkLink copied to clipboard!
You can use Java serialization with Data Grid to marshall your objects, but only if your Java objects implement the Java Serializable interface.
Procedure
-
Configure Data Grid to use
JavaSerializationMarshalleras the marshaller. Add your Java classes to the deserialization whitelist.
Programmatically:
GlobalConfigurationBuilder builder = new GlobalConfigurationBuilder(); builder.serialization() .marshaller(new JavaSerializationMarshaller()) .whiteList() .addRegexps("org.infinispan.example.", "org.infinispan.concrete.SomeClass");Declaratively:
<serialization marshaller="org.infinispan.commons.marshall.JavaSerializationMarshaller"> <white-list> <class>org.infinispan.concrete.SomeClass</class> <regex>org.infinispan.example.*</regex> </white-list> </serialization>
5.3.3. Using the Kryo Marshaller Copy linkLink copied to clipboard!
Data Grid provides a marshalling implementation that uses Kryo libraries.
Prerequisites for Data Grid Servers
To use Kryo marshalling with Data Grid servers, add a JAR that includes the runtime class files for the Kryo marshalling implementation as follows:
-
Copy
infinispan-marshaller-kryo-bundle.jarfrom the Data Grid Maven repository. -
Add the JAR file to the
server/libdirectory in your Data Grid server installation directory.
Prerequisites for Data Grid Library Mode
To use Kryo marshalling with Data Grid as an embedded library in your application, do the following:
Add the
infinispan-marshaller-kryodependency to yourpom.xml.<dependency> <groupId>org.infinispan</groupId> <artifactId>infinispan-marshaller-kryo</artifactId> <version>${version.infinispan}</version> </dependency>Specify the
org.infinispan.marshaller.kryo.KryoMarshallerclass as the marshaller.GlobalConfigurationBuilder builder = new GlobalConfigurationBuilder(); builder.serialization() .marshaller(new org.infinispan.marshaller.kryo.KryoMarshaller());
Procedure
-
Implement a service provider for the
SerializerRegistryService.javainterface. Place all serializer registrations in the
register(Kryo)method; where serializers are registered with the suppliedKryoobject using the Kryo API, for example:kryo.register(ExampleObject.class, new ExampleObjectSerializer())Specify the full path of implementing classes in your deployment JAR file within:
META-INF/services/org/infinispan/marshaller/kryo/SerializerRegistryService
5.3.4. Using the Protostuff Marshaller Copy linkLink copied to clipboard!
Data Grid provides a marshalling implementation that uses Protostuff libraries.
Prerequisites for Data Grid Servers
To use Protostuff marshalling with Data Grid servers, add a JAR that includes the runtime class files for the Protostuff marshalling implementation as follows:
-
Copy
infinispan-marshaller-protostuff-bundle.jarfrom the Data Grid Maven repository. -
Add the JAR file to the
server/libdirectory in your Data Grid server installation directory.
Prerequisites for Data Grid Library Mode
To use Protostuff marshalling with Data Grid as an embedded library in your application, do the following:
Add the
infinispan-marshaller-protostuffdependency to yourpom.xml.<dependency> <groupId>org.infinispan</groupId> <artifactId>infinispan-marshaller-protostuff</artifactId> <version>${version.infinispan}</version> </dependency>Specify the
org.infinispan.marshaller.protostuff.ProtostuffMarshallerclass as the marshaller.GlobalConfigurationBuilder builder = new GlobalConfigurationBuilder(); builder.serialization() .marshaller(new org.infinispan.marshaller.protostuff.ProtostuffMarshaller());
Procedure
Do one of the following to register custom Protostuff schemas for object marshalling:
Call the
register()method.RuntimeSchema.register(ExampleObject.class, new ExampleObjectSchema());Implement a service provider for the
SerializerRegistryService.javainterface that places all schema registrations in theregister()method.You should then specify the full path of implementing classes in your deployment JAR file within:
META-INF/services/org/infinispan/marshaller/protostuff/SchemaRegistryService
5.3.5. Using Custom Marshallers Copy linkLink copied to clipboard!
Data Grid provides a Marshaller interface that you can implement for custom marshallers.
Procedure
-
Implement the
Marshallerinterface. - Configure Data Grid to use your marshaller.
Add your Java classes to the deserialization whitelist.
Programmatically:
GlobalConfigurationBuilder builder = new GlobalConfigurationBuilder(); builder.serialization() .marshaller(new org.infinispan.example.marshall.CustomMarshaller()) .whiteList().addRegexp("org.infinispan.example.*");Declaratively:
<serialization marshaller="org.infinispan.example.marshall.CustomMarshaller"> <white-list> <class>org.infinispan.concrete.SomeClass</class> <regex>org.infinispan.example.*</regex> </white-list> </serialization>
Custom marshaller implementations can access a configured white list via the initialize() method, which is called during startup.
5.3.6. Adding Java Classes to Deserialization White Lists Copy linkLink copied to clipboard!
Data Grid does not allow deserialization of arbritrary Java classes for security reasons, which applies to JSON, XML, and marshalled byte[] content.
You must add Java classes to a deserialization whitelist, either using system properties or specifying them in the Data Grid configuration.
System properties
// Specify a comma-separated list of fully qualified class names
-Dinfinispan.deserialization.whitelist.classes=java.time.Instant,com.myclass.Entity
// Specify a regular expression to match classes
-Dinfinispan.deserialization.whitelist.regexps=.*
Declarative
<cache-container>
<serialization version="1.0" marshaller="org.infinispan.marshall.TestObjectStreamMarshaller">
<white-list>
<class>org.infinispan.test.data.Person</class>
<regex>org.infinispan.test.data.*</regex>
</white-list>
</serialization>
</cache-container>
Java classes that you add to the deserialization whitelist apply to the Data Grid CacheContainer and can be deserialized by all caches that the CacheContainer controls.