Chapter 8. Marshalling Custom Java Objects with ProtoStream
Data Grid uses a ProtoStream API to encode and decode Java objects into Protocol Buffers (Protobuf); a language-neutral, backwards compatible format.
8.1. Protobuf Schemas
Protocol Buffers, Protobuf, schemas provide structured representations of your Java objects.
You define Protobuf message types .proto
schema files as in the following example:
package book_sample; message Book { optional string title = 1; optional string description = 2; optional int32 publicationYear = 3; // no native Date type available in Protobuf repeated Author authors = 4; } message Author { optional string name = 1; optional string surname = 2; }
The preceding .library.proto
file defines an entity (Protobuf message type) named Book that is contained in the book_sample package. Book declares several fields of primitive types and an array (Protobuf repeatable field) named authors, which is the Author message type.
Protobuf Messages
- You can nest messages but the resulting structure is strictly a tree, never a graph.
- Type inheritance is not possible.
- Collections are not supported but you can emulate arrays with repeated fields.
Reference
8.2. ProtoStream Serialization Contexts
A ProtoStream SerializationContext
contains Protobuf type definitions for custom Java objects, loaded from .proto
schema files, and the accompanying Marshallers for the objects.
The SerializationContextInitializer
interface registers Java objects and marshallers so that the ProtoStream library can encode your custom objects to Protobuf format, which then enables Data Grid to transmit and store your data.
8.3. ProtoStream Types
ProtoStream can handle the following types, as well as the unboxed equivalents in the case of primitive types, without any additional configuration:
-
String
-
Integer
-
Long
-
Double
-
Float
-
Boolean
-
byte[]
-
Byte
-
Short
-
Character
-
java.util.Date
-
java.time.Instant
To marshall any other Java objects, you must generate, or manually create, SerializationContextInitializer
implementations that register .proto
schemas and marshallers with a SerializationContext
.
8.4. Generating Serialization Context Initializers
Data Grid provides an protostream-processor
artifact that can generate .proto
schemas and SerializationContextInitializer
implementations from annotated Java classes.
Procedure
Add the
protostream-processor
dependency to yourpom.xml
.<dependencyManagement> <dependencies> <dependency> <groupId>org.infinispan</groupId> <artifactId>infinispan-bom</artifactId> <version>${version.infinispan}</version> <type>pom</type> </dependency> <dependency> <groupId>org.infinispan.protostream</groupId> <artifactId>protostream-processor</artifactId> <scope>provided</scope> </dependency> </dependencies> </dependencyManagement>
Annotate the Java objects that you want to marshall with
@ProtoField
and@ProtoFactory
.Book.java
import org.infinispan.protostream.annotations.ProtoFactory; import org.infinispan.protostream.annotations.ProtoField; ... public class Book { @ProtoField(number = 1) final String title; @ProtoField(number = 2) final String description; @ProtoField(number = 3, defaultValue = "0") final int publicationYear; @ProtoField(number = 4, collectionImplementation = ArrayList.class) final List<Author> authors; @ProtoFactory Book(String title, String description, int publicationYear, List<Author> authors) { this.title = title; this.description = description; this.publicationYear = publicationYear; this.authors = authors; } // public Getter methods omitted for brevity }
Author.java
import org.infinispan.protostream.annotations.ProtoFactory; import org.infinispan.protostream.annotations.ProtoField; public class Author { @ProtoField(number = 1) final String name; @ProtoField(number = 2) final String surname; @ProtoFactory Author(String name, String surname) { this.name = name; this.surname = surname; } // public Getter methods omitted for brevity }
Define an interface that extends
SerializationContextInitializer
and is annotated with@AutoProtoSchemaBuilder
.@AutoProtoSchemaBuilder( includeClasses = { Book.class, Author.class, }, schemaFileName = "library.proto", 1 schemaFilePath = "proto/", 2 schemaPackageName = "book_sample") interface LibraryInitializer extends SerializationContextInitializer { }
During compile-time, protostream-processor
generates a concrete implementation of the interface that you can use to initialize a ProtoStream SerializationContext
. By default, implementation names are the annotated class name with an "Impl" suffix.
Examples
The following are examples of a generated schema file and implementation:
target/classes/proto/library.proto
// File name: library.proto // Generated from : org.infinispan.commons.marshall.LibraryInitializer syntax = "proto2"; package book_sample; message Book { optional string title = 1; optional string description = 2; optional int32 publicationYear = 3 [default = 0]; repeated Author authors = 4; } message Author { optional string name = 1; optional string surname = 2; }
LibraryInitializerImpl.java
/* Generated by org.infinispan.protostream.annotations.impl.processor.AutoProtoSchemaBuilderAnnotationProcessor for class org.infinispan.commons.marshall.LibraryInitializer annotated with @org.infinispan.protostream.annotations.AutoProtoSchemaBuilder(dependsOn=, service=false, autoImportClasses=false, excludeClasses=, includeClasses=org.infinispan.commons.marshall.Book,org.infinispan.commons.marshall.Author, basePackages={}, value={}, schemaPackageName="book_sample", schemaFilePath="proto/", schemaFileName="library.proto", className="") */ package org.infinispan.commons.marshall; /** * WARNING: Generated code! */ @javax.annotation.Generated(value = "org.infinispan.protostream.annotations.impl.processor.AutoProtoSchemaBuilderAnnotationProcessor", comments = "Please do not edit this file!") @org.infinispan.protostream.annotations.impl.OriginatingClasses({ "org.infinispan.commons.marshall.Author", "org.infinispan.commons.marshall.Book" }) /*@org.infinispan.protostream.annotations.AutoProtoSchemaBuilder( className = "LibraryInitializerImpl", schemaFileName = "library.proto", schemaFilePath = "proto/", schemaPackageName = "book_sample", service = false, autoImportClasses = false, classes = { org.infinispan.commons.marshall.Author.class, org.infinispan.commons.marshall.Book.class } )*/ public class LibraryInitializerImpl implements org.infinispan.commons.marshall.LibraryInitializer { @Override public String getProtoFileName() { return "library.proto"; } @Override public String getProtoFile() { return org.infinispan.protostream.FileDescriptorSource.getResourceAsString(getClass(), "/proto/library.proto"); } @Override public void registerSchema(org.infinispan.protostream.SerializationContext serCtx) { serCtx.registerProtoFiles(org.infinispan.protostream.FileDescriptorSource.fromString(getProtoFileName(), getProtoFile())); } @Override public void registerMarshallers(org.infinispan.protostream.SerializationContext serCtx) { serCtx.registerMarshaller(new org.infinispan.commons.marshall.Book$___Marshaller_cdc76a682a43643e6e1d7e43ba6d1ef6f794949a45e1a8bc961046cda44c9a85()); serCtx.registerMarshaller(new org.infinispan.commons.marshall.Author$___Marshaller_9b67e1c1ecea213b4207541b411fb9af2ae6f658610d2a4ca9126484d57786d1()); } }
8.5. Manually Implementing Serialization Context Initializers
In some cases you might need to manually define .proto
schema files and implement ProtoStream marshallers. For example, if you cannot modify Java object classes to add annotations.
Procedure
Create a
.proto
schema with Protobuf messages.package book_sample; message Book { optional string title = 1; optional string description = 2; optional int32 publicationYear = 3; // no native Date type available in Protobuf repeated Author authors = 4; } message Author { optional string name = 1; optional string surname = 2; }
Use the
org.infinispan.protostream.MessageMarshaller
interface to implement marshallers for your classes.BookMarshaller.java
import org.infinispan.protostream.MessageMarshaller; public class BookMarshaller implements MessageMarshaller<Book> { @Override public String getTypeName() { return "book_sample.Book"; } @Override public Class<? extends Book> getJavaClass() { return Book.class; } @Override public void writeTo(MessageMarshaller.ProtoStreamWriter writer, Book book) throws IOException { writer.writeString("title", book.getTitle()); writer.writeString("description", book.getDescription()); writer.writeInt("publicationYear", book.getPublicationYear()); writer.writeCollection("authors", book.getAuthors(), Author.class); } @Override public Book readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { String title = reader.readString("title"); String description = reader.readString("description"); int publicationYear = reader.readInt("publicationYear"); List<Author> authors = reader.readCollection("authors", new ArrayList<>(), Author.class); return new Book(title, description, publicationYear, authors); } }
AuthorMarshaller.java
import org.infinispan.protostream.MessageMarshaller; public class AuthorMarshaller implements MessageMarshaller<Author> { @Override public String getTypeName() { return "book_sample.Author"; } @Override public Class<? extends Author> getJavaClass() { return Author.class; } @Override public void writeTo(MessageMarshaller.ProtoStreamWriter writer, Author author) throws IOException { writer.writeString("name", author.getName()); writer.writeString("surname", author.getSurname()); } @Override public Author readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { String name = reader.readString("name"); String surname = reader.readString("surname"); return new Author(name, surname); } }
Create a
SerializationContextInitializer
implementation that registers the.proto
schema and the ProtoStream marshaller implementations with aSerializationContext
.ManualSerializationContextInitializer.java
import org.infinispan.protostream.FileDescriptorSource; import org.infinispan.protostream.SerializationContext; import org.infinispan.protostream.SerializationContextInitializer; ... public class ManualSerializationContextInitializer implements SerializationContextInitializer { @Override public String getProtoFileName() { return "library.proto"; } @Override public String getProtoFile() throws UncheckedIOException { // Assumes that the file is located in a Jar's resources, we must provide the path to the library.proto file return FileDescriptorSource.getResourceAsString(getClass(), "/" + getProtoFileName()); } @Override public void registerSchema(SerializationContext serCtx) { serCtx.registerProtoFiles(FileDescriptorSource.fromString(getProtoFileName(), getProtoFile())); } @Override public void registerMarshallers(SerializationContext serCtx) { serCtx.registerMarshaller(new AuthorMarshaller()); serCtx.registerMarshaller(new BookMarshaller()); } }