7.3. Protobuf Encoding
The Infinispan Query DSL can be used remotely via the Hot Rod client. In order to do this, protocol buffers are used to adopt a common format for storing cache entries and marshalling them.
For more information, see https://developers.google.com/protocol-buffers/docs/overview
7.3.1. Storing Protobuf Encoded Entities
Protobuf requires data to be structured. This is achieved by declaring Protocol Buffer message types in
.proto files
For example:
Example 7.2. .library.proto
package book_sample;
message Book {
required string title = 1;
required string description = 2;
required int32 publicationYear = 3; // no native Date type available in Protobuf
repeated Author authors = 4;
}
message Author {
required string name = 1;
required string surname = 2;
}
The provided example:
- An entity named
Bookis placed in a package namedbook_sample.package book_sample; message Book { - The entity declares several fields of primitive types and a repeatable field named
authors.required string title = 1; required string description = 2; required int32 publicationYear = 3; // no native Date type available in Protobuf repeated Author authors = 4; } - The
Authormessage instances are embedded in theBookmessage instance.message Author { required string name = 1; required string surname = 2; }
7.3.2. About Protobuf Messages
There are a few important things to note about Protobuf messages:
- Nesting of messages is possible, however the resulting structure is strictly a tree, and never a graph.
- There is no type inheritance.
- Collections are not supported, however arrays can be easily emulated using repeated fields.
7.3.3. Using Protobuf with Hot Rod
Protobuf can be used with JBoss Data Grid's Hot Rod using the following two steps:
- Configure the client to use a dedicated marshaller, in this case, the
ProtoStreamMarshaller. This marshaller uses theProtoStreamlibrary to assist in encoding objects.Important
In order to use theProtoStreamMarshaller, the infinispan-remote-query-client Maven dependency must be added. - Instruct
ProtoStreamlibrary on how to marshall message types by registering per entity marshallers.
Example 7.3. Use the ProtoStreamMarshaller to Encode and Marshall Messages
import org.infinispan.client.hotrod.configuration.ConfigurationBuilder;
import org.infinispan.client.hotrod.marshall.ProtoStreamMarshaller;
import org.infinispan.protostream.SerializationContext;
ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
.host("127.0.0.1").port(11234)
.marshaller(new ProtoStreamMarshaller());
RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build());
SerializationContext srcCtx = ProtoStreamMarshaller.getSerializationContext(remoteCacheManager);
serCtx.registerProtofile("/library.protobin");
serCtx.registerMarshaller(Book.class, new BookMarshaller());
serCtx.registerMarshaller(Author.class, new AuthorMarshaller());
// Book and Author classes omitted for brevity
In the provided example,
- The
SerializationContextis provided by theProtoStreamlibrary. - The
SerializationContext.registerProtofilemethod receives the name of a classpath resource that is a serialized protobuf binary descriptor containing the type declarations. The binary descriptor,.protobin, is compiled with Protobuf's protoc generator tool using the--descriptor_set_out
command line option for thelibrary.protofile. - The
SerializationContextassociated with theRemoteCacheManageris obtained, thenProtoStreamis instructed to marshall the protobuf types.
Note
A
RemoteCacheManager has no SerializationContext associated with it unless it was configured to use ProtoStreamMarshaller.
7.3.4. Registering Per Entity Marshallers
When using the
ProtoStreamMarshaller for remote querying purposes, registration of per entity marshallers for domain model types must be provided by the user for each type or marshalling will fail. When writing marshallers, it is essential that they are stateless and threadsafe, as a single instance of them is being used.
The following example shows how to write a marshaller.
Example 7.4. BookMarshaller.java
import org.infinispan.protostream.MessageMarshaller;
...
public class BookMarshaller implements MessageMarshaller<Book> {
@Override
public String getTypeName() {
return "book_sample.Book";
}
@Override
public Class<? extends Book> getJavaClass() {
return Book.class;
}
@Override
public void writeTo(ProtoStreamWriter writer, Book book) throws IOException {
writer.writeString("title", book.getTitle());
writer.writeString("description", book.getDescription());
writer.writeCollection("authors", book.getAuthors(), Author.class);
}
@Override
public Book readFrom(ProtoStreamReader reader) throws IOException {
String title = reader.readString("title");
String description = reader.readString("description");
int publicationYear = reader.readInt("publicationYear");
Set<Author> authors = reader.readCollection("authors", new HashSet<Author>(), Author.class);
return new Book(title, description, publicationYear, authors);
}
}
Once the client has been set up, reading and writing Java objects to the remote cache. The actual data stored in the cache will be protobuf encoded, provided that marshallers were registered with the remote client for all involved types. In the provided example, this would be
Book and Author.
Objects stored in protobuf format are able to be utilized with compatible clients written in different languages.
7.3.5. Indexing Protobuf Encoded Entities
Once the client has been configured to use Protobuf, indexing can be configured for caches on the server side.
To be able to index the entries, the server must extract relevant metadata from the same binary descriptor as the client, that is, the
.protobin file. The descriptor is supplied to the server by remotely invoking the ProtobufMetadataManager MBean via JMX.
The
ProtobufMetadataManager is a cluster-wide replicated repository of protobuf descriptors. For each running cache manager a separate ProtobufMetadataManager MBean instance exists. The ProtobufMetadataManager ObjectName uses the following pattern:
<jmx domain>:type=RemoteQuery,name=<cache manager name>,component=ProtobufMetadataManager
The following signature is used by the method that registers the Protobuf descriptor file:
void registerProtofile(byte[] descriptorFile)
If indexing is enabled for a cache, all fields of Protobuf-encoded entries are indexed. All Protobuf-encoded entries are searchable, regardless of whether indexing is enabled.
Note
Indexing is recommended for improved performance but is not mandatory when using Remote Queries. Using indexing improves the searching speed but can also reduce the insert/update speeds due to the overhead required to maintain the index.
