7.3. Protobuf Encoding
The Infinispan Query DSL can be used remotely via the Hot Rod client. In order to do this, protocol buffers are used to adopt a common format for storing cache entries and marshalling them.
For more information, see https://developers.google.com/protocol-buffers/docs/overview
7.3.1. Storing Protobuf Encoded Entities
Protobuf requires data to be structured. This is achieved by declaring Protocol Buffer message types in
.proto
files
For example:
Example 7.2. .library.proto
package book_sample; message Book { required string title = 1; required string description = 2; required int32 publicationYear = 3; // no native Date type available in Protobuf repeated Author authors = 4; } message Author { required string name = 1; required string surname = 2; }
The provided example:
- An entity named
Book
is placed in a package namedbook_sample
.package book_sample; message Book {
- The entity declares several fields of primitive types and a repeatable field named
authors
.required string title = 1; required string description = 2; required int32 publicationYear = 3; // no native Date type available in Protobuf repeated Author authors = 4; }
- The
Author
message instances are embedded in theBook
message instance.message Author { required string name = 1; required string surname = 2; }
7.3.2. About Protobuf Messages
There are a few important things to note about Protobuf messages:
- Nesting of messages is possible, however the resulting structure is strictly a tree, and never a graph.
- There is no type inheritance.
- Collections are not supported, however arrays can be easily emulated using repeated fields.
7.3.3. Using Protobuf with Hot Rod
Protobuf can be used with JBoss Data Grid's Hot Rod using the following two steps:
- Configure the client to use a dedicated marshaller, in this case, the
ProtoStreamMarshaller
. This marshaller uses theProtoStream
library to assist in encoding objects.Important
In order to use theProtoStreamMarshaller
, the infinispan-remote-query-client Maven dependency must be added. - Instruct
ProtoStream
library on how to marshall message types by registering per entity marshallers.
Example 7.3. Use the ProtoStreamMarshaller
to Encode and Marshall Messages
import org.infinispan.client.hotrod.configuration.ConfigurationBuilder; import org.infinispan.client.hotrod.marshall.ProtoStreamMarshaller; import org.infinispan.protostream.SerializationContext; ConfigurationBuilder clientBuilder = new ConfigurationBuilder(); clientBuilder.addServer() .host("127.0.0.1").port(11234) .marshaller(new ProtoStreamMarshaller()); RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build()); SerializationContext srcCtx = ProtoStreamMarshaller.getSerializationContext(remoteCacheManager); serCtx.registerProtofile("/library.protobin"); serCtx.registerMarshaller(Book.class, new BookMarshaller()); serCtx.registerMarshaller(Author.class, new AuthorMarshaller()); // Book and Author classes omitted for brevity
In the provided example,
- The
SerializationContext
is provided by theProtoStream
library. - The
SerializationContext.registerProtofile
method receives the name of a classpath resource that is a serialized protobuf binary descriptor containing the type declarations. The binary descriptor,.protobin
, is compiled with Protobuf's protoc generator tool using the--descriptor_set_out
command line option for thelibrary.proto
file. - The
SerializationContext
associated with theRemoteCacheManager
is obtained, thenProtoStream
is instructed to marshall the protobuf types.
Note
A
RemoteCacheManager
has no SerializationContext
associated with it unless it was configured to use ProtoStreamMarshaller
.
7.3.4. Registering Per Entity Marshallers
When using the
ProtoStreamMarshaller
for remote querying purposes, registration of per entity marshallers for domain model types must be provided by the user for each type or marshalling will fail. When writing marshallers, it is essential that they are stateless and threadsafe, as a single instance of them is being used.
The following example shows how to write a marshaller.
Example 7.4. BookMarshaller.java
import org.infinispan.protostream.MessageMarshaller; ... public class BookMarshaller implements MessageMarshaller<Book> { @Override public String getTypeName() { return "book_sample.Book"; } @Override public Class<? extends Book> getJavaClass() { return Book.class; } @Override public void writeTo(ProtoStreamWriter writer, Book book) throws IOException { writer.writeString("title", book.getTitle()); writer.writeString("description", book.getDescription()); writer.writeCollection("authors", book.getAuthors(), Author.class); } @Override public Book readFrom(ProtoStreamReader reader) throws IOException { String title = reader.readString("title"); String description = reader.readString("description"); int publicationYear = reader.readInt("publicationYear"); Set<Author> authors = reader.readCollection("authors", new HashSet<Author>(), Author.class); return new Book(title, description, publicationYear, authors); } }
Once the client has been set up, reading and writing Java objects to the remote cache. The actual data stored in the cache will be protobuf encoded, provided that marshallers were registered with the remote client for all involved types. In the provided example, this would be
Book
and Author
.
Objects stored in protobuf format are able to be utilized with compatible clients written in different languages.
7.3.5. Indexing Protobuf Encoded Entities
Once the client has been configured to use Protobuf, indexing can be configured for caches on the server side.
To be able to index the entries, the server must extract relevant metadata from the same binary descriptor as the client, that is, the
.protobin
file. The descriptor is supplied to the server by remotely invoking the ProtobufMetadataManager
MBean via JMX.
The
ProtobufMetadataManager
is a cluster-wide replicated repository of protobuf descriptors. For each running cache manager a separate ProtobufMetadataManager
MBean instance exists. The ProtobufMetadataManager
ObjectName uses the following pattern:
<jmx domain>:type=RemoteQuery,name=<cache manager name>,component=ProtobufMetadataManager
The following signature is used by the method that registers the Protobuf descriptor file:
void registerProtofile(byte[] descriptorFile)
If indexing is enabled for a cache, all fields of Protobuf-encoded entries are indexed. All Protobuf-encoded entries are searchable, regardless of whether indexing is enabled.
Note
Indexing is recommended for improved performance but is not mandatory when using Remote Queries. Using indexing improves the searching speed but can also reduce the insert/update speeds due to the overhead required to maintain the index.