Chapter 5. Migrating applications to Data Grid 8


5.1. Marshalling in Data Grid 8

Marshalling capabilities are significantly refactored in Data Grid 8 to isolate internal objects and user objects.

Because Data Grid now handles marshalling of internal classes, you no longer need to handle those internal classes when configuring marshallers with embedded or remote caches.

5.1.1. ProtoStream marshalling

By default, Data Grid 8 uses the ProtoStream API to marshall data as Protocol Buffers, a language-neutral, backwards compatible format.

Protobuf encoding is a schema-defined format that is now a default standard for many applications and allows greater flexibility when transcoding data in comparison with JBoss Marshalling, which was the default in Data Grid 7.

Because the ProtoStream marshaller is based on the Protobuf format, Data Grid can convert to other encodings without first converting to a Java object. When using JBoss Marshalling, it is necessary to convert keys and values to Java objects before converting to any other format.

As part of your migration to Data Grid 8, you should start using ProtoStream marshalling for your Java classes.

From a high-level, to use the ProtoStream marshaller, you generate SerializationContextInitializer implementations with the ProtoStream processor. First, you add @Proto annotations to your Java classes and then use a ProtoStream processor that Data Grid provides to generate serialization contexts that contain:

  • .proto schemas that provide a structured representation of your Java objects as Protobuf message types.
  • Marshaller implementations to encode your Java objects to Protobuf format.

Depending on whether you use embedded or remote caches, Data Grid can automatically register your SerializationContextInitializer implementations.

Marshalling with Data Grid Server

You should use only Protobuf encoding for remote caches in combination with the ProtoStream marshaller for any custom types.

Other marshaller implementations, such as JBoss marshalling, require you to use different cache encodings that are not compatible with the Data Grid CLI, Data Grid Console, or with Ickle queries.

Cache stores and ProtoStream

In Data Grid 7.x, data that you persist to a cache store is not compatible with the ProtoStream marshaller in Data Grid 8. You must use the StoreMigrator utility to migrate data from any Data Grid 7.x cache store to a Data Grid 8 cache store.

5.1.2. Alternative marshaller implementations

Data Grid does provide alternative marshaller implementations to ProtoStream help ease migration from older versions. You should use those alternative marshallers only as an interim solution while you migrate to ProtoStream marshalling.

Note

For new projects Red Hat strongly recommends you use only ProtoStream marshalling to avoid any issues with future upgrades or migrations.

JBoss marshalling

In Data Grid 7, JBoss Marshalling is the default marshaller. In Data Grid 8, ProtoStream marshalling is the default.

Note

You should use JavaSerializationMarshaller instead of JBoss Marshalling if you have a client requirement to use Java serialization.

If you must use JBoss Marshalling as a temporary solution during migration to Data Grid 8, do the following:

Embedded caches

  1. Add the infinispan-jboss-marshalling dependency to your classpath.
  2. Configure Data Grid to use the JBossUserMarshaller, for example:

    <serialization marshaller="org.infinispan.jboss.marshalling.core.JBossUserMarshaller"/>
  3. Add your classes to the list of classes that Data Grid allows for deserialization.

Remote caches

Data Grid Server does not support JBoss Marshalling and the GenericJBossMarshaller is no longer automatically configured if the infinispan-jboss-marshalling module is on the classpath.

You must configure Hot Rod Java clients to use JBoss Marshalling as follows:

  • RemoteCacheManager

    .marshaller("org.infinispan.jboss.marshalling.commons.GenericJBossMarshaller");
  • hotrod-client.properties

    infinispan.client.hotrod.marshaller = GenericJBossMarshaller

Additional resources

5.2. Migrating applications to the AutoProtoSchemaBuilder annotation

Previous versions of Data Grid use the MessageMarshaller interface in the ProtoStream API to configure marshalling.

Both the MessageMarshaller API and the ProtoSchemaBuilder annotation are deprecated as of Data Grid 8.1.1, which corresponds to ProtoStream 4.3.4.

Using the MessageMarshaller interface involves either:

  • Manually creating Protobuf schema.
  • Adding the ProtoSchemaBuilder annotation to Java classes and then generating Protobuf schema.

However, these techniques for configuring ProtoStream marshalling are not as efficient and reliable as the AutoProtoSchemaBuilder annotation, which is available starting with Data Grid 8.1.1. Simply add the AutoProtoSchemaBuilder annotation to your Java classes and to generate SerializationContextInitializer implementations that include Protobuf schema and associated marshallers.

Red Hat recommends that you start using the AutoProtoSchemaBuilder annotation to get the best results from the ProtoStream marshaller.

The following code examples demonstrate how you can migrate applications from the MessageMarshaller API to the AutoProtoSchemaBuilder annotation.

5.2.1. Basic MessageMarshaller implementation

This example contains some fields that use non-default types. The text field has a different order and the fixed32 field conflicts with the generated Protobuf schema type because the code generator uses int type by default.

SimpleEntry.java

public class SimpleEntry {

  private String description;
  private Collection<String> text;
  private int intDefault;
  private Integer fixed32;

  // public Getter, Setter, equals and HashCode methods omitted for brevity
}

SimpleEntryMarshaller.java

import org.infinispan.protostream.MessageMarshaller;

public class SimpleEntryMarshaller implements MessageMarshaller<SimpleEntry> {

  @Override
  public void writeTo(ProtoStreamWriter writer, SimpleEntry testEntry) throws IOException {
    writer.writeString("description", testEntry.getDescription());
    writer.writeInt("intDefault", testEntry.getIntDefault());
    writer.writeInt("fix32", testEntry.getFixed32());
    writer.writeCollection("text", testEntry.getText(), String.class);
  }

  @Override
  public SimpleEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException {
    SimpleEntry x = new SimpleEntry();

    x.setDescription(reader.readString("description"));
    x.setIntDefault(reader.readInt("intDefault"));
    x.setFixed32(reader.readInt("fix32"));
    x.setText(reader.readCollection("text", new LinkedList<String>(), String.class));

    return x;
  }
}

Resulting Protobuf schema

syntax = "proto2";

package example;

message SimpleEntry {
   required string description = 1;
   optional int32 intDefault = 2;
   optional fixed32 fix32 = 3;
   repeated string text = 4;
}

Migrated to the AutoProtoSchemaBuilder annotation

SimpleEntry.java

import org.infinispan.protostream.annotations.ProtoField;
import org.infinispan.protostream.descriptors.Type;

public class SimpleEntry {

  private String description;
  private Collection<String> text;
  private int intDefault;
  private Integer fixed32;

  @ProtoField(number = 1)
  public String getDescription() {...}

  @ProtoField(number = 4, collectionImplementation = LinkedList.class)
  public Collection<String> getText() {...}

  @ProtoField(number = 2, defaultValue = "0")
  public int getIntDefault() {...}

  @ProtoField(number = 3, type = Type.FIXED32)
  public Integer getFixed32() {...}

  // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity
}

SimpleEntryInitializer.java

import org.infinispan.protostream.GeneratedSchema;
import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder;

@AutoProtoSchemaBuilder(includeClasses = { SimpleEntry.class }, schemaFileName = "simple.proto", schemaFilePath = "proto", schemaPackageName = "example")
public interface SimpleEntryInitializer extends GeneratedSchema {
}

Important observations
  • Field 2 is defined as int which the ProtoStream marshaller in previous versions did not check.
  • Because the Java int field is not nullable the ProtoStream processor will fail.
    The Java int field must be required or initialized with a defaultValue.

    From a Java application perspective, the int field is initialized with "0" so you can use defaultValue without any impact as any put operation will set it. Change to required is not a problem from the stored data perspective if always present, but it might cause issues for different clients.

  • Field 3 must be explicitly set to Type.FIXED32 for compatibility.
  • The text collection must be set in the correct order for the resulting Protobuf schema.
Important

The order of the text collection in your Protobuf schema must be the same before and after migration. Likewise, you must set the fixed32 type during migration.

If not, client applications might throw the following exception and fail to start:

Exception ( ISPN004034: Unable to unmarshall bytes )

In other cases, you might observe incomplete or inaccurate results in your cached data.

5.2.2. MessageMarshaller implementation with custom types

This section provides an example migration for a MessageMarshaller implementation that contains fields that ProtoStream does not natively handle.

The following example uses the BigInteger class but applies to any class, even a Data Grid adapter or a custom class.

Note

The BigInteger class is immutable so does not have a no-argument constructor.

CustomTypeEntry.java

import java.math.BigInteger;

public class CustomTypeEntry {

  final String description;
  final BigInteger bigInt;

  // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity
}

CustomTypeEntryMarshaller.java

import org.infinispan.protostream.MessageMarshaller;

public class CustomTypeEntryMarshaller implements MessageMarshaller<CustomTypeEntry> {

  @Override
  public void writeTo(ProtoStreamWriter writer, CustomTypeEntry testEntry) throws IOException {
    writer.writeString("description", testEntry.description);
    writer.writeString("bigInt", testEntry.bigInt.toString());
  }

  @Override
  public CustomTypeEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException {
    final String desc = reader.readString("description");
    final BigInteger bInt = new BigInteger(reader.readString("bigInt"));

    return new CustomTypeEntry(desc, bInt);
  }
}

CustomTypeEntry.proto

syntax = "proto2";

package example;

message CustomTypeEntry {
   required string description = 1;
   required string bigInt = 2;
}

Migrated code with an adapter class

You can use the ProtoAdapter annotation to marshall a CustomType class in a way that generates Protobuf schema that is compatible with Protobuf schema that you created with MessageMarshaller implementations.

With this approach, you:

  • Must not add annotations to the CustomTypeEntry class.
  • Create a CustomTypeEntryAdapter class that uses the @ProtoAdapter annotation to control how the Protobuf schema and marshaller is generated.
  • Include the CustomTypeEntryAdapter class with the @AutoProtoSchemaBuilder annotation.

    Note

    Because the AutoProtoSchemaBuilder annotation does not reference the CustomTypeEntry class, any annotations contained in that class are ignored.

The following example shows the CustomTypeEntryAdapter class that contains ProtoStream annotations for the CustomTypeEntry class:

CustomTypeEntryAdapter.java

import java.math.BigInteger;

import org.infinispan.protostream.annotations.ProtoAdapter;
import org.infinispan.protostream.annotations.ProtoFactory;
import org.infinispan.protostream.annotations.ProtoField;

@ProtoAdapter(CustomTypeEntry.class)
public class CustomTypeEntryAdapter {

  @ProtoFactory
  public CustomTypeEntry create(String description, String bigInt) {
    return new CustomTypeEntry(description, new BigInteger(bigInt));
  }

  @ProtoField(number = 1, required = true)
  public String getDescription(CustomTypeEntry t) {
    return t.description;
  }

  @ProtoField(number = 2, required = true)
  public String getBigInt(CustomTypeEntry t) {
    return t.bigInt.toString();
  }
}

The following example shows the SerializationContextInitializer with AutoProtoSchemaBuilder annotations that reference the CustomTypeEntryAdapter class:

CustomTypeEntryInitializer.java

import org.infinispan.protostream.GeneratedSchema;
import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder;

@AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntryAdapter.class },
   schemaFileName = "custom.proto",
   schemaFilePath = "proto",
   schemaPackageName = "example")
public interface CustomTypeAdapterInitializer extends GeneratedSchema { }

Migrated code without an adapter class

Instead of creating an adapter class, you can add ProtoStream annotations directly to the CustomTypeEntry class.

Important

In this example, the generated Protobuf schema is not compatible with data in caches that was added via the MessageMarshaller interface because the BigInteger is a separate message. Even if the adapter field writes the same String, it is not possible to unmarshall the data.

The following example shows the CustomTypeEntry class that directly contains ProtoStream annotations:

CustomTypeEntry.java

import java.math.BigInteger;

public class CustomTypeEntry {

  @ProtoField(number = 1)
  final String description;
  @ProtoField(number = 2)
  final BigInteger bigInt;

  @ProtoFactory
  public CustomTypeEntry(String description, BigInteger bigInt) {
    this.description = description;
    this.bigInt = bigInt;
  }

  // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity
}

The following example shows the SerializationContextInitializer with AutoProtoSchemaBuilder annotations that reference the CustomTypeEntry and BigIntegerAdapter classes:

CustomTypeEntryInitializer.java

import org.infinispan.protostream.GeneratedSchema;
import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder;
import org.infinispan.protostream.types.java.math.BigIntegerAdapter;

@AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntry.class,
   BigIntegerAdapter.class },
   schemaFileName = "customtype.proto",
   schemaFilePath = "proto",
   schemaPackageName = "example")
public interface CustomTypeInitializer extends GeneratedSchema { }

When you generate the Protobuf schema from the preceding SerializationContextInitializer implementation, it results in the following Protobuf schema:

CustomTypeEntry.proto

syntax = "proto2";

package example;

message BigInteger {
   optional bytes bytes = 1;
}

message CustomTypeEntry {
   optional string description = 1;
   optional BigInteger bigInt = 2;
}

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.