Chapter 5. Migrating applications to Data Grid 8

By default, Data Grid 8 uses the ProtoStream API to marshall data as Protocol Buffers, a language-neutral, backwards compatible format.

Protobuf encoding is a schema-defined format that is now a default standard for many applications and allows greater flexibility when transcoding data in comparison with JBoss Marshalling, which was the default in Data Grid 7.

Because the ProtoStream marshaller is based on the Protobuf format, Data Grid can convert to other encodings without first converting to a Java object. When using JBoss Marshalling, it is necessary to convert keys and values to Java objects before converting to any other format.

As part of your migration to Data Grid 8, you should start using ProtoStream marshalling for your Java classes.

From a high-level, to use the ProtoStream marshaller, you generate SerializationContextInitializer implementations with the ProtoStream processor. First, you add @Proto annotations to your Java classes and then use a ProtoStream processor that Data Grid provides to generate serialization contexts that contain:

Depending on whether you use embedded or remote caches, Data Grid can automatically register your SerializationContextInitializer implementations.

Nested ProtoStream annotations

Data Grid 8.2 upgrades to ProtoStream 4.4.0.Final, which requires migration in some cases.

In previous versions, the ProtoStream API did not correctly nest message types with the result that the messages were generated as top-level only.

If you have Protobuf-encoded entries in persistent cache stores, you should modify your Java classes so that ProtoStream annotations are at top-level. This ensures that the nesting in your persisted messages matches the nesting in your Java classes, otherwise data incompatibility issues can occur.

For example, if you have nested Java classes such as the following:

class OuterClass {
   class InnerClass {
       @ProtoField(1)
       int someMethod() {
       }
   }
}

You should adapt the classes so that InnerClass is no longer a child of OuterClass:

class InnerClass {
    @ProtoField(1)
    int someMethod() {
    }
}

Marshalling with Data Grid Server

You should use only Protobuf encoding for remote caches in combination with the ProtoStream marshaller for any custom types.

Other marshaller implementations, such as JBoss marshalling, require you to use different cache encodings that are not compatible with the Data Grid CLI, Data Grid Console, or with Ickle queries.

Cache stores and ProtoStream

In Data Grid 7.x, data that you persist to a cache store is not compatible with the ProtoStream marshaller in Data Grid 8. You must use the StoreMigrator utility to migrate data from any Data Grid 7.x cache store to a Data Grid 8 cache store.

Data Grid does provide alternative marshaller implementations to ProtoStream help ease migration from older versions. You should use those alternative marshallers only as an interim solution while you migrate to ProtoStream marshalling.

Deserialization Allow List

In keeping with Red Hat’s commitment to using inclusive language the term "white list" has been changed to "allow list" for configuring serialization of your Java classes.

<cache-container>
   <serialization>
      <white-list>
         <class>org.infinispan.test.data.Person</class>
         <regex>org.infinispan.test.data.*</regex>
       </white-list>
   </serialization>
</cache-container>

<cache-container>
   <serialization>
      <allow-list>
         <class>org.infinispan.test.data.Person</class>
         <regex>org.infinispan.test.data.*</regex>
       </allow-list>
   </serialization>
</cache-container>

JBoss marshalling

In Data Grid 7, JBoss Marshalling is the default marshaller. In Data Grid 8, ProtoStream marshalling is the default.

If you must use JBoss Marshalling as a temporary solution during migration to Data Grid 8, do the following:

You must configure Hot Rod Java clients to use JBoss Marshalling as follows:

This example contains some fields that use non-default types. The text field has a different order and the fixed32 field conflicts with the generated Protobuf schema type because the code generator uses int type by default.

public class SimpleEntry {

  private String description;
  private Collection<String> text;
  private int intDefault;
  private Integer fixed32;

  // public Getter, Setter, equals and HashCode methods omitted for brevity
}

import org.infinispan.protostream.MessageMarshaller;

public class SimpleEntryMarshaller implements MessageMarshaller<SimpleEntry> {

  @Override
  public void writeTo(ProtoStreamWriter writer, SimpleEntry testEntry) throws IOException {
    writer.writeString("description", testEntry.getDescription());
    writer.writeInt("intDefault", testEntry.getIntDefault());
    writer.writeInt("fix32", testEntry.getFixed32());
    writer.writeCollection("text", testEntry.getText(), String.class);
  }

  @Override
  public SimpleEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException {
    SimpleEntry x = new SimpleEntry();

    x.setDescription(reader.readString("description"));
    x.setIntDefault(reader.readInt("intDefault"));
    x.setFixed32(reader.readInt("fix32"));
    x.setText(reader.readCollection("text", new LinkedList<String>(), String.class));

    return x;
  }
}

syntax = "proto2";

package example;

message SimpleEntry {
   required string description = 1;
   optional int32 intDefault = 2;
   optional fixed32 fix32 = 3;
   repeated string text = 4;
}

Migrated to the AutoProtoSchemaBuilder annotation

import org.infinispan.protostream.annotations.ProtoField;
import org.infinispan.protostream.descriptors.Type;

public class SimpleEntry {

  private String description;
  private Collection<String> text;
  private int intDefault;
  private Integer fixed32;

  @ProtoField(number = 1)
  public String getDescription() {...}

  @ProtoField(number = 4, collectionImplementation = LinkedList.class)
  public Collection<String> getText() {...}

  @ProtoField(number = 2, defaultValue = "0")
  public int getIntDefault() {...}

  @ProtoField(number = 3, type = Type.FIXED32)
  public Integer getFixed32() {...}

  // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity
}

import org.infinispan.protostream.GeneratedSchema;
import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder;

@AutoProtoSchemaBuilder(includeClasses = { SimpleEntry.class }, schemaFileName = "simple.proto", schemaFilePath = "proto", schemaPackageName = "example")
public interface SimpleEntryInitializer extends GeneratedSchema {
}

Important observations

Exception ( ISPN004034: Unable to unmarshall bytes )

This section provides an example migration for a MessageMarshaller implementation that contains fields that ProtoStream does not natively handle.

The following example uses the BigInteger class but applies to any class, even a Data Grid adapter or a custom class.

import java.math.BigInteger;

public class CustomTypeEntry {

  final String description;
  final BigInteger bigInt;

  // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity
}

import org.infinispan.protostream.MessageMarshaller;

public class CustomTypeEntryMarshaller implements MessageMarshaller<CustomTypeEntry> {

  @Override
  public void writeTo(ProtoStreamWriter writer, CustomTypeEntry testEntry) throws IOException {
    writer.writeString("description", testEntry.description);
    writer.writeString("bigInt", testEntry.bigInt.toString());
  }

  @Override
  public CustomTypeEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException {
    final String desc = reader.readString("description");
    final BigInteger bInt = new BigInteger(reader.readString("bigInt"));

    return new CustomTypeEntry(desc, bInt);
  }
}

syntax = "proto2";

package example;

message CustomTypeEntry {
   required string description = 1;
   required string bigInt = 2;
}

Migrated code with an adapter class

You can use the ProtoAdapter annotation to marshall a CustomType class in a way that generates Protobuf schema that is compatible with Protobuf schema that you created with MessageMarshaller implementations.

With this approach, you:

The following example shows the CustomTypeEntryAdapter class that contains ProtoStream annotations for the CustomTypeEntry class:

import java.math.BigInteger;

import org.infinispan.protostream.annotations.ProtoAdapter;
import org.infinispan.protostream.annotations.ProtoFactory;
import org.infinispan.protostream.annotations.ProtoField;

@ProtoAdapter(CustomTypeEntry.class)
public class CustomTypeEntryAdapter {

  @ProtoFactory
  public CustomTypeEntry create(String description, String bigInt) {
    return new CustomTypeEntry(description, new BigInteger(bigInt));
  }

  @ProtoField(number = 1, required = true)
  public String getDescription(CustomTypeEntry t) {
    return t.description;
  }

  @ProtoField(number = 2, required = true)
  public String getBigInt(CustomTypeEntry t) {
    return t.bigInt.toString();
  }
}

The following example shows the SerializationContextInitializer with AutoProtoSchemaBuilder annotations that reference the CustomTypeEntryAdapter class:

import org.infinispan.protostream.GeneratedSchema;
import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder;

@AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntryAdapter.class },
   schemaFileName = "custom.proto",
   schemaFilePath = "proto",
   schemaPackageName = "example")
public interface CustomTypeAdapterInitializer extends GeneratedSchema { }

Migrated code without an adapter class

Instead of creating an adapter class, you can add ProtoStream annotations directly to the CustomTypeEntry class.

The following example shows the CustomTypeEntry class that directly contains ProtoStream annotations:

import java.math.BigInteger;

public class CustomTypeEntry {

  @ProtoField(number = 1)
  final String description;
  @ProtoField(number = 2)
  final BigInteger bigInt;

  @ProtoFactory
  public CustomTypeEntry(String description, BigInteger bigInt) {
    this.description = description;
    this.bigInt = bigInt;
  }

  // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity
}

The following example shows the SerializationContextInitializer with AutoProtoSchemaBuilder annotations that reference the CustomTypeEntry and BigIntegerAdapter classes:

import org.infinispan.protostream.GeneratedSchema;
import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder;
import org.infinispan.protostream.types.java.math.BigIntegerAdapter;

@AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntry.class,
   BigIntegerAdapter.class },
   schemaFileName = "customtype.proto",
   schemaFilePath = "proto",
   schemaPackageName = "example")
public interface CustomTypeInitializer extends GeneratedSchema { }

When you generate the Protobuf schema from the preceding SerializationContextInitializer implementation, it results in the following Protobuf schema:

syntax = "proto2";

package example;

message BigInteger {
   optional bytes bytes = 1;
}

message CustomTypeEntry {
   optional string description = 1;
   optional BigInteger bigInt = 2;
}