Migrating Apicurio Registry deployments


Red Hat build of Apicurio Registry 2.4

Migrate from Apicurio Registry version 1.1 to 2.4

Red Hat build of Apicurio Registry Documentation Team

Abstract

This guide describes the major changes in Apicurio Registry version 2.x, and explains how to migrate an existing Apicurio Registry version 1.1 deployment to version 2.4.

Preface

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.

Providing feedback on Red Hat documentation

We appreciate your feedback on our documentation. To provide feedback, highlight text in a document and add comments.

Prerequisites

  • You are logged in to the Red Hat Customer Portal.
  • In the Red Hat Customer Portal, the document is in the Multi-page HTML viewing format.

Procedure

To provide your feedback, perform the following steps:

  1. Click the Feedback button in the top-right corner of the document to see existing feedback.

    Note

    The feedback feature is enabled only in the Multi-page HTML format.

  2. Highlight the section of the document where you want to provide feedback.
  3. Click the Add Feedback pop-up that appears near the highlighted text.

    A text box appears in the feedback section on the right side of the page.

  4. Enter your feedback in the text box and click Submit.

    A documentation issue is created.

  5. To view the issue, click the issue link in the feedback view.

Chapter 1. Migration from Apicurio Registry 1.1 to 2.x

Apicurio Registry 2.x includes new features with breaking changes from the previous Apicurio Registry 1.1 release. This section describes the major changes between Apicurio Registry 1.1 and version 2.x.

Because of the breaking changes in 2.x, there is no automatic upgrade and a migration process is required. This involves moving the data in your existing registry to a new registry. You must also review your existing registry client applications and update their configuration to meet the new requirements.

When migrating to version 2.x, you must take the following major changes into account:

1.1. New data storage options

The existing registry data storage options in Apicurio Registry 1.1 (streams,jpa, and infinispan) have been replaced with new storage options in version 2.x (sql and kafkasql). These new storage options provide more robust, performant, and maintainable Apicurio Registry deployments.

For details on how to deploy Apicurio Registry 2.x with your chosen storage option, see Installing and deploying Red Hat build of Apicurio Registry on OpenShift.

1.2. New v2 REST API

Apicurio Registry 2.x includes a new REST API with support for artifact groups and improved long term maintainability. Apicurio Registry still supports the original registry v1 REST API and compatibility APIs, for example, Confluent and IBM schema registry APIs. Apicurio Registry now also implements the Schema Registry specification provided in the CNCF Cloud Events specification.

1.3. Refactored Java client libraries

  • The Apicurio Registry Java client classes are available in version 2.x in a different Maven module named apicurio-registry-client.
  • The Kafka client serializer and deserializer (SerDes) classes are available in version 2.x in three different Maven modules, one for each supported data format: Apache Avro, Protobuf, and JSON Schema. You can now use only the module you want without pulling in transitive dependencies that you are not concerned with.

Additional resources

Chapter 2. Migrating Apicurio Registry data

Migrating data to Apicurio Registry 2.x requires exporting all data from your existing 1.1 deployment and importing it into the new 2.x deployment. If you are using Apicurio Registry as a schema registry for Kafka applications, data migration is critical because each Kafka message carries the global identifier for the schema stored in Apicurio Registry. This identifier must be preserved during registry data migration.

Apicurio Registry 2.x provides an API to bulk import/export all data from your registry deployment, which guarantees that all identifiers are kept when importing data from your existing registry. The export API downloads a custom .zip file containing all the information for your artifacts. The import API accepts this .zip and loads all artifacts into the registry in a single batch.

Apicurio Registry 1.1 does not provide an import/export API. However, version 2.x provides an export tool compatible with Apicurio Registry 1.1 to export a .zip, which you can import into your 2.x registry. This tool uses common existing APIs to export all content in the registry. However, it is less performant than the 2.x export API, and should only be used when exporting from a 1.1 registry.

Prerequisites

  • Running Apicurio Registry instances of the 1.1 server you are exporting from and the 2.x server you are importing into.
  • Download the Apicurio Registry exportV1 tool from the Red Hat Customer Portal. This is a Java application that you can run on the command line.

Procedure

  1. Export all the data from Apicurio Registry 1.1 using the exportV1 tool. This generates a registry-export.zip file in your current directory:

    java -jar apicurio-registry-utils-exportV1-2.4.4.SP1-redhat-00001.jar http://old-registry.my-company.com/api
  2. Import the .zip file into Apicurio Registry 2.x using the import API:

    curl -X POST "http://new-registry.my-company.com/apis/registry/v2/admin/import" \
      -H "Accept: application/json" -H "Content-Type: application/zip" \
      --data-binary @registry-export.zip
  3. Check that all the artifacts have been imported into the new 2.x registry by running these commands and comparing the count field:

    curl "http://old-registry.my-company.com/api/search/artifacts"
    curl "http://new-registry.my-company.com/apis/registry/v2/search/artifacts"

Additional resources

Chapter 3. Migrating Apicurio Registry client applications

You must review your existing Apicurio Registry client applications to ensure that the Maven dependencies and Java client configuration meet the new requirements for version 2.x. For example, this includes new Maven dependencies for the Apicurio Registry Java REST client libraries or Kafka client serializer/deserializer (Serdes) libraries. You must also update your Java application configuration with the new registry v2 API path.

Prerequisites

  • Existing Apicurio Registry 1.1 Java client application or Kafka client producer and consumer Java applications with SerDes

Procedure

  1. If you are using the Apicurio Registry Java REST client, you must change the Maven dependencies for the Apicurio Registry Java client libraries, which have been repackaged in version 2.x:

    <dependency>
        <groupId>io.apicurio</groupId>
        <artifactId>apicurio-registry-client</artifactId>
        <version>2.4.4.SP1-redhat-00001</version>
    </dependency>
  2. In your Java client application, you must change your registry URL configuration, from pointing to the existing v1 API path to the new v2 path. For example:

    public class ClientExample {
    
        private static final RegistryRestClient client;
    
         public static void main(String[] args) throws Exception {
            // Create a registry client
            String registryUrl = "https://new-registry.my-company.com/apis/registry/v2";
            RegistryClient client = RegistryClientFactory.create(registryUrl);
        }
    }

    You can find more details on the Java client in the Red Hat build of Apicurio Registry User Guide.

  3. If you are using the Apicurio Registry SerDes libraries, you must change the Maven dependencies, which have been repackaged in version 2.x. In Apicurio Registry 1.1, the SerDes libraries were all provided with only one Maven dependency:

    <dependency>
        <groupId>io.apicurio</groupId>
        <artifactId>apicurio-registry-utils-serde</artifactId>
        <version>1.3.2.Final-redhat-00002</version>
    </dependency>

    In Apicurio Registry 2.x, the SerDes libraries have been split into three Maven dependencies, one for each supported data format: avro, protobuf, and json schema, depending on your use cases:

    <dependency>
        <groupId>io.apicurio</groupId>
        <artifactId>apicurio-registry-serdes-avro-serde</artifactId>
        <version>2.4.4.SP1-redhat-00001</version>
    </dependency>
    <dependency>
        <groupId>io.apicurio</groupId>
        <artifactId>apicurio-registry-serdes-protobuf-serde</artifactId>
        <version>2.4.4.SP1-redhat-00001</version>
    </dependency>
    <dependency>
        <groupId>io.apicurio</groupId>
        <artifactId>apicurio-registry-serdes-jsonschema-serde</artifactId>
        <version>2.4.4.SP1-redhat-00001</version>
    </dependency>
  4. In your Kafka producer and consumer Java applications, you must change your registry URL configuration from pointing to the existing v1 API path to the new v2 path. For example:

    Existing registry v1 API path:

    props.putIfAbsent(AbstractKafkaSerDe.REGISTRY_URL_CONFIG_PARAM, "http://old-registry.my-company.com/api");

    New registry v2 API path:

    props.putIfAbsent(SerdeConfig.REGISTRY_URL, "http://new-registry.my-company.com/apis/registry/v2");

    The refactored SerDes libraries also include other important changes to configuration properties. For more details on SerDes configuration, see the Red Hat build of Apicurio Registry User Guide.

Additional resources

Appendix A. Using your subscription

Apicurio Registry is provided through a software subscription. To manage your subscriptions, access your account at the Red Hat Customer Portal.

Accessing your account

  1. Go to access.redhat.com.
  2. If you do not already have an account, create one.
  3. Log in to your account.

Activating a subscription

  1. Go to access.redhat.com.
  2. Navigate to My Subscriptions.
  3. Navigate to Activate a subscription and enter your 16-digit activation number.

Downloading ZIP and TAR files

To access ZIP or TAR files, use the customer portal to find the relevant files for download. If you are using RPM packages, this step is not required.

  1. Open a browser and log in to the Red Hat Customer Portal Product Downloads page at access.redhat.com/downloads.
  2. Locate the Red Hat Integration entries in the Integration and Automation category.
  3. Select the desired Apicurio Registry product. The Software Downloads page opens.
  4. Click the Download link for your component.

Revised on 2023-11-07 12:56:20 UTC

Legal Notice

Copyright © 2023 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.