Migrating Apicurio Registry deployments
Migrate from Apicurio Registry version 1.1 to 2.4
Abstract
Preface
Making open source more inclusive
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.
Providing feedback on Red Hat documentation
We appreciate your feedback on our documentation. To provide feedback, highlight text in a document and add comments.
Prerequisites
- You are logged in to the Red Hat Customer Portal.
- In the Red Hat Customer Portal, the document is in the Multi-page HTML viewing format.
Procedure
To provide your feedback, perform the following steps:
Click the Feedback button in the top-right corner of the document to see existing feedback.
NoteThe feedback feature is enabled only in the Multi-page HTML format.
- Highlight the section of the document where you want to provide feedback.
Click the Add Feedback pop-up that appears near the highlighted text.
A text box appears in the feedback section on the right side of the page.
Enter your feedback in the text box and click Submit.
A documentation issue is created.
- To view the issue, click the issue link in the feedback view.
Chapter 1. Migration from Apicurio Registry 1.1 to 2.x
Apicurio Registry 2.x includes new features with breaking changes from the previous Apicurio Registry 1.1 release. This section describes the major changes between Apicurio Registry 1.1 and version 2.x.
Because of the breaking changes in 2.x, there is no automatic upgrade and a migration process is required. This involves moving the data in your existing registry to a new registry. You must also review your existing registry client applications and update their configuration to meet the new requirements.
When migrating to version 2.x, you must take the following major changes into account:
1.1. New data storage options
The existing registry data storage options in Apicurio Registry 1.1 (streams
,jpa
, and infinispan
) have been replaced with new storage options in version 2.x (sql
and kafkasql
). These new storage options provide more robust, performant, and maintainable Apicurio Registry deployments.
For details on how to deploy Apicurio Registry 2.x with your chosen storage option, see Installing and deploying Red Hat build of Apicurio Registry on OpenShift.
1.2. New v2 REST API
Apicurio Registry 2.x includes a new REST API with support for artifact groups and improved long term maintainability. Apicurio Registry still supports the original registry v1 REST API and compatibility APIs, for example, Confluent and IBM schema registry APIs. Apicurio Registry now also implements the Schema Registry specification provided in the CNCF Cloud Events specification.
1.3. Refactored Java client libraries
-
The Apicurio Registry Java client classes are available in version 2.x in a different Maven module named
apicurio-registry-client
. - The Kafka client serializer and deserializer (SerDes) classes are available in version 2.x in three different Maven modules, one for each supported data format: Apache Avro, Protobuf, and JSON Schema. You can now use only the module you want without pulling in transitive dependencies that you are not concerned with.
Additional resources
- For more details on the v2 REST API, see the Registry REST API documentation.
Chapter 2. Migrating Apicurio Registry data
Migrating data to Apicurio Registry 2.x requires exporting all data from your existing 1.1 deployment and importing it into the new 2.x deployment. If you are using Apicurio Registry as a schema registry for Kafka applications, data migration is critical because each Kafka message carries the global identifier for the schema stored in Apicurio Registry. This identifier must be preserved during registry data migration.
Apicurio Registry 2.x provides an API to bulk import/export all data from your registry deployment, which guarantees that all identifiers are kept when importing data from your existing registry. The export API downloads a custom .zip
file containing all the information for your artifacts. The import API accepts this .zip
and loads all artifacts into the registry in a single batch.
Apicurio Registry 1.1 does not provide an import/export API. However, version 2.x provides an export tool compatible with Apicurio Registry 1.1 to export a .zip
, which you can import into your 2.x registry. This tool uses common existing APIs to export all content in the registry. However, it is less performant than the 2.x export API, and should only be used when exporting from a 1.1 registry.
Prerequisites
- Running Apicurio Registry instances of the 1.1 server you are exporting from and the 2.x server you are importing into.
- Download the Apicurio Registry exportV1 tool from the Red Hat Customer Portal. This is a Java application that you can run on the command line.
Procedure
Export all the data from Apicurio Registry 1.1 using the
exportV1
tool. This generates aregistry-export.zip
file in your current directory:java -jar apicurio-registry-utils-exportV1-2.4.4.SP1-redhat-00001.jar http://old-registry.my-company.com/api
Import the
.zip
file into Apicurio Registry 2.x using the import API:curl -X POST "http://new-registry.my-company.com/apis/registry/v2/admin/import" \ -H "Accept: application/json" -H "Content-Type: application/zip" \ --data-binary @registry-export.zip
Check that all the artifacts have been imported into the new 2.x registry by running these commands and comparing the count field:
curl "http://old-registry.my-company.com/api/search/artifacts"
curl "http://new-registry.my-company.com/apis/registry/v2/search/artifacts"
Additional resources
- For more details on the import/export REST API, see the Red Hat build of Apicurio Registry User Guide.
- For more details on the export tool for migrating from version 1.x to 2.x, see the Apicurio Registry export utility for 1.x versions.
Chapter 3. Migrating Apicurio Registry client applications
You must review your existing Apicurio Registry client applications to ensure that the Maven dependencies and Java client configuration meet the new requirements for version 2.x. For example, this includes new Maven dependencies for the Apicurio Registry Java REST client libraries or Kafka client serializer/deserializer (Serdes) libraries. You must also update your Java application configuration with the new registry v2 API path.
Prerequisites
- Existing Apicurio Registry 1.1 Java client application or Kafka client producer and consumer Java applications with SerDes
Procedure
If you are using the Apicurio Registry Java REST client, you must change the Maven dependencies for the Apicurio Registry Java client libraries, which have been repackaged in version 2.x:
<dependency> <groupId>io.apicurio</groupId> <artifactId>apicurio-registry-client</artifactId> <version>2.4.4.SP1-redhat-00001</version> </dependency>
In your Java client application, you must change your registry URL configuration, from pointing to the existing v1 API path to the new v2 path. For example:
public class ClientExample { private static final RegistryRestClient client; public static void main(String[] args) throws Exception { // Create a registry client String registryUrl = "https://new-registry.my-company.com/apis/registry/v2"; RegistryClient client = RegistryClientFactory.create(registryUrl); } }
You can find more details on the Java client in the Red Hat build of Apicurio Registry User Guide.
If you are using the Apicurio Registry SerDes libraries, you must change the Maven dependencies, which have been repackaged in version 2.x. In Apicurio Registry 1.1, the SerDes libraries were all provided with only one Maven dependency:
<dependency> <groupId>io.apicurio</groupId> <artifactId>apicurio-registry-utils-serde</artifactId> <version>1.3.2.Final-redhat-00002</version> </dependency>
In Apicurio Registry 2.x, the SerDes libraries have been split into three Maven dependencies, one for each supported data format:
avro
,protobuf
, andjson schema
, depending on your use cases:<dependency> <groupId>io.apicurio</groupId> <artifactId>apicurio-registry-serdes-avro-serde</artifactId> <version>2.4.4.SP1-redhat-00001</version> </dependency> <dependency> <groupId>io.apicurio</groupId> <artifactId>apicurio-registry-serdes-protobuf-serde</artifactId> <version>2.4.4.SP1-redhat-00001</version> </dependency> <dependency> <groupId>io.apicurio</groupId> <artifactId>apicurio-registry-serdes-jsonschema-serde</artifactId> <version>2.4.4.SP1-redhat-00001</version> </dependency>
In your Kafka producer and consumer Java applications, you must change your registry URL configuration from pointing to the existing v1 API path to the new v2 path. For example:
Existing registry v1 API path:
props.putIfAbsent(AbstractKafkaSerDe.REGISTRY_URL_CONFIG_PARAM, "http://old-registry.my-company.com/api");
New registry v2 API path:
props.putIfAbsent(SerdeConfig.REGISTRY_URL, "http://new-registry.my-company.com/apis/registry/v2");
The refactored SerDes libraries also include other important changes to configuration properties. For more details on SerDes configuration, see the Red Hat build of Apicurio Registry User Guide.
Additional resources
- For detailed configuration examples, see the Apicurio Registry example applications.
Appendix A. Using your subscription
Apicurio Registry is provided through a software subscription. To manage your subscriptions, access your account at the Red Hat Customer Portal.
Accessing your account
- Go to access.redhat.com.
- If you do not already have an account, create one.
- Log in to your account.
Activating a subscription
- Go to access.redhat.com.
- Navigate to My Subscriptions.
- Navigate to Activate a subscription and enter your 16-digit activation number.
Downloading ZIP and TAR files
To access ZIP or TAR files, use the customer portal to find the relevant files for download. If you are using RPM packages, this step is not required.
- Open a browser and log in to the Red Hat Customer Portal Product Downloads page at access.redhat.com/downloads.
- Locate the Red Hat Integration entries in the Integration and Automation category.
- Select the desired Apicurio Registry product. The Software Downloads page opens.
- Click the Download link for your component.
Revised on 2023-11-07 12:56:20 UTC