Release Notes for Red Hat build of Debezium 2.3.7
What's new in Red Hat build of Debezium
Abstract
Chapter 1. Red Hat build of Debezium Copy linkLink copied to clipboard!
Red Hat build of Debezium is a comprehensive set of integration and event processing technologies for creating, extending, and deploying container-based integration services across hybrid and multicloud environments. Red Hat build of Debezium provides an agile, distributed, and API-centric solution that organizations can use to connect and share data between applications and systems required in a digital world.
Red Hat build of Debezium includes the following capabilities:
- Real-time messaging
- Cross-datacenter message streaming
- API connectivity
- Application connectors
- Enterprise integration patterns
- API management
- Data transformation
- Service composition and orchestration
Chapter 2. Debezium 2.3.7 release notes Copy linkLink copied to clipboard!
Debezium is a distributed change data capture platform that captures row-level changes that occur in database tables and then passes corresponding change event records to Apache Kafka topics. Applications can read these change event streams and access the change events in the order in which they occurred. Debezium is built on Apache Kafka and is deployed and integrated with AMQ Streams on OpenShift Container Platform or on Red Hat Enterprise Linux.
The following topics provide release details:
- Section 2.1, “Debezium database connectors”
- Section 2.2, “Debezium supported configurations”
- Section 2.3, “Debezium installation options”
- Section 2.4, “Upgrading Debezium from version 1.x to 2.3.7”
- Section 2.5, “Debezium 2.3.7 features and improvements”
- Section 2.5.1, “Breaking changes”
- Section 2.5.2, “General availability features”
- Section 2.5.3, “Technology Preview features”
- Section 2.5.4, “Developer Preview features”
- Section 2.6, “Deprecated features”
- Section 2.7, “Known issues”
2.1. Debezium database connectors Copy linkLink copied to clipboard!
Debezium provides connectors based on Kafka Connect for the following common databases:
- Db2
- JDBC Sink connector (Developer preview)
- MongoDB
- MySQL
- Oracle
- PostgreSQL
- SQL Server
2.1.1. Connector usage notes Copy linkLink copied to clipboard!
Db2
-
The Debezium Db2 connector does not include the Db2 JDBC driver (
jcc-11.5.0.0.jar). See the Db2 connector deployment instructions for information about how to deploy the necessary JDBC driver. - The Db2 connector requires the use of the abstract syntax notation (ASN) libraries, which are available as a standard part of Db2 for Linux.
- To use the ASN libraries, you must have a license for IBM InfoSphere Data Replication (IIDR). You do not have to install IIDR to use the libraries.
-
The Debezium Db2 connector does not include the Db2 JDBC driver (
Oracle
-
The Debezium Oracle connector does not include the Oracle JDBC driver (
ojdbc8.jar). See the Oracle connector deployment instructions for information about how to deploy the necessary JDBC driver.
-
The Debezium Oracle connector does not include the Oracle JDBC driver (
PostgreSQL
-
To use the Debezium PostgreSQL connector you must use the
pgoutputlogical decoding output plug-in, which is the default for PostgreSQL versions 10 and later.
-
To use the Debezium PostgreSQL connector you must use the
2.2. Debezium supported configurations Copy linkLink copied to clipboard!
For information about Debezium supported configurations, including information about supported database versions, see the Debezium 2.3.7 Supported configurations page.
2.2.1. AMQ Streams API version Copy linkLink copied to clipboard!
Debezium runs on AMQ Streams 2.5.
AMQ Streams supports the v1beta2 API version, which updates the schemas of the AMQ Streams custom resources. Older API versions are deprecated. After you upgrade to AMQ Streams 1.7, but before you upgrade to AMQ Streams 1.8 or later, you must upgrade your custom resources to use API version v1beta2.
For more information, see the Debezium User Guide.
2.3. Debezium installation options Copy linkLink copied to clipboard!
You can install Debezium with AMQ Streams on OpenShift or on Red Hat Enterprise Linux:
2.4. Upgrading Debezium from version 1.x to 2.3.7 Copy linkLink copied to clipboard!
The current version of Debezium includes changes that require you to follow specific steps when you upgrade from versions earlier than 2.1.4.
2.4.1. Upgrading connectors to Debezium 2.3.7 Copy linkLink copied to clipboard!
The Debezium 2.3.7 release includes some changes that are not backward-compatible with versions of Debezium earlier than 2.x. As a result, to preserve data and ensure continued operation when you upgrade from Debezium 1.x versions to 2.3.7, you must complete some manual steps during the upgrade process.
One significant change is that the names of some connector parameters have changed. To accommodate these changes, review the configuration properties updates in the 2023.Q2 release notes, and note the properties that are present in your connector configuration. Before you upgrade, edit the configuration of each Debezium connector to add the new names of any changed properties. Before you upgrade, edit the configuration of any 1.x connector instances so that both the old and new property names are present. After the upgrade, you can remove the old configuration options.
Prerequisites
- Debezium is now compatible with Kafka versions up to 3.5.0.
- The Java 11 runtime is required and must be available prior to upgrading. AMQ Streams 2.5 supports Java 11. Use Java 11 when developing new applications. Java 11 enables use of recent language updates, such as the new String API and changes in predicate support, while also benefiting from Java performance improvements. Java 8 is no longer supported in AMQ Streams 2.5.
- Check the backward-incompatible changes in the current list of breaking changes and in the 2023.Q2 release notes.
- Verify that your environment complies with the Debezium 2.3.7 Supported Configurations.
Procedure
- From the OpenShift console, review the Kafka Connector YAML to identify the connector configuration that are no longer valid in Debezium 2.3.7. Refer to the 2023.Q2 release notes for details.
- Edit the configuration to add the 2.x equivalents for the properties that you identify in Step 1, so that both the old and new property names are present. Set the values of the new properties to the values that were previously specified for the old properties.
- From the OpenShift console, stop Kafka Connect to gracefully stop the connector.
- From the OpenShift console, edit the Kafka Connect image YAML to reference the Debezium 2.3.7.Final version of the connector zip file.
- From the OpenShift console, edit the Kafka Connector YAML to remove any configuration options that are no longer valid for your connector.
- Adjust your application’s storage dependencies, as needed, depending on the storage module implementation dependencies in your code. For more information, see Changes to Debezium storage in the 2023.Q2 release notes.
- Restart Kafka Connect to start the connector. After you restart the connector, it continues to process events from the point where it stopped before the upgrade. Change events records that the connector wrote to Kafka before the upgrade are not modified.
2.5. Debezium 2.3.7 features and improvements Copy linkLink copied to clipboard!
Debezium 2.3.7 provides fixes for the following problems that were found in Debezium 2.3.4, which became available in 2023 Q4.
- DBZ-6942 Connector frequently misses commit operations
- DBZ-6957 Remove spaces from Signal and Notification MBean’s ObjectName
- DBZ-6963 Missing events from Oracle 19c
- DBZ-6978 ParsingException (MySQL/MariaDB): User specification with whitespace
- DBZ-6986 RecordsStreamProducerIT#shouldReceiveChangesForInfinityNumericWithInfinity fails on Postgres < 14
- DBZ-6999 ALTER TABLE fails when adding multiple columns to JDBC sink target
- DBZ-7006 Single quote replication and loss of data
- DBZ-7018 Oracle connector: Payload size over 76020 bytes is truncated
- DBZ-7158 Log sequence check should treat each redo thread independently
- DBZ-7263 DBZ-6517 HotFix: CloudEventsConverter throws static error on Kafka Connect 3.5+
- DBZ-7345 Oracle connector is occasionally unable to find SCN
For information about changes that were introduced in the Debezium 2.3.4 release, see the release notes.
2.5.1. Breaking changes Copy linkLink copied to clipboard!
There are no breaking changes in this release. For a list of breaking changes in the previous release, see the 2.3.4 Release Notes.
2.5.2. General availability features Copy linkLink copied to clipboard!
There are no new General Availability features in this release.
2.5.3. Technology Preview features Copy linkLink copied to clipboard!
Debezium 2.3.7 includes the following Technology Preview features that were introduced in earlier releases:
- MongoDB sharded cluster improvements
- MongoDB incremental snapshots for multi-replica and sharded clusters
- Parallel initial snapshots
- CloudEvents converter
- Custom-developed converters
-
Use of the
BLOB,CLOB, andNCLOBdata types with the Oracle connector
Technology Preview features are not supported with Red Hat production service-level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend implementing any Technology Preview features in production environments. Technology Preview features provide early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. For more information about support scope, see Technology Preview Features Support Scope.
For more information about Technology Preview features that were carried over from the previous release, see the 2.3.4 Release Notes
2.5.4. Developer Preview features Copy linkLink copied to clipboard!
Debezium 2.3.7 includes the following Developer Preview features that were introduced in earlier releases:
- JDBC Sink connector
- MySQL connector: parallel snapshots
- Oracle connector: Ingesting changes from a logical standby
- PostgreSQL connector: Exactly-once delivery
Developer Preview features are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Developer Preview software for production or business-critical workloads. Developer Preview software provides early access to upcoming product software in advance of its possible inclusion in a Red Hat product offering. Customers can use this software to test functionality and provide feedback during the development process. This software might not have any documentation, is subject to change or removal at any time, and has received limited testing. Red Hat might provide ways to submit feedback on Developer Preview software without an associated SLA.
For more information about the support scope of Red Hat Developer Preview software, see Developer Preview Support Scope.
For more information about Developer Preview features that were carried over from the previous release, see the 2.3.4 Release Notes
2.6. Deprecated features Copy linkLink copied to clipboard!
There are no new deprecated features in this release. For information about features that were deprecated in the previous release, see the 2.3.4 Release Notes
2.7. Known issues Copy linkLink copied to clipboard!
The following known issue affects Debezium 2.3.7: