Release Notes for Red Hat build of Debezium 3.4.3

The schema_only_recovery and schema_only snapshot modes have been removed. In place of those options, use the recovery and no_data snapshot modes.

DBZ-8171

Debezium is now built and tested using Apache Kafka 4.1.0 and 4.1.1

For details about compatibility between Debezium, Streams for Apache Kafka, and Kafka, check the Debezium supported configurations document.

DBZ-9460, DBZ-9709

Beginning with this release, Debezium is transitioning away from using Jira to report bugs and request features or enhancements. The existng Jira instance will remain available for a period, in read-only mode, but it is scheduled to be retired in the near future.

You should now submit all bug reports and enhancement requests, including feedback on the documentation, to the Debezium Issues page at https://github.com/debezium/dbz/issues.

Due to problems that the Db2 connector experienced when attempting to validate offsets, the when_needed snapshot mode is removed in this release. If the snapshot.mode in your connector configuration is set to use this option, edit the configuration to specify a different mode. For more information, see Known problems and limitations.

DBZ-9470

PostgreSQL 13 is no longer tested or supported.

DBZ-9376

Support for exactly-once semantics for all core connectors (MariaDB, MongoDB, MySQL, Oracle, PostgreSQL, SQL Server), ensuring events are delivered and written to Kafka topics only once.

DBZ-9177

New extended.headers.enabled configuration option can be set to false to disable OpenLineage context headers, and help to minimize event payloads.

DBZ-9248

New guardrail.collections.max and guardrail.collections.limit.action configuration options prevent OutOfMemoryError issues in environments that process hundreds to thousands of tables.

DBZ-9427

The ConvertTimezone SMT now supports converting ts_ms, ts_us, and ts_ns fields from the source information block.

DBZ-9372

The connector now automatically retries SQL exceptions during change processing, providing resilience for transient issues like lock conflicts in multi-task environments.

DBZ-7772

Debezium source connectors support for various geometry and geography column types. To serialize these types, source connectors use custom Debezium logical types that are based on the Struct semantic type in Kafka. This is a complex field type that Debezium uses to encode both Well-Known Binary (WKB) and Extended Well-Known Binary) (EWKB) byte arrays along with the spatial reference identifier or SRID.

When you pair the Debezium JDBC sink with a source connector, the JDBC sink connector can convert geometry column values to the format required by the target database, if it supports such column types. This feature requires no special configuration.

For more information about how the connector maps geometry types from source databases to sink databases, see the JDBC connector documentation in the {NameDebeziumUserGuide].

DBZ-9508

MariaDB 11.7 introduced the VECTOR(n) data type, so that the relational database can store values as if it were a vector database. With the vector data type, the database can store and search for Vector values that are generated by an AI model.

In this release, the Debezium provides support for processing the MariaDB VECTOR data type in the MariaDB source connector and in the JDBC sink connector when writing to a MariaDB target database For more information about how Debezium handles Vector types, see the Debezium MariaDB connector documentation.

DBZ-8582

Debezium processes PostgreSQL events that use the tsvector data type. The Debezium JDBC sink connector can write TSVECTOR values to PostgreSQL targets. For non-PostgreSQL sink databases, the connector writes to character-based columns.

For more information see, Documentation for PostgreSQL connector pgvector types.

DBZ-8121

When an incremental snapshot was in progress for a SQL Server connector, if a rebalance or connector restart was triggered for any reason, the incremental snapshots would fail to resume after the connector restarted. This behavior was due to regression in the way that the connector handled offsets during incremental snapshots.

This issue is now fixed, and SQL Server users should now observe that if the connector restarts during an incremental snapshot, after a restart the snapshot resumes as intended.

DBZ-9533

In this release, the the earlier MongoDB sink connector Developer Preview feature is promoted to Technology Preview. The Debezium MongoDB sink connector differs from other vendor implementations in that it can ingest raw change events emitted by Debezium connectors without first applying an event flattening transformation. The MongoDB sink connector can take advantage of native Debezium source connector features, such as column type propagation, enabling you to potentially reduce the processing footprint of your data pipeline, and simplify its configuration.

Unlike the JDBC sink relational connector that requires an additional plug-in to be installed to use it, the MongoDB sink connector is bundled alongside the MongoDB source connector in the same artifact. So if you install the Debezium 3.4.3 MongoDB source connector, you also have the MongoDB sink connector.

Minimal configuration is required to get started with the MongoDB sink connector, for example:

{
  "connector.class": "io.debezium.connector.mongodb.MongoDbSinkConnector",
  "connection.string": "...",
  "topics": "topic1,topic2",
  "sink.database": "targetdb"
}

The following configuration properties are mandatory:

For more information, see the Debezium MongoDB sink connector documentation.

DBZ-8339

The Debezium Oracle connector can process the XMLType data type that Oracle uses for handling XML data in the database.

For information about the mappings that the connector uses for the Oracle XMLTYPE, see XML types in the connector documentation.

You can integrate Debezium with OpenLineage to track how data moves through the various systems, transformations, and processes in your data pipeline. The integration with OpenLineage maps events in your data pipeline to artifacts in the OpenLineage data model to provides visibility into where data originates, how it moves, and what dependencies exist in the data pipeline.

For more information about how to enable and configure the OpenLineage integration, see the OpenLineage integration documentation in the Debezium User Guide.

DBZ-9110

OpenLineage enables the capture of rich metadata from data pipelines. With this release, you can use the OpenLineage integration with the JDBC and MongoDB sink connectors.

DBZ-9469

The ability for the Oracle connector to capture changes from a logical standby continues to be available as a Developer Preview feature. When the Debezium connector for Oracle connects to a production or primary database, it uses an internal flush table to manage the flush cycles of the Oracle Log Writer Buffer (LGWR) process. The flush process requires that the user account through which the connector accesses the database has permission to create and write to this flush table. If stand-by database policies restrict data manipulation or prohibit write operations, the connector cannot write to the flush table.

To support an Oracle read-only logical stand-by database, Debezium introduces a property to disable the creation and management of the flush table. You can use this feature with both Oracle Standalone and Oracle RAC installations.

To enable the Oracle connector to use a read-only logical stand-by, add the following connector option:

internal.log.mining.read.only=true

For more information, see the Oracle connector documentation in the Debezium User Guide.