Commit Graph

737 Commits

Author SHA1 Message Date
mfvitale
e0659f1a0e DBZ-6496 Fix performance issue on testNotification test with Oracle and MS SQL Server connectors 2023-05-24 18:51:49 +02:00
Debezium Builder
a5b44767bc [maven-release-plugin] prepare for next development iteration 2023-05-11 11:05:24 +00:00
Debezium Builder
1b1a7531f2 [maven-release-plugin] prepare release v2.3.0.Alpha1 2023-05-11 11:05:24 +00:00
mfvitale
7429b4e6f1 DBZ-1973 Add incremental snapshot notifications 2023-05-10 12:22:02 +02:00
mfvitale
811b800f55 DBZ-1973 Add support for Notifications
DBZ-1973 Add more tests for other connectors

DBZ-1973 Add send method with offset parameter

DBZ-1973 Instantiate NotificationService in the task class

DBZ-1973 Instantiate NotificationService in the task class

DBZ-4027 Move specific sink channel configuration to SinkNotificationChannel

DBZ-4027 Remove not used SPI file

DBZ-1973 Moved SPI file definition to debezium-core

DBZ-1973 Rename KafkaNotificationChannel to more generic SinkNotificationChannel

DBZ-1973 Code refactor

DBZ-1973 Improve configuration property description

DBZ-1973 Improve test

DBZ-1973 Add close method to NotificationChannel

DBZ-1973 Implement KafkaNotificationChannel

DBZ-1973 Add NotificationService and LogNotificationChannel
2023-05-10 12:22:02 +02:00
mfvitale
952340286e DBZ-4027 Make signalling channel configurable
DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Code style

DBZ-4027 Make SignalPayload more generic and extensible

DBZ-4027 Rename DatabaseSignalChannel to SourceSignalChannel

DBZ-4027 Improve logging

DBZ-4027 Moved SPI file definition to debezium-core

DBZ-4027 Move SignalProcessor synchronization point to be processed only when a signal cdc event arrives.

DBZ-4027 Add EventDispatcher constructor without signalProcessor for spanner connector

DBZ-4027 Fix NPE

DBZ-4027 Fix NPE

DBZ-4027 Formatting

DBZ-4027 Correctly manage signal on not supported connector

DBZ-4027 Use the correct MongoDbOffset

DBZ-4027 Correctly initialize offset for Oracle and SqlServer connectors

DBZ-4027 Register SPI implementations

DBZ-4027 Improve SignalProcessor instantiation

DBZ-4027 Pass source info in case of SchemaChanges action

DBZ-4027 Manage close event in a synchronous way

DBZ-4027 Correctly init offset context also in case of snapshot mode 'never'

DBZ-4027 Fix MySqlMetricsIT test

DBZ-4027 Move KafkaSignalChannel to core

DBZ-4027 Move KafkaSignalChannel to core

DBZ-4027 Set pass offset context after initial snapshot to SignalProcessor

DBZ-4027 Pass OffsetContext to signal processor

DBZ-4027 Pass CommonConnectorConfig to SignalChannelReader init method

DBZ-4027 Move Incremental snapshot window actions to dedicated package

DBZ-4027 Align SignalsIT test with new code

DBZ-4027 Fix SignalsIT test

DBZ-4027 Fix SignalProcessor scheduling

DBZ-4027 Moved DatabaseSignalChannel and SignalChannelReader to dedicated package

DBZ-4027 Start SignalProcessor from ChangeEventSourceCoordinator

DBZ-4027 Create SignalProcessor and renamed Signal to DatabaseSignalChannel

DBZ-4027 Initial refactoring of signal feature
2023-05-05 13:35:40 +02:00
Jiri Pechanec
58ef4f0b98 DBZ-6157 Disable potentially dangerous MySQL JDBC props 2023-05-04 11:54:04 +02:00
harveyyue
3860a4faca DBZ-6388 __source_ts_ms r (read) operation date is set to future for SQL Server 2023-05-02 08:11:02 -04:00
ani-sha
a453c3ca19 DBZ-6259 Add missing description of various snapshot modes in the connector field definitions 2023-05-01 11:03:04 -04:00
Debezium Builder
bb1b79f3bd [maven-release-plugin] prepare for next development iteration 2023-04-20 09:09:43 +02:00
Debezium Builder
5483b4de99 [maven-release-plugin] prepare release v2.2.0.Final 2023-04-20 09:09:43 +02:00
Jiri Pechanec
f4c200ac3e DBZ-2979 Fix tests 2023-04-19 09:04:29 +02:00
Ronak Jain
465b355dd0 DBZ-2979 If column.include.list/column.exclude.list are used and the target table receives an update for the excluded (or not included) column - such events should be ignored
* Add More Test Cases

Addresses/Closes DBZ-2979
2023-04-19 09:04:29 +02:00
Ronak Jain
762122ecf5 DBZ-2979 If column.include.list/column.exclude.list are used and the target table receives an update for the excluded (or not included) column - such events should be ignored
* Add Mysql and SqlServer Connector Changes
* Change Docs
* Clean Test Case
* Formatting and Checkstyle

Addresses/Closes DBZ-2979
2023-04-19 09:04:29 +02:00
Jiri Pechanec
5b2638b988 DBZ-4475 Mark test as flaky 2023-04-19 09:04:02 +02:00
Debezium Builder
976a47a3a3 [maven-release-plugin] prepare for next development iteration 2023-04-14 08:46:06 +00:00
Debezium Builder
1ca8399e8b [maven-release-plugin] prepare release v2.2.0.CR1 2023-04-14 08:46:06 +00:00
Jiri Pechanec
8a648fbb4d DBZ-4488 Refactor code to keep error handler immutable 2023-04-13 08:22:45 +02:00
Jiri Pechanec
1c58055819 DBZ-4488 Use more descritpive names 2023-04-13 08:22:45 +02:00
Jacob Gminder
5e615ff796 DBZ-4488: Reuse the ErrorHandler between retries 2023-04-13 08:22:45 +02:00
Jiri Pechanec
84d0d580bc DBZ-4488 Refactor the code 2023-04-13 08:22:45 +02:00
Jacob Gminder
9d5b6df0f7 DBZ-4488: Allow SQL Server connectors to set a maximum number of retriable exception restarts. 2023-04-13 08:22:45 +02:00
harveyyue
c525f34cd0 DBZ-6070 DDL events not stored in schema history topic for excluded tables 2023-04-06 12:10:30 +02:00
Debezium Builder
5c650fd054 [maven-release-plugin] prepare for next development iteration 2023-03-31 10:00:16 +00:00
Debezium Builder
68ffc4833b [maven-release-plugin] prepare release v2.2.0.Beta1 2023-03-31 10:00:16 +00:00
harveyyue
1877906dd8 DBZ-6197 Remove duplicated createDdlFilter method from historized connector config 2023-03-13 10:56:43 +01:00
Debezium Builder
55d65e9618 [maven-release-plugin] prepare for next development iteration 2023-03-08 11:38:17 +00:00
Debezium Builder
6d183c3dc1 [maven-release-plugin] prepare release v2.2.0.Alpha3 2023-03-08 11:38:16 +00:00
Jiri Pechanec
3da2434c6c DBZ-823 Restrict the type with generics 2023-03-03 12:47:15 +01:00
Jiri Pechanec
5e666f00ee DBZ-823 Add JavaDoc to new interfaces 2023-03-03 12:47:15 +01:00
Jiri Pechanec
7d92dc0211 DBZ-823 Don't use getter based name 2023-03-03 12:47:15 +01:00
Jiri Pechanec
0f4adb9fe6 DBZ-823 Use better name for CF with main connection 2023-03-03 12:47:15 +01:00
ggaborg
e875e784a0 DBZ-823 CR changes 2023-03-03 12:47:15 +01:00
ggaborg
065dc997b9 DBZ-823 Optionally parallelize initial snapshots 2023-03-03 12:47:15 +01:00
Debezium Builder
2245ceda44 [maven-release-plugin] prepare for next development iteration 2023-02-16 11:28:25 +00:00
Debezium Builder
20f3b16e8a [maven-release-plugin] prepare release v2.2.0.Alpha2 2023-02-16 11:28:25 +00:00
Jacob Gminder
6beea85425 DBZ-6084: Retrieve the database names from the configuration instead of re-parsing them from scratch. 2023-02-07 13:11:23 +01:00
Jacob Gminder
0a1862a5cf DBZ-6084: Validate that database.names is not empty in the SQL server connector configuration. 2023-02-07 13:11:23 +01:00
Jacob Gminder
4acb9304d1 DBZ-6084: Set the number of tasks to the minimum of maxTasks or the number of databases. 2023-02-07 13:11:23 +01:00
harveyyue
57b902ff8c DBZ-5743 Introduce alternative AVRO_UNICODE mode for field name selector 2023-02-02 13:21:45 +01:00
Vojtech Juranek
c52ffe2fcd DBZ-6035 Await transactions also in IncrementalSnapshotWithRecompileIT
Override no-op waitForCdcTransactionPropagation() also in
IncrementalSnapshotWithRecompileIT which also fails very often.
2023-01-31 06:55:40 +01:00
Vojtech Juranek
f53237c249 DBZ-6035 Await for transactions to be propagated to CDC table
Lowering polling interval intorduced in previous commit doesn't fully
fix the issue and there is still small race condition. To fully fix it
add function for awaiting transaction to be propagate to CDC table and
await transaction in tests which randomly fails.
2023-01-26 07:33:10 +01:00
Vojtech Juranek
895fd0b99f DBZ-6035 Add function for adjusting SQL server polling interval
Some tests fail randomly as they insert some records at the begging
of the test, start Debezium and asserts that there are no records.
However, as SQL server poll for the changes and copies them into
CDC table not immediately but in specified interval (by default 5
secods), there can be a race condition when the entries inserted
at the beginning of the test are copied into CDC table after starting
Debezium and it resutls into the test failure as there are unexpected
records.

Add function for adjusting the polling interval and decrease polling
interval in `IncrementalSnapshotIT` which fails very often.

This doesn't fully fix the race condition, but decreases the changes
the issue is hit.
2023-01-26 07:33:10 +01:00
Debezium Builder
04f5291a2d [maven-release-plugin] prepare for next development iteration 2023-01-19 11:32:10 +00:00
Debezium Builder
027527963e [maven-release-plugin] prepare release v2.2.0.Alpha1 2023-01-19 11:32:10 +00:00
Anil Dasari
e1f9d64fbb DBZ-5852:Snapshotter#snapshotCompleted is invoked regardless of snapshot result 2023-01-19 09:24:02 +01:00
Jeremy Ford
8da86b78e8 DBZ-5879 Fix unit tests 2023-01-18 14:46:14 +01:00
Jiri Pechanec
8ca5e0a9b5 DBZ-4669 Reformat after plugin version consolidation 2023-01-11 06:44:53 +01:00
Mark Lambert
e5d16b14a6 DBZ-2439 Remove redundant modifiers 2023-01-11 06:44:53 +01:00
Debezium Builder
cf1720e4cb [maven-release-plugin] prepare for next development iteration 2022-12-22 10:11:54 +00:00