alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
9,514 Commits 3 Branches 0 Tags 51 MiB
dfdeab7ab8
Commit Graph

570 Commits

Author SHA1 Message Date
Vojtech Juranek
dfdeab7ab8 DBZ-7024 Add method to await engine shutdown 2024-02-12 13:43:21 +01:00
Vojtech Juranek
3ec22951ce DBZ-7024 Create testing engine and base class for async engine tests 2024-02-12 13:43:21 +01:00
Vojtech Juranek
293b84645d DBZ-7024 Make default RecordCommitter thread unsafe 
Default implmentation of `RecordCommitter`, the `SourceRecordCommitter`,
is always created for each task and withing given task is called
sequentially, always in the same thread. There's no need to aquire locks
for each method call.

Make `SourceRecordCommitter` thread unsafe.
 2024-02-12 13:43:21 +01:00
Vojtech Juranek
998f00f811 DBZ-7024 Create initial implementation of async embedded engine 
Implementation is based on the proposed Debezium design document for
asynchronous embedded engine, which is currently still WIP:
https://github.com/debezium/debezium-design-documents/pull/8
 2024-02-12 13:43:21 +01:00
Vojtech Juranek
69bbed1fa3 DBZ-7024 Allow to override type into AbstractConnectorTest 2024-02-12 13:43:21 +01:00
Vojtech Juranek
1eec31b3f3 DBZ-7024 Allow to specify multiple records when error should be thrown during processing 2024-02-12 13:43:21 +01:00
Vojtech Juranek
542b0fec7f DBZ-7024 Intorduce retryable callable 2024-02-12 13:43:21 +01:00
Vojtech Juranek
d7b7768071 DBZ-7024 Add more testing connectors 
Add connector which runs mutiple tasks and connector whose some of
the tasks fail.
 2024-02-12 13:43:21 +01:00
Vojtech Juranek
7eaf0fc288 DBZ-7024 Move reuseable testing functions for DebeziumEngine into common util class 2024-02-12 13:43:21 +01:00
Vojtech Juranek
b8e16ee89f DBZ-7024 Move reuseable interface implementations for DebeziumEngine into common class 2024-02-12 13:43:21 +01:00
Vojtech Juranek
ba35b395c5 DBZ-7024 Move required setup into EmbeddedWorkerConfig 
Move required by `WorkerConfig` validators into `EmbeddedWorkerConfig`
so we have all Kafka related stuff in the same class.
 2024-02-12 13:43:21 +01:00
Vojtech Juranek
4452e3d095 DBZ-7024 Move EmbeddedConfig into separate class 
`EmbeddedConfig` needs to be shared with other implmentations of
`DebeziumEngine` as long as Debezium embedded depends on the Kafka
model.
 2024-02-12 13:43:21 +01:00
Chris Cranford
a597a82c19 DBZ-7439 Remove unnecessary log/output entries 2024-02-07 15:00:27 +01:00
Chris Cranford
de9364bb4d DBZ-7439 Fix test compatibility 2024-02-07 15:00:27 +01:00
Debezium Builder
65d63ed42d [maven-release-plugin] prepare for next development iteration 2024-01-21 10:12:45 +00:00
Debezium Builder
485fa82a8f [maven-release-plugin] prepare release v2.6.0.Alpha1 2024-01-21 10:12:44 +00:00
Artem Shubovych
27f42d101a DBZ-7342 Replace temporary variable with immediate and early return 2024-01-17 09:50:54 +01:00
Artem Shubovych
d9de6ceba2 DBZ-7342 Respect the max error retries setting 2024-01-17 09:50:52 +01:00
Roman Kudryashov
0c80f1f38d DBZ-7284 Provide config option to customize CloudEvents.data schema name 2024-01-11 13:20:35 +01:00
Jakub Cechacek
aa0c53ec2a DBZ-7260 Offset consolidation test coverage 2024-01-11 09:58:24 +01:00
mfvitale
47cbdee526 DBZ-7311 Permits to execute a blocking snapshot even if snapshot.mode=never 2024-01-08 12:52:03 +01:00
Jiri Pechanec
7a80c9dae8 DBZ-7098 Converting engine should honor unsupported tombstone flag 2024-01-05 08:25:23 +01:00
Debezium Builder
3853d20f44 [maven-release-plugin] prepare for next development iteration 2023-12-21 06:52:01 +00:00
Debezium Builder
5d35e9caaa [maven-release-plugin] prepare release v2.5.0.Final 2023-12-21 06:52:01 +00:00
Roman Kudryashov
516aa87fad DBZ-7235 Add option to customize CloudEvents schema name 2023-12-20 06:53:37 +01:00
Debezium Builder
2c1def7241 [maven-release-plugin] prepare for next development iteration 2023-12-14 09:43:13 +00:00
Debezium Builder
ef8260f802 [maven-release-plugin] prepare release v2.5.0.CR1 2023-12-14 09:43:12 +00:00
Chris Cranford
2b02b3982e DBZ-4321 Rework configuration options 2023-12-13 11:27:40 -05:00
Chris Cranford
59027ed5ed DBZ-4321 New PostProcessor contract and Column Reselection 2023-12-13 11:27:40 -05:00
mfvitale
5ed16284f4 DBZ-6834 Provide INSERT/DELETE semantics for MongoDb incremental snapshot watermarking 2023-12-06 14:10:26 +01:00
mfvitale
4fedfbba03 DBZ-6834 Provide INSERT/DELETE semantics for incremental snapshot watermarking 2023-12-06 14:10:26 +01:00
Debezium Builder
0fd1c0dc9a [maven-release-plugin] prepare for next development iteration 2023-12-04 13:55:35 +00:00
Debezium Builder
3e2d75f0da [maven-release-plugin] prepare release v2.5.0.Beta1 2023-12-04 13:55:35 +00:00
“vsantonastaso”
8c1c369449 DBZ-6878 add table specific notification in initial snapshot 2023-11-29 08:32:21 +01:00
Sebastiaan Knijnenburg
036eda0c64 DBZ-6723 Expose partition number in ChangeEvent interface 
Based on feedback in https://github.com/debezium/debezium-server/pull/33 this
commit adds the partition() method to the ChangeEvent interface and implements
it in the EmbeddedEngineChangeEvent. This allows reading the assigned partition
for an event in downstream processors, for example in custom Sinks that need
the assigned partition for routing purposes.

Link:  https://issues.redhat.com/browse/DBZ-6723
 2023-11-28 09:38:18 +01:00
Roman Kudryashov
1992c1e7e4 DBZ-7159 Fail fast during deserialization if a value is not a CloudEvent 2023-11-23 14:19:37 +01:00
Vojtech Juranek
60939c8965 DBZ-7099 Provide default value for PeriodicCommitOffsetPolicy 
In 7b4cf1901 deprecated `io.debezium.embedded.spi.OffsetCommitPolicy`,
which provided also constructor for `Configuration`, was removed.
This constructor was actually used for creating `OffsetCommitPolicy`.
`Configuration` provides default values for options which are not
explicitly set, while `Properties` based constructor cannot do that
and therefore with this switch the default value for
`offset.flush.interval.ms` is now missing.

As debezium-api package has no knowledge about `Configuration`
interface, which is part of debezium-core, and thus about the default
values, specify default value direcly in the `PeriodicCommitOffsetPolicy`
class.
 2023-11-14 06:58:11 +01:00
Debezium Builder
1521445908 [maven-release-plugin] prepare for next development iteration 2023-11-10 10:26:05 +00:00
Debezium Builder
6c6f6e9138 [maven-release-plugin] prepare release v2.5.0.Alpha2 2023-11-10 10:26:05 +00:00
Vojtech Juranek
cf7f0f3801 DBZ-7110 Use better name for DebeziumEngine.Builder implementation 2023-11-07 10:13:50 +01:00
Vojtech Juranek
c34cf6920c DBZ-7110 Remove deprecated EmbeddedEngine interface 2023-11-07 10:13:50 +01:00
Vojtech Juranek
11d2ff0b9b DBZ-7007 Use better name for auxiliary variable detecing if engine is running 2023-11-06 10:50:21 +01:00
Vojtech Juranek
89054bf8a8 DBZ-7007 Introduce TestingDebeziumEngine and switch to it in AbstractConnectorTest 
Postgres `RecordsStreamProducerIT` reliaes on
EmebeddedEngine.runWithTask(). As this method effectively expose
engine's internal task and tests do the asserts against the state
of the task, it's hard to replace it. If we want to keep the tests,
the most simple approach seems to expose engine task in similar way
how EmbeddedEngine does that.

Add interface for testing Debezium engine, which would define minimal
set of methods which needs to be exposed by the implementing classes to
be able to run the testsuite against the Debezium engine. The number of
such methods should be as low as possible. Implementing classes would
typically act as proxies to actual `DebeziumEngine` implementations.

Add `TestingDebeziumEngine` implementation for `EmbeddedEngine` and
switch to `TestingDebeziumEngine` in `AbstractConnectorTest`.
 2023-11-06 10:50:21 +01:00
Vojtech Juranek
fdab2cd61b DBZ-7007 Remove engine.isRunning() from testsuite 2023-11-06 10:50:21 +01:00
Vojtech Juranek
89277591dd DBZ-7007 Replace engine.await() with Awaitility condition 
In the second case replace infinite loop by one minute wait. If this is
not sufficient for any case, we can increase the timeout in the future.
 2023-11-06 10:50:21 +01:00
Vojtech Juranek
9b1d5172ff DBZ-6213 Remove unneeded code from the testsuite 
After fixing start method to really start the connector, this wait loop
is not needed any more.
 2023-11-06 10:50:21 +01:00
Vojtech Juranek
1a58845507 DBZ-7007 Use close() instead of EmbeddedEngine#stop() method 2023-11-06 10:50:21 +01:00
Vojtech Juranek
953f1c77aa DBZ-7007 Switch to DebeziuEngine in ConnectorOutputTest 2023-11-06 10:50:21 +01:00
Vojtech Juranek
d0c2658c43 DBZ-7007 Switch to DebeziuEngine in EmbeddedEngineTest 2023-11-06 10:50:21 +01:00
Vojtech Juranek
3295844cc3 DBZ-7007 Use property based configuration in the emebdded tests 2023-11-06 10:50:21 +01:00