Commit Graph

9722 Commits

Author SHA1 Message Date
Vojtech Juranek
001cb2a640 DBZ-7024 Improve javadocs and comments, fix typos 2024-02-12 13:43:21 +01:00
Vojtech Juranek
78f7c40410 DBZ-7024 Don't stop engine before tasks are fully running 2024-02-12 13:43:21 +01:00
Vojtech Juranek
834b522739 DBZ-7024 Fix ReadOnlyIncrementalSnapshotIT after switch to async engine 2024-02-12 13:43:21 +01:00
Vojtech Juranek
6c71cb7f88 DBZ-7024 Fix StreamingSourceIT after switch to async engine 2024-02-12 13:43:21 +01:00
Vojtech Juranek
ec58dd6a1c DBZ-7024 Fix MySqlRestartIT after switch to async engine 2024-02-12 13:43:21 +01:00
Vojtech Juranek
3a902ff1d6 DBZ-7024 Fix MySqlConnectorSchemaValidateIT after switch to async engine 2024-02-12 13:43:21 +01:00
Vojtech Juranek
4689db90c0 DBZ-7024 Switch abstract embedded tests to async engine 2024-02-12 13:43:21 +01:00
Vojtech Juranek
0137d5f15e DBZ-7024 Switch MySQL tests inheriting from AbstractConnectorTest to async engine 2024-02-12 13:43:21 +01:00
Vojtech Juranek
dfdeab7ab8 DBZ-7024 Add method to await engine shutdown 2024-02-12 13:43:21 +01:00
Vojtech Juranek
3ec22951ce DBZ-7024 Create testing engine and base class for async engine tests 2024-02-12 13:43:21 +01:00
Vojtech Juranek
293b84645d DBZ-7024 Make default RecordCommitter thread unsafe
Default implmentation of `RecordCommitter`, the `SourceRecordCommitter`,
is always created for each task and withing given task is called
sequentially, always in the same thread. There's no need to aquire locks
for each method call.

Make `SourceRecordCommitter` thread unsafe.
2024-02-12 13:43:21 +01:00
Vojtech Juranek
998f00f811 DBZ-7024 Create initial implementation of async embedded engine
Implementation is based on the proposed Debezium design document for
asynchronous embedded engine, which is currently still WIP:
https://github.com/debezium/debezium-design-documents/pull/8
2024-02-12 13:43:21 +01:00
Vojtech Juranek
69bbed1fa3 DBZ-7024 Allow to override type into AbstractConnectorTest 2024-02-12 13:43:21 +01:00
Vojtech Juranek
1eec31b3f3 DBZ-7024 Allow to specify multiple records when error should be thrown during processing 2024-02-12 13:43:21 +01:00
Vojtech Juranek
542b0fec7f DBZ-7024 Intorduce retryable callable 2024-02-12 13:43:21 +01:00
Vojtech Juranek
d7b7768071 DBZ-7024 Add more testing connectors
Add connector which runs mutiple tasks and connector whose some of
the tasks fail.
2024-02-12 13:43:21 +01:00
Vojtech Juranek
7eaf0fc288 DBZ-7024 Move reuseable testing functions for DebeziumEngine into common util class 2024-02-12 13:43:21 +01:00
Vojtech Juranek
b8e16ee89f DBZ-7024 Move reuseable interface implementations for DebeziumEngine into common class 2024-02-12 13:43:21 +01:00
Vojtech Juranek
ba35b395c5 DBZ-7024 Move required setup into EmbeddedWorkerConfig
Move required by `WorkerConfig` validators into `EmbeddedWorkerConfig`
so we have all Kafka related stuff in the same class.
2024-02-12 13:43:21 +01:00
Vojtech Juranek
4452e3d095 DBZ-7024 Move EmbeddedConfig into separate class
`EmbeddedConfig` needs to be shared with other implmentations of
`DebeziumEngine` as long as Debezium embedded depends on the Kafka
model.
2024-02-12 13:43:21 +01:00
Jakub Cechacek
7f3210214c DBZ-7451 Support for standard BSON representation of UUID in incremental snapshots 2024-02-12 13:19:13 +01:00
Chris Cranford
ce87f06751 [ci] Fix Oracle test workflow worker space failure
A GitHub worker only has so much space and currently when this stage of the job runs concurrently with the
"test-connector" stage, we are facing a worker out of space problem. To attempt to combat this issue, the
REST extension tests will run sequentially after the "test-connector" stage has concluded.
2024-02-09 18:24:00 -05:00
Chris Cranford
9fbcd6baa1 DBZ-7456 Skip PK change LOB column reselect if column is excluded 2024-02-09 16:43:13 -05:00
Chris Cranford
8265e4e42e DBZ-7468 Support parsing MariaDB SYSTEM VERSIONING DDL 2024-02-09 16:40:20 -05:00
Vadzim Ramanenka
237755331a DBZ-7463: SQL Server queries with special characters fail after applying DBZ-7273
Enclose column names in square brackets to account for spaces and
special characters.
2024-02-09 14:46:38 -05:00
mfvitale
6dc3b578f8 DBZ-7467 Fix missing signaling actions 2024-02-09 11:45:56 -05:00
Lars M Johansson
20ccffa223 DBZ-7460 [docs] The expected value pattern for table.include.list does not align with the documentation 2024-02-09 08:23:38 -05:00
rkerner
5c817c4fa3 DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Added deduplication with a Map with a key based on the SMT's fully-qualified class name and version.
closes https://issues.redhat.com/browse/DBZ-7416
2024-02-08 15:58:53 +01:00
mfvitale
2c9b77fc1c DBZ-7452 defaultJdbcConfig will use correct hostname and port from parameters 2024-02-08 14:31:55 +01:00
Chris Cranford
e2d03d9936 DBZ-7425 Use adapter-specific offset loaders when copying offsets 2024-02-07 19:40:30 -05:00
Chris Cranford
a597a82c19 DBZ-7439 Remove unnecessary log/output entries 2024-02-07 15:00:27 +01:00
Chris Cranford
de9364bb4d DBZ-7439 Fix test compatibility 2024-02-07 15:00:27 +01:00
Ondrej Babec
c806a04531 DBZ-7439 [ci] Add Oracle 23 initialisation scripts
* Create oracle-init dir with SQL init scripts
* Create init-db.sh to set default password and run init scripts
* Update test.sh to run init-db.sh for oracle-23
2024-02-07 15:00:27 +01:00
Chris Cranford
3d22ed30a3 DBZ-7439 [ci] Update TF with Oracle 23
* Introduce new InstantClient version property
  Needed because ojdbc8 versions for Oracle 23 have different version semantics
  Also adjusted documentation/build pipelines
* Introduce oracle-23 profile
  Allows testing Oracle 23 separately from distribution builds
2024-02-07 15:00:27 +01:00
Ondrej Babec
02700d75b4 DBZ-7439 [ci] Introduce Oracle 23 to Testing Farm
* Update .packit.yaml with oracle 23 image
* Modify startup script
2024-02-07 15:00:27 +01:00
Chris Cranford
8aa2b7577c
[ci] Mark test disabled for now 2024-02-06 12:28:48 -05:00
Chris Cranford
8a2c098e78 [ci] Mark testEmptyChangesProducesHeartbeat as flaky 2024-02-06 09:49:05 -05:00
Chris Cranford
c4b0da4de2 DBZ-7449 Upgrade Quarkus (Debezium Server) to 3.2.9.Final 2024-02-06 12:30:20 +01:00
Sergey Ivanov
d96c30ef3f DBZ-7437: ReselectColumnsPostProcessor filter not use exclude predicate. 2024-02-06 12:27:55 +01:00
Fiore Mario Vitale
e2f0123e51 DBZ-7300 Improve snapshotter docs
Co-authored-by: roldanbob <broldan@redhat.com>
2024-02-06 09:32:47 +01:00
mfvitale
62ff1a735f DBZ-7300 Review PostgreSQL documentations with new Snapshotter interfaces 2024-02-06 09:32:47 +01:00
mfvitale
3ff775ea6c DBZ-7301 Add test for a custom Snapshotter and SnapshotQuery 2024-02-06 07:12:42 +01:00
mfvitale
a8a07e35f1 DBZ-7301 Implement SnapshotLock for MySQL connector 2024-02-06 07:12:42 +01:00
mfvitale
6cb01f67d1 DBZ-7301 Implement SnapshotQuery for MySQL connector 2024-02-06 07:12:42 +01:00
mfvitale
0e68bc642d DBZ-7301 Support Snapshotter for MariaDB 2024-02-06 07:12:42 +01:00
mfvitale
1cdf2836dd DBZ-7301 Implement Snapshotter for MySQL connector 2024-02-06 07:12:42 +01:00
mfvitale
f78d0cbd19 DBZ-7301 Add SnapshotterService to ChangeEventSourceFactory and SnapshotChangeEventSource 2024-02-06 07:12:42 +01:00
mfvitale
c9458f4f58 DBZ-7300 Snapshotter, SnapshotLock and SnapshotQuery are now services registered in the ServiceRegistry 2024-02-06 07:12:42 +01:00
jchipmunk
c51ef3099a DBZ-7445 LogMiner batch size does not increase automatically
The condition for increasing the LogMiner batch size must depend on log.mining.batch.size.max, and not on log.mining.batch.size.min.
2024-02-05 12:28:16 -05:00
Chris Cranford
c669857760 DBZ-7446 Fix test failure 2024-02-05 12:26:39 -05:00