Commit Graph

9706 Commits

Author SHA1 Message Date
Vojtech Juranek
7eaf0fc288 DBZ-7024 Move reuseable testing functions for DebeziumEngine into common util class 2024-02-12 13:43:21 +01:00
Vojtech Juranek
b8e16ee89f DBZ-7024 Move reuseable interface implementations for DebeziumEngine into common class 2024-02-12 13:43:21 +01:00
Vojtech Juranek
ba35b395c5 DBZ-7024 Move required setup into EmbeddedWorkerConfig
Move required by `WorkerConfig` validators into `EmbeddedWorkerConfig`
so we have all Kafka related stuff in the same class.
2024-02-12 13:43:21 +01:00
Vojtech Juranek
4452e3d095 DBZ-7024 Move EmbeddedConfig into separate class
`EmbeddedConfig` needs to be shared with other implmentations of
`DebeziumEngine` as long as Debezium embedded depends on the Kafka
model.
2024-02-12 13:43:21 +01:00
Jakub Cechacek
7f3210214c DBZ-7451 Support for standard BSON representation of UUID in incremental snapshots 2024-02-12 13:19:13 +01:00
Chris Cranford
ce87f06751 [ci] Fix Oracle test workflow worker space failure
A GitHub worker only has so much space and currently when this stage of the job runs concurrently with the
"test-connector" stage, we are facing a worker out of space problem. To attempt to combat this issue, the
REST extension tests will run sequentially after the "test-connector" stage has concluded.
2024-02-09 18:24:00 -05:00
Chris Cranford
9fbcd6baa1 DBZ-7456 Skip PK change LOB column reselect if column is excluded 2024-02-09 16:43:13 -05:00
Chris Cranford
8265e4e42e DBZ-7468 Support parsing MariaDB SYSTEM VERSIONING DDL 2024-02-09 16:40:20 -05:00
Vadzim Ramanenka
237755331a DBZ-7463: SQL Server queries with special characters fail after applying DBZ-7273
Enclose column names in square brackets to account for spaces and
special characters.
2024-02-09 14:46:38 -05:00
mfvitale
6dc3b578f8 DBZ-7467 Fix missing signaling actions 2024-02-09 11:45:56 -05:00
Lars M Johansson
20ccffa223 DBZ-7460 [docs] The expected value pattern for table.include.list does not align with the documentation 2024-02-09 08:23:38 -05:00
rkerner
5c817c4fa3 DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Added deduplication with a Map with a key based on the SMT's fully-qualified class name and version.
closes https://issues.redhat.com/browse/DBZ-7416
2024-02-08 15:58:53 +01:00
mfvitale
2c9b77fc1c DBZ-7452 defaultJdbcConfig will use correct hostname and port from parameters 2024-02-08 14:31:55 +01:00
Chris Cranford
e2d03d9936 DBZ-7425 Use adapter-specific offset loaders when copying offsets 2024-02-07 19:40:30 -05:00
Chris Cranford
a597a82c19 DBZ-7439 Remove unnecessary log/output entries 2024-02-07 15:00:27 +01:00
Chris Cranford
de9364bb4d DBZ-7439 Fix test compatibility 2024-02-07 15:00:27 +01:00
Ondrej Babec
c806a04531 DBZ-7439 [ci] Add Oracle 23 initialisation scripts
* Create oracle-init dir with SQL init scripts
* Create init-db.sh to set default password and run init scripts
* Update test.sh to run init-db.sh for oracle-23
2024-02-07 15:00:27 +01:00
Chris Cranford
3d22ed30a3 DBZ-7439 [ci] Update TF with Oracle 23
* Introduce new InstantClient version property
  Needed because ojdbc8 versions for Oracle 23 have different version semantics
  Also adjusted documentation/build pipelines
* Introduce oracle-23 profile
  Allows testing Oracle 23 separately from distribution builds
2024-02-07 15:00:27 +01:00
Ondrej Babec
02700d75b4 DBZ-7439 [ci] Introduce Oracle 23 to Testing Farm
* Update .packit.yaml with oracle 23 image
* Modify startup script
2024-02-07 15:00:27 +01:00
Chris Cranford
8aa2b7577c
[ci] Mark test disabled for now 2024-02-06 12:28:48 -05:00
Chris Cranford
8a2c098e78 [ci] Mark testEmptyChangesProducesHeartbeat as flaky 2024-02-06 09:49:05 -05:00
Chris Cranford
c4b0da4de2 DBZ-7449 Upgrade Quarkus (Debezium Server) to 3.2.9.Final 2024-02-06 12:30:20 +01:00
Sergey Ivanov
d96c30ef3f DBZ-7437: ReselectColumnsPostProcessor filter not use exclude predicate. 2024-02-06 12:27:55 +01:00
Fiore Mario Vitale
e2f0123e51 DBZ-7300 Improve snapshotter docs
Co-authored-by: roldanbob <broldan@redhat.com>
2024-02-06 09:32:47 +01:00
mfvitale
62ff1a735f DBZ-7300 Review PostgreSQL documentations with new Snapshotter interfaces 2024-02-06 09:32:47 +01:00
mfvitale
3ff775ea6c DBZ-7301 Add test for a custom Snapshotter and SnapshotQuery 2024-02-06 07:12:42 +01:00
mfvitale
a8a07e35f1 DBZ-7301 Implement SnapshotLock for MySQL connector 2024-02-06 07:12:42 +01:00
mfvitale
6cb01f67d1 DBZ-7301 Implement SnapshotQuery for MySQL connector 2024-02-06 07:12:42 +01:00
mfvitale
0e68bc642d DBZ-7301 Support Snapshotter for MariaDB 2024-02-06 07:12:42 +01:00
mfvitale
1cdf2836dd DBZ-7301 Implement Snapshotter for MySQL connector 2024-02-06 07:12:42 +01:00
mfvitale
f78d0cbd19 DBZ-7301 Add SnapshotterService to ChangeEventSourceFactory and SnapshotChangeEventSource 2024-02-06 07:12:42 +01:00
mfvitale
c9458f4f58 DBZ-7300 Snapshotter, SnapshotLock and SnapshotQuery are now services registered in the ServiceRegistry 2024-02-06 07:12:42 +01:00
jchipmunk
c51ef3099a DBZ-7445 LogMiner batch size does not increase automatically
The condition for increasing the LogMiner batch size must depend on log.mining.batch.size.max, and not on log.mining.batch.size.min.
2024-02-05 12:28:16 -05:00
Chris Cranford
c669857760 DBZ-7446 Fix test failure 2024-02-05 12:26:39 -05:00
jchipmunk
04b8597239 DBZ-7446 Reduce string creation during SQL_REDO column read
Oracle Connector can create a new string when reading SQL_REDO column only if CSF column value is 1.
2024-02-05 12:26:39 -05:00
mfvitale
d0e4ad7e14 DBZ-7441 Postpone SignalProcessor start after streaming is initialized
This will avoid that channels that not depends on the event streaming like the 'source' channel, will start processing signals before the IncrementalSnapshotChangeEventSource is initialized.
2024-02-05 14:06:40 +01:00
Clifford Cheefoon
d53ca0daec DBZ-7443 Correct debezium.sink.pubsub.flowcontrol variables
Correct case discrepancy for configuration variables between docs and pubsub sink connector
df95323322/debezium-server-pubsub/src/main/java/io/debezium/server/pubsub/PubSubChangeConsumer.java (L98C3-L98C88)
2024-02-05 10:42:54 +01:00
mfvitale
57c4e2a876 DBZ-7300 Add new properties to config definition 2024-02-02 13:41:45 +01:00
mfvitale
881442de3d DBZ-7300 Fix duplicate field assignment for CONNECTOR_SNAPSHOT group 2024-02-02 13:41:45 +01:00
mfvitale
3b92786a41 DBZ-7300 Snapshotter, SnapshotLock and SnapshotQuery are now services registered in the ServiceRegistry 2024-02-02 13:41:45 +01:00
mfvitale
8e17724392 DBZ-7300 Refactor Snapshotter interface to be an SPI through SnapshotterProvider.
Also, this will split snapshot locking and query behavior into a separate interface
2024-02-02 13:41:45 +01:00
mfvitale
fa338b61c4 DBZ-7436 Add empty statistics for SingleDuration to avoid NPE 2024-02-02 12:36:24 +01:00
Jiri Pechanec
c994001d6a DBZ-7442 Upgrade protobuf to 3.25.2 2024-02-02 11:50:19 +01:00
Ondrej Babec
8c305e22bc DBZ-7432 Create systemtests for Unwrap SMT 2024-02-01 13:40:50 +01:00
Jakub Cechacek
31dac84135 DBZ-7379 Cleaned up MongoClient creating and added ability to use connector specific keystore/truststore 2024-01-31 15:01:04 +01:00
Chris Cranford
2bfa92e6af DBZ-7429 Resolve primary key values from after struct 2024-01-31 12:05:07 +01:00
Fiore Mario Vitale
1c93a92b84 DBZ-7421 Change access modifier for SnapshotDataCollection to private
Co-authored-by: Jiri Pechanec <jpechane@redhat.com>
2024-01-31 11:05:04 +01:00
mfvitale
f9034f2108 DBZ-7421 Encapsulate data collection queue and its json version into a dedicated class 2024-01-31 11:05:04 +01:00
mfvitale
81298865a5 DBZ-7421 Improve incremental snapshot performance with increasing number of collections to snapshot 2024-01-31 11:05:04 +01:00
Chris Cranford
0db1a3eaae DBZ-7431 Document JDBC connector sink option connection.provider 2024-01-31 11:01:30 +01:00