Commit Graph

703 Commits

Author SHA1 Message Date
rkerner
56e63b98a8 DBZ-7223 Add the MongoDB sink connector
* add basic tests for MongoDB sink connector (MySQL to MongoDB 1 node ReplicaSet and multi-node sharded cluster)

closes https://issues.redhat.com/browse/DBZ-7223
2024-07-04 13:21:46 +02:00
Jiri Pechanec
6ba5a1e631 DBZ-7223 Use Optional attributes 2024-07-04 13:21:46 +02:00
rkerner
3799e3d76b DBZ-7223 Add the MongoDB sink connector
closes https://issues.redhat.com/browse/DBZ-7223
2024-07-04 13:21:46 +02:00
ani-sha
bb32d47f6b DBZ-7760 Documentation for collection scoped streaming 2024-06-28 05:39:10 +02:00
ani-sha
7df6cc7b48 DBZ-7760 MongoDb collection scoped streaming 2024-06-28 05:39:10 +02:00
ani-sha
1315a857f9 DBZ-6522 Revert test changes for shouldFailWithoutPermissions() 2024-06-19 13:27:01 +02:00
ani-sha
5f6f40d591 DBZ-6522 Obtain offset _data string using mongo router to validate tests 2024-06-19 13:27:01 +02:00
ani-sha
b26aec4e9b DBZ-6522 Reorder assert statements 2024-06-19 13:27:01 +02:00
ani-sha
774edcacfa DBZ-6522 Resume token handling for documents with complex ids 2024-06-19 13:27:01 +02:00
Jakub Cechacek
ab9e85a59a DBZ-7862 Updated MongoDB connector docs to reflect replica_set connection mode removal 2024-05-24 20:04:22 +02:00
Jiri Pechanec
b052d8a472 DBZ-7880 Make schema backward compatible 2024-05-22 14:20:02 +02:00
Jiri Pechanec
c09c2c6712 DBZ-7641 Update tests 2024-05-15 05:42:12 +02:00
mfvitale
c3a8ba4afb DBZ-7828 Correct filter query in snapshot task to handle SQLServer table quote 2024-05-10 07:42:09 +02:00
ani-sha
a26281d837 DBZ-7138 Provide separate method for stripping parenthesis 2024-04-23 14:38:29 +02:00
ani-sha
fbe31a77cd DBZ-7138 Additional Conditions for MongoDb Incremental Snapshot 2024-04-23 14:38:29 +02:00
ani-sha
4f7a5c9fa6 DBZ-7485 Refine testing for database and collection filters in MongoDb 2024-04-22 11:31:14 +02:00
twthorn
261253d9f0 DBZ-7698 Refactor to generic transaction block customization 2024-04-19 10:23:30 +02:00
twthorn
24e4e4eb80 DBZ-7698 fix some failing tests 2024-04-19 10:23:30 +02:00
ismail simsek
f7c4511432 DBZ-7695 add _id field to tombstone rewrite events, for delete events 2024-04-18 14:53:02 +02:00
ani-sha
5fad9e57a7 DBZ-7759 Use Testing.print method 2024-04-12 13:49:16 +02:00
ani-sha
5401041514 DBZ-7759 Modify test to ensure streaming of events smoothly 2024-04-12 13:49:16 +02:00
Jakub Cechacek
725041d62f DBZ-7759 Await fetcher thread shutdown when BufferingChangeStreamCursor gets closed 2024-04-12 13:49:16 +02:00
ani-sha
6b2f275142 DBZ-7759 Add appropriate logging 2024-04-12 09:19:05 +02:00
ani-sha
719d2325c5 DBZ-7759 Suggestions for code review to simplyfy the error handling 2024-04-12 09:19:05 +02:00
ani-sha
aef55a1ccd DBZ-7759 Use MongoException in error handling 2024-04-12 09:19:05 +02:00
ani-sha
1b7481d357 DBZ-7759 Ensure error handling of oversized events swallowed by BufferedChangeStreamCursor thread 2024-04-12 09:19:05 +02:00
Jakub Cechacek
4061a02f8b DBZ-7770 Added test verifying invalid log position is properly logged for MongoDB 2024-04-11 14:58:07 +02:00
Jakub Cechacek
9242d45240 DBZ-7770 Ability to disable log position validation on startup for MongoDb conenctor 2024-04-11 14:58:07 +02:00
Jakub Cechacek
6e9235215f DBZ-7770 Fixed the return values of MongoDbConnection#isValidResumeToken() 2024-04-11 14:58:07 +02:00
Roman Kudryashov
50c9f042a3 DBZ-7755 Refactor exporting to CloudEvents: remove RecordParser 2024-04-08 13:22:31 +02:00
mfvitale
ae7628a732 DBZ-7732 Add ConnectorSpecific annotation to manage multiple SPI connector specific implementation 2024-04-08 12:46:53 +02:00
Jakub Cechacek
1bf9e802f1 DBZ-7744 RS name is not required when cluster type is not REPLICA_SET instead of when cluster type is SHARDED in order to accomodate LOAD_BALANCED clusters 2024-04-05 09:40:35 -04:00
mfvitale
213456c5af DBZ-7497 Properties related to configuration based snapshot mode are now optional 2024-03-28 08:59:54 +01:00
mfvitale
ee5f25ccc9 DBZ-7497 Add a configuration based snapshot modes configurable via connector properties 2024-03-28 08:59:54 +01:00
mfvitale
689818aec3 DBZ-7308 Remove MongoDbSnapshotterServiceProvider.java 2024-03-25 05:46:46 +01:00
mfvitale
c7a28021a6 DBZ-7308 Remove connector specific SnapshotLockProvider and SnapshotterServiceProvider 2024-03-25 05:46:46 +01:00
mfvitale
3c4eed0918 DBZ-7304 Add tests for Always and Custom snapshot mode for MongoDb connector 2024-03-21 08:23:30 +01:00
mfvitale
3962e34664 DBZ-7304 Align snapshot modes for MongoDb connector 2024-03-21 08:23:30 +01:00
harveyyue
48c67e40fb DBZ-7618 Implement Versioned interfaces in Transformation and Converter plugins 2024-03-14 06:35:42 +01:00
mfvitale
8a47b2333d DBZ-7588 Disable debug prints on tests 2024-03-02 11:33:48 -05:00
mfvitale
ce17bdcce6 DBZ-7461 Add generic properties for redo log archive and retention 2024-03-01 14:12:31 +01:00
mfvitale
4ade54351f DBZ-7461 Rename SCHEMA_ONLY_RECOVERY to RECOVERY and SCHEMA_ONLY to NO_DATA 2024-03-01 14:12:31 +01:00
ani-sha
04061eeaf5 DBZ-6434 Handle NPE due to missing credentials 2024-03-01 10:25:01 +01:00
Bue-Von-Hun
94e24532f6 DBZ-2015 Remove redundant fields 2024-02-27 15:30:48 +01:00
rkerner
1e69e40ec2 DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Moved deduplication from Map to LinkedHashSet.
+ minor fixes added for cleanup and centralization of common code

closes https://issues.redhat.com/browse/DBZ-7416
2024-02-22 13:34:08 -05:00
Jiri Pechanec
68b6591142 DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Moved deduplication from Map to LinkedHashSet.
closes https://issues.redhat.com/browse/DBZ-7416
2024-02-22 13:34:08 -05:00
harveyyue
82f5e6ea77 DBZ-7480 Allow special characters in signal table name 2024-02-19 11:43:20 +01:00
Chris Cranford
928aa26cdb DBZ-7107 Fix source info test failures 2024-02-16 12:52:20 +01:00
Vojtech Juranek
4452e3d095 DBZ-7024 Move EmbeddedConfig into separate class
`EmbeddedConfig` needs to be shared with other implmentations of
`DebeziumEngine` as long as Debezium embedded depends on the Kafka
model.
2024-02-12 13:43:21 +01:00
Jakub Cechacek
7f3210214c DBZ-7451 Support for standard BSON representation of UUID in incremental snapshots 2024-02-12 13:19:13 +01:00