Debezium Builder
3e2d75f0da
[maven-release-plugin] prepare release v2.5.0.Beta1
2023-12-04 13:55:35 +00:00
Jakub Cechacek
355041dbaf
DBZ-7184 buffer lock is now acquired with timeout and reattempted on failure
2023-12-04 06:58:28 +01:00
Jakub Cechacek
88a64bf3d5
DBZ-7184 Resumable events are now guaranteed to contain resume token
2023-12-04 06:58:28 +01:00
Jakub Cechacek
9ad6f9c7db
DBZ-7184 Streaming loop now always uses resumable event to get resume token
2023-12-04 06:58:28 +01:00
Jakub Cechacek
8a5007a738
DBZ-7184 Refactored streaming loop of MongoDB connector
2023-12-04 06:58:28 +01:00
Jakub Cechacek
c7e87c5382
DBZ-7184 Using buffering cursor in straming change event source
2023-12-04 06:58:28 +01:00
Jakub Cechacek
93e8570fb6
DBZ-7184 Implementation of buffering change stream cursor
2023-12-04 06:58:28 +01:00
Jakub Cechacek
2c8277fcf6
DBZ-7162 Removed duplicit source event count metric for MongoDB
2023-12-01 11:38:21 +01:00
Jakub Cechacek
79230c6014
DBZ-7206 Always check for blocking snapshot pause request when processing change events in MongoDB connector
2023-11-29 14:56:41 +01:00
“vsantonastaso”
8c1c369449
DBZ-6878 add table specific notification in initial snapshot
2023-11-29 08:32:21 +01:00
Jakub Cechacek
0ff0ff54ae
DBZ-7186 Removing unsued offset parameter
2023-11-28 07:27:34 +01:00
Jakub Cechacek
03e305e835
DBZ-7184 Extracted split event handling from streaming source
2023-11-28 07:27:34 +01:00
Jakub Cechacek
3f0eeebf10
DBZ-7184 Extracted change stream initialisation
2023-11-28 07:27:34 +01:00
Roman Kudryashov
1992c1e7e4
DBZ-7159 Fail fast during deserialization if a value is not a CloudEvent
2023-11-23 14:19:37 +01:00
Jiri Pechanec
e57c607bcf
DBZ-6762 Rename to general name
2023-11-22 06:10:40 +01:00
rkerner
aeaba3ae70
DBZ-6762 Add "validate filters" endpoint/s for connector-specific Connect REST Extensions
...
closes to https://issues.redhat.com/browse/DBZ-6762
2023-11-22 06:10:40 +01:00
Jakub Cechacek
ae6e7d03cc
DBZ-7157 Unified MongoDB document key serialisation for snapshot and streaming
2023-11-21 13:29:12 +01:00
Jakub Cechacek
fb23ce13a1
DBZ-7157 JsonSerialization#getDocumentIdChangeStream now uses the same implementation as getDocumentIdSnaphsot
2023-11-21 13:29:12 +01:00
Jakub Cechacek
d2eca450d5
DBZ-7157 Added test case for key generation with composite document ids
2023-11-21 13:29:12 +01:00
mfvitale
2f8482d284
DBZ-7139 Use namespace instead of identifier for matching collections to be snapshotted
2023-11-21 11:13:39 +01:00
Jakub Cechacek
687f2858e3
DBZ-7162 Added various streaming metrics to MongoDB connector
2023-11-21 11:09:09 +01:00
Jakub Cechacek
7d4ca05ef6
DBZ-7162 Passing streaming metrics to MongoDbStreamingChangeEventSource
2023-11-21 11:09:09 +01:00
Jakub Cechacek
17464f5fe2
DBZ-7146 Configurable inactivity pause duration in MongoDB connector
2023-11-20 13:59:44 +01:00
Jiri Pechanec
e654dc228f
DBZ-7066 Code reorganization
2023-11-14 11:56:16 +01:00
harveyyue
41f9423caf
DBZ-7066 Add test cases synchronously from "DBZ-6809"
2023-11-14 11:56:16 +01:00
harveyyue
223b6e4335
DBZ-7066 Extract common creating transformation methods to ConnectRecordUtil
2023-11-14 11:56:16 +01:00
harveyyue
4c3b5540ec
DBZ-7066 Add NewExtractNewDocumentStateTest and NewExtractNewDocumentStateTestIT for new option "delete.tombstone.handling.mode"
2023-11-14 11:56:16 +01:00
harveyyue
5e8287eb40
DBZ-7066 The DefaultDeleteHandlingStrategy couldn't add the rewrite "__deleted" field to a non-struct value
2023-11-14 11:56:16 +01:00
Jiri Pechanec
ef69dbe91c
DBZ-6809 Fix imports
2023-11-10 16:06:43 +01:00
Jakub Cechacek
e88d00badb
DBZ-6809 Added test for ExtractNewDocumentState with avro enabled on connector and transformation levels
2023-11-10 16:06:43 +01:00
Debezium Builder
1521445908
[maven-release-plugin] prepare for next development iteration
2023-11-10 10:26:05 +00:00
Debezium Builder
6c6f6e9138
[maven-release-plugin] prepare release v2.5.0.Alpha2
2023-11-10 10:26:05 +00:00
Jiri Pechanec
9d84d01f64
DBZ-6726 Switch to ArrayList for repeated iteration
2023-11-09 07:57:12 +01:00
Jakub Cechacek
89b07a013e
DBZ-6726 support for $changeStreamSplitLargeEvent aggregation in MongoDB connector
2023-11-09 07:57:12 +01:00
Jakub Cechacek
bddf8802ec
DBZ-7108 Switched default connection mode for sharded clusters from replica_set to sharded
2023-11-07 10:36:53 +01:00
Vojtech Juranek
11d2ff0b9b
DBZ-7007 Use better name for auxiliary variable detecing if engine is running
2023-11-06 10:50:21 +01:00
Vojtech Juranek
fdab2cd61b
DBZ-7007 Remove engine.isRunning() from testsuite
2023-11-06 10:50:21 +01:00
Jakub Cechacek
a3affaf6a8
DBZ-7058 Fixed index handling in FieldSelector
2023-11-02 13:29:39 +01:00
Debezium Builder
78c5204444
[maven-release-plugin] prepare for next development iteration
2023-10-26 15:39:35 +00:00
Debezium Builder
356e488e83
[maven-release-plugin] prepare release v2.5.0.Alpha1
2023-10-26 15:39:35 +00:00
Roman Kudryashov
b0d7d60b5d
DBZ-7016 Optimize retrieval of id and type
2023-10-19 11:35:18 +02:00
Roman Kudryashov
22f6b083b1
DBZ-7016 Change metadata.location property syntax
2023-10-19 11:35:18 +02:00
Roman Kudryashov
ee64c70546
DBZ-7016 id
and type
of a CloudEvent can be retrieved from headers
2023-10-19 11:35:18 +02:00
harveyyue
4836fcf1ea
DBZ-6907 Introduce a new handling deleted record option "delete.tombstone.handling.mode"
2023-10-19 09:35:15 +02:00
harveyyue
fccaf37549
DBZ-6907 Remove unused method from ExtractNewDocumentState
2023-10-19 09:35:15 +02:00
harveyyue
f64ea1a5f8
DBZ-6907 Extract an abstract transformation class from ExtractNewRecordState
2023-10-19 09:35:15 +02:00
Roman Kudryashov
01485aceb3
DBZ-6982 Provide configuration option to exclude extension attributes from a CloudEvent
2023-10-04 13:58:53 +02:00
Debezium Builder
40a8f4026d
[maven-release-plugin] prepare for next development iteration
2023-10-03 14:18:54 +00:00
Debezium Builder
412c6f7979
[maven-release-plugin] prepare release v2.4.0.Final
2023-10-03 14:18:54 +00:00
Jakub Cechacek
9b3b718612
DBZ-6973 Fixed insufficient test isolation
2023-10-02 08:25:53 +02:00
Jakub Cechacek
baafe602ce
DBZ-6973 Literal matching mode for database/collection filters and ability to override internal aggregation pipeline for MongoDB conenctor
2023-10-02 08:25:53 +02:00
mfvitale
56795732b5
DBZ-6793 Add timestamp for notifications into documentation
2023-09-28 14:20:17 -04:00
Roman Kudryashov
98c426dec4
DBZ-3642 CloudEventsConverter can retrieve metadata info from headers
2023-09-26 13:06:09 +02:00
Debezium Builder
cc1de0aa51
[maven-release-plugin] prepare for next development iteration
2023-09-22 09:00:45 +00:00
Debezium Builder
6809978083
[maven-release-plugin] prepare release v2.4.0.CR1
2023-09-22 09:00:45 +00:00
mfvitale
5d7ba3e951
DBZ-6957 Remove spaces in Signal and Notification JMX ObjectName
2023-09-21 12:04:33 -04:00
rkerner
6d3528d4fb
DBZ-4395 Add connector specific Debezium Connect REST Extension/s and move logic from UI backend to the Debezium Connect REST Extension (part 2 of the re-architecturing/re-factoring)
...
DBZ-6761 Add "validate connection" endpoints in connector specific Connect REST extensions
closes to https://issues.redhat.com/browse/DBZ-4395
closes to https://issues.redhat.com/browse/DBZ-6761
2023-09-21 13:29:59 +02:00
mfvitale
ae199d2053
DBZ-6828 Blocking snapshot will generate schema events only for signaled tables
2023-09-20 08:18:33 +02:00
Jiri Pechanec
a064fc42e5
DBZ-6518 Fix imports
2023-09-13 13:00:03 +02:00
Yue Wang
199bf775ed
DBZ-6518 use multipe threads to query chunks
2023-09-13 13:00:03 +02:00
Debezium Builder
65169a9315
[maven-release-plugin] prepare for next development iteration
2023-09-13 09:41:46 +00:00
Debezium Builder
845e3e9d09
[maven-release-plugin] prepare release v2.4.0.Beta2
2023-09-13 09:41:46 +00:00
Jakub Cechacek
fd5c8353b2
DBZ-6883 Documenting oversized document handling
2023-09-13 11:04:07 +02:00
Jakub Cechacek
9aadc38414
DBZ-6892 Removing all code related to previously deprecated MongoDB conenctor configuration
2023-09-12 13:03:40 +02:00
Jakub Cechacek
eaa24ea91e
DBZ-6871 MongoDB connector is now able to skip events for oversized documents
2023-09-11 15:28:27 +02:00
Jakub Cechacek
3efe8c0e51
DBZ-6872 Ability to switch the order of user defined and internal aggregation pipeline used by MongoDB connector when streaming changes
2023-09-11 13:15:10 +02:00
Nir Levy
454a9ac914
DBZ-6741 Support custom authentication on MongoDB connector
2023-09-07 14:49:04 +02:00
Jiri Pechanec
41135efa74
DBZ-6773 Remove typo
2023-09-06 15:08:24 +02:00
Jakub Cechacek
9fad9b6895
DBZ-6773 Adding support for new field/header mappings to ExtractNewDocumentState transformation
2023-09-06 15:08:24 +02:00
Jakub Cechacek
842e92bc52
DBZ-6773 Cleaning unused fields and variables in tests
2023-09-06 15:08:24 +02:00
Jakub Cechacek
5a20419a98
DBZ-6867 streaming pipeline should match events from collection include list as well as signal collection
2023-09-04 13:28:20 +02:00
Debezium Builder
d97a18e8dc
[maven-release-plugin] prepare for next development iteration
2023-08-29 11:41:08 +00:00
Debezium Builder
5ce25f2cfe
[maven-release-plugin] prepare release v2.4.0.Beta1
2023-08-29 11:41:08 +00:00
“vsantonastaso”
43adc59292
DBZ-6742 use JSON format for JMX Notitication userData
2023-08-22 06:42:51 +02:00
Jiri Pechanec
fdafbc07cf
DBZ-6774 Use private modifier
2023-08-16 10:20:49 +02:00
harveyyue
625eec5aa0
DBZ-6774 MongoDB New Document State Extraction: nonexistent field for add.headers
2023-08-16 10:20:49 +02:00
ericpangiawan
5407d42014
DBZ-6725 Add Formatting
2023-08-15 13:34:17 +02:00
ericpangiawan
bc77f2488e
DBZ-6725 Include beforeRecord if the delete handling mode is REWRITE
...
Adding Integration Test and the feature to add beforeRecord in delete events of event flattening with delete handling mode rewrite. Works only if the collection has ChangeStreamPreAndPostImagesOptions turned on and capture mode is *_with_pre_image (MongoDB 6.0 or later)
2023-08-15 13:34:17 +02:00
mfvitale
542b3619fa
DBZ-6731 Blocking snapshot takes configuration from signal payload
2023-08-15 12:43:43 +02:00
harveyyue
063244a0ad
DBZ-6760 Outbox transformation can cause connector to crash
2023-08-10 08:59:43 +02:00
Jiri Novotny
0d18eb6768
DBZ-6779 Change approach to version parsing
2023-08-10 06:59:36 +02:00
Jiri Novotny
554163a3f1
DBZ-6779 Fix version parsing of non-numerical chars
2023-08-10 06:59:36 +02:00
Debezium Builder
3c72f071b0
[maven-release-plugin] prepare for next development iteration
2023-08-09 10:33:06 +00:00
Debezium Builder
915c465440
[maven-release-plugin] prepare release v2.4.0.Alpha2
2023-08-09 10:33:06 +00:00
mfvitale
524987227f
DBZ-6566 Shutdown blockingSnapshotExecutor when ChangeEventSourceCoordinator stops
2023-08-03 09:22:24 +02:00
mfvitale
61651a94b5
DBZ-6566 Force blocking snapshot even if a snapshot has already executed
2023-08-03 09:22:24 +02:00
mfvitale
3718d9818e
DBZ-6566 Support Blocking snapshot for MongoDb
2023-08-03 09:22:24 +02:00
jcechace
699a1612ae
DBZ-6670 Replace error handler instead of recreate in order to keep retry counter
2023-07-25 13:46:56 +02:00
jcechace
0fbad0c60c
DBZ-6700 using hello command instead of ping as resume backup in order to ensure that operationTime field is present across supported MongoDB platforms
2023-07-24 10:00:16 -04:00
Debezium Builder
267dfda8a6
[maven-release-plugin] prepare for next development iteration
2023-07-17 10:40:52 +00:00
Debezium Builder
0d3cef6a29
[maven-release-plugin] prepare release v2.4.0.Alpha1
2023-07-17 10:40:52 +00:00
harveyyue
5bc7fd1f7c
DBZ-6603 Support for custom tags in the connector metrics
2023-07-13 09:52:33 +02:00
jcechace
c620081cd1
DBZ-6182 Simplifying inheritance hierarchy in MongoDB tests
2023-06-28 09:51:51 +02:00
jcechace
d1e77e6165
DBZ-6182 Allowing mongodb connector to run with database restricted privileges
2023-06-28 09:51:51 +02:00
mfvitale
d5febcc054
DBZ-6417 Provide extensible signal action mechanism through SPI
2023-06-23 15:03:27 +02:00
mfvitale
34e28ac52d
DBZ-6416 Add initial snapshot notifications
2023-06-22 13:22:34 +02:00
jcechace
ee7586e6d8
DBZ-6578 Configurable connection string parameters for individual shard connections when using connection.mode=replica_set with sharded clusters
2023-06-21 12:22:23 +02:00
jcechace
8f70de42c1
DBZ-6578 Cleaning up connection string propagation between discovery and tasks
2023-06-21 12:22:23 +02:00
jcechace
dd97456877
DBZ-6578 Honoring connection string read preference configuration
2023-06-21 12:22:23 +02:00
Debezium Builder
60f730d383
[maven-release-plugin] prepare for next development iteration
2023-06-20 13:21:31 +00:00
Debezium Builder
150396e4c8
[maven-release-plugin] prepare release v2.3.0.Final
2023-06-20 13:21:31 +00:00
Jiri Pechanec
067683e56a
DBZ-6076 Consistent config for all connectors
2023-06-15 11:55:28 +02:00
Anil Dasari
0431a8b9fc
DBZ-6076: Add support for custom SourceInfoStructMaker for adding new fields to source field
2023-06-15 11:55:28 +02:00
kate
24d3425fdf
DBZ-6552 Notify when incremental snapshot fails
2023-06-12 15:14:40 +02:00
Debezium Builder
5d1b00588a
[maven-release-plugin] prepare for next development iteration
2023-06-09 09:23:38 +00:00
Debezium Builder
d1e6107517
[maven-release-plugin] prepare release v2.3.0.CR1
2023-06-09 09:23:37 +00:00
Ronak Jain
3f3967c2bb
DBZ-6366: Code Improvements for skip.messages.without.change
...
Addresses/Closes DBZ-6366
2023-06-05 14:50:25 +02:00
Debezium Builder
aa5b2131fc
[maven-release-plugin] prepare for next development iteration
2023-05-26 10:35:20 +00:00
Debezium Builder
9cc9b3c29a
[maven-release-plugin] prepare release v2.3.0.Beta1
2023-05-26 10:35:20 +00:00
mfvitale
547ecd02ca
DBZ-6447 Fix incremental snapshot notification on MongoDb connector
2023-05-23 09:57:05 +02:00
mfvitale
dc00f64bbf
DBZ-6447 Incremental snapshot notification ids are now equal to the id sent in the execute-snapshot
signal.
2023-05-23 09:57:05 +02:00
jcechace
2de3f0d9a4
DBZ-6421 Ability to recover from cursor which doesn't return resume token when there is no change document (e.g. in case of DocumentDB connection)
2023-05-15 14:08:57 +02:00
jcechace
5dfe52dbc4
DBZ-6421 Streaming pipeline is now compatible with DocumentDB
2023-05-15 14:08:57 +02:00
jcechace
dad7d0ce14
DBZ-6421 Ability to run tests against external deployment of MongoDB and AWS DocumentDB specifically
2023-05-15 14:08:57 +02:00
Debezium Builder
a5b44767bc
[maven-release-plugin] prepare for next development iteration
2023-05-11 11:05:24 +00:00
Debezium Builder
1b1a7531f2
[maven-release-plugin] prepare release v2.3.0.Alpha1
2023-05-11 11:05:24 +00:00
mfvitale
7429b4e6f1
DBZ-1973 Add incremental snapshot notifications
2023-05-10 12:22:02 +02:00
mfvitale
811b800f55
DBZ-1973 Add support for Notifications
...
DBZ-1973 Add more tests for other connectors
DBZ-1973 Add send method with offset parameter
DBZ-1973 Instantiate NotificationService in the task class
DBZ-1973 Instantiate NotificationService in the task class
DBZ-4027 Move specific sink channel configuration to SinkNotificationChannel
DBZ-4027 Remove not used SPI file
DBZ-1973 Moved SPI file definition to debezium-core
DBZ-1973 Rename KafkaNotificationChannel to more generic SinkNotificationChannel
DBZ-1973 Code refactor
DBZ-1973 Improve configuration property description
DBZ-1973 Improve test
DBZ-1973 Add close method to NotificationChannel
DBZ-1973 Implement KafkaNotificationChannel
DBZ-1973 Add NotificationService and LogNotificationChannel
2023-05-10 12:22:02 +02:00
mfvitale
952340286e
DBZ-4027 Make signalling channel configurable
...
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Code style
DBZ-4027 Make SignalPayload more generic and extensible
DBZ-4027 Rename DatabaseSignalChannel to SourceSignalChannel
DBZ-4027 Improve logging
DBZ-4027 Moved SPI file definition to debezium-core
DBZ-4027 Move SignalProcessor synchronization point to be processed only when a signal cdc event arrives.
DBZ-4027 Add EventDispatcher constructor without signalProcessor for spanner connector
DBZ-4027 Fix NPE
DBZ-4027 Fix NPE
DBZ-4027 Formatting
DBZ-4027 Correctly manage signal on not supported connector
DBZ-4027 Use the correct MongoDbOffset
DBZ-4027 Correctly initialize offset for Oracle and SqlServer connectors
DBZ-4027 Register SPI implementations
DBZ-4027 Improve SignalProcessor instantiation
DBZ-4027 Pass source info in case of SchemaChanges action
DBZ-4027 Manage close event in a synchronous way
DBZ-4027 Correctly init offset context also in case of snapshot mode 'never'
DBZ-4027 Fix MySqlMetricsIT test
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Set pass offset context after initial snapshot to SignalProcessor
DBZ-4027 Pass OffsetContext to signal processor
DBZ-4027 Pass CommonConnectorConfig to SignalChannelReader init method
DBZ-4027 Move Incremental snapshot window actions to dedicated package
DBZ-4027 Align SignalsIT test with new code
DBZ-4027 Fix SignalsIT test
DBZ-4027 Fix SignalProcessor scheduling
DBZ-4027 Moved DatabaseSignalChannel and SignalChannelReader to dedicated package
DBZ-4027 Start SignalProcessor from ChangeEventSourceCoordinator
DBZ-4027 Create SignalProcessor and renamed Signal to DatabaseSignalChannel
DBZ-4027 Initial refactoring of signal feature
2023-05-05 13:35:40 +02:00
ani-sha
a453c3ca19
DBZ-6259 Add missing description of various snapshot modes in the connector field definitions
2023-05-01 11:03:04 -04:00
jcechace
d3d194f34d
DBZ-6402 Using MongoCommandException to detect invalid change stream resume token in order to perform snapshot properly
2023-04-29 14:06:55 -04:00
Debezium Builder
bb1b79f3bd
[maven-release-plugin] prepare for next development iteration
2023-04-20 09:09:43 +02:00
Debezium Builder
5483b4de99
[maven-release-plugin] prepare release v2.2.0.Final
2023-04-20 09:09:43 +02:00
jcechace
69ee48dd95
DBZ-4427 Reverting trace logging configuration in MongoDB tests
2023-04-19 16:03:43 +02:00
jcechace
8e1d83ce88
DBZ-4427 Added sharding log messages to container infrastructure
2023-04-19 12:05:59 +02:00
jcechace
753c900a15
DBZ-4427 basic test for incremental snapshot support with sharded MongoDB deployments
2023-04-19 12:05:59 +02:00
Debezium Builder
976a47a3a3
[maven-release-plugin] prepare for next development iteration
2023-04-14 08:46:06 +00:00
Debezium Builder
1ca8399e8b
[maven-release-plugin] prepare release v2.2.0.CR1
2023-04-14 08:46:06 +00:00
Debezium Builder
5c650fd054
[maven-release-plugin] prepare for next development iteration
2023-03-31 10:00:16 +00:00
Debezium Builder
68ffc4833b
[maven-release-plugin] prepare release v2.2.0.Beta1
2023-03-31 10:00:16 +00:00
jcechace
2fdd3fbc86
DBZ-6170 Added basic test scenario for running mongodb connector with sharded mongodb cluster
2023-03-27 10:31:47 +02:00
jcechace
033396bf80
DBZ-6170 Support for connecting to mongodb sharded clusters either with single connection to mongos or to each shard individually
2023-03-27 10:31:47 +02:00
Vojtech Juranek
f32bf343c7
DBZ-6216 Remove obsolete Mongo notes
...
Mongo tests were migrated to testcontainers and Mongo is not run via
Maven plugin or manually any more.
2023-03-23 13:29:19 +01:00
jcechace
a393930de9
DBZ-5987 Removing unnecessary getOffset() call during streaming
2023-03-16 13:20:19 +01:00
jcechace
a536952779
DBZ-5987 Code cleanup in MongoDb SourceInfo
2023-03-16 13:20:19 +01:00
jcechace
0f10c71642
DBZ-5987 Using consistent approach when extracting change event timestampt acroos different event types
2023-03-16 13:20:19 +01:00
jcechace
6d7b655273
DBZ-5987 cleanup of connection management in *ChangeEventSource classes
2023-03-16 13:20:19 +01:00
jcechace
04dbae17ad
DBZ-5987 Cleaning up and updating its test coverage
2023-03-16 13:20:19 +01:00
jcechace
a39c084259
DBZ-5987 Replacing remaining usages of oplog with resume tokens retrieved from either change stream even or cursor
2023-03-16 13:20:19 +01:00
jcechace
bfdba1fb25
DBZ-5987 Removing obsolete ReplicaSetChangeStreamsContext from MongoDbStreamingChangeEventSource in order to simplify the implementation
2023-03-16 13:20:19 +01:00
Debezium Builder
55d65e9618
[maven-release-plugin] prepare for next development iteration
2023-03-08 11:38:17 +00:00
Debezium Builder
6d183c3dc1
[maven-release-plugin] prepare release v2.2.0.Alpha3
2023-03-08 11:38:16 +00:00
Jiri Pechanec
98dcdb8f55
DBZ-6023 Remove nnecessary code
2023-03-06 10:25:59 +01:00
PlugaruT
472162e46e
DBZ-6023 Add support for providing a surrogate key when triggering incremental snapshots
...
For tables that have a composite Primary Key, SQL queries for incremental snapshot perform bad. This change allows to provide a surrogate key that will be used for the queries. The key should be unique in order for the snapshot to guarantee consistency.
2023-03-06 10:25:59 +01:00
ggaborg
065dc997b9
DBZ-823 Optionally parallelize initial snapshots
2023-03-03 12:47:15 +01:00
jcechace
aedb4fdd6a
DBZ-6032 Making MongoDbConnectorConfig.REPLICA_SETS internal
2023-03-03 05:46:33 +01:00
jcechace
7d3cca89bd
DBZ-6032 MongoConnection allways rethrows InterruptedException
2023-03-03 05:46:33 +01:00
jcechace
67d3f5b373
DBZ-6032 Removing obsolete retry configuration
2023-03-03 05:46:33 +01:00
jcechace
f305bcd9ab
DBZ-6032 Further simplifying connection logic
2023-03-03 05:46:33 +01:00
jcechace
ecdb4729af
DBZ-6032 Introducing connection package to mongo connector
2023-03-03 05:46:33 +01:00
jcechace
922191cdd7
DBZ-6032 Connection management refactoring
2023-03-03 05:46:33 +01:00
Bobby Tiernay
fe3135f28d
DBZ-6131: Support change stream filtering using MongoDBs aggregation pipeline step.
2023-02-28 20:17:05 +01:00
Bobby Tiernay
37a6fea350
DBZ-6152: Remove hardcoded list of system database exclusions that are not required for change streaming.
2023-02-24 09:43:39 +01:00
Bobby Tiernay
5d80712781
DBZ-5201: Add MongoDB connector support for server-side filtering of replica set change streams.
...
This change migrates from doing client-side filtering of streaming
change events to performing it server-side via MongoDB's support
for aggregation pipelines. This can drastically cut down on the amount
of data sent between the client and server. In addition, a new approach
to heartbeating has been added which relies on the `postBatchResumeToken`
to advance the the cursor even when no changes occur on the pipeline.
2023-02-23 18:14:08 +01:00
Xinbin Huang
76a8f8dbe1
DBZ-6118: Update ExtractNewDocumentState to parse change stream output
2023-02-23 09:09:04 +01:00
Xinbin Huang
2c57b93384
DBZ-6118: Cleanup Mongo oplog code logic
2023-02-23 09:09:04 +01:00
Eugene Abramchuk
fcc44edf0b
DBZ-6113 Correct max reconnect attempts
...
Originally there are max+1 attempts
2023-02-21 08:10:26 +01:00
Xinbin Huang
a3d75e798a
DBZ-6116: Support String type for key in Mongo incremental snapshot
2023-02-20 13:55:54 +01:00
Debezium Builder
2245ceda44
[maven-release-plugin] prepare for next development iteration
2023-02-16 11:28:25 +00:00
Debezium Builder
20f3b16e8a
[maven-release-plugin] prepare release v2.2.0.Alpha2
2023-02-16 11:28:25 +00:00
Yang Wu
e98089e9a3
DBZ-6038 Expose walltime from mongo record parser
2023-02-14 09:48:17 +01:00
harveyyue
57b902ff8c
DBZ-5743 Introduce alternative AVRO_UNICODE mode for field name selector
2023-02-02 13:21:45 +01:00
jcechace
e982033961
DBZ-6064 masking sensitive information in connection string on close
2023-02-01 11:05:16 +01:00
jcechace
0918c34b38
DBZ-5973 Support for more types as key in mongodb incremental snapshot implementation
2023-02-01 08:47:14 +01:00
Yang Wu
2a366cf398
DBZ-6038 Add wallTime in mongo source info
2023-01-25 10:30:32 +01:00
Debezium Builder
04f5291a2d
[maven-release-plugin] prepare for next development iteration
2023-01-19 11:32:10 +00:00
Debezium Builder
027527963e
[maven-release-plugin] prepare release v2.2.0.Alpha1
2023-01-19 11:32:10 +00:00
Jiri Pechanec
8ca5e0a9b5
DBZ-4669 Reformat after plugin version consolidation
2023-01-11 06:44:53 +01:00
Mark Lambert
e5d16b14a6
DBZ-2439 Remove redundant modifiers
2023-01-11 06:44:53 +01:00
Debezium Builder
cf1720e4cb
[maven-release-plugin] prepare for next development iteration
2022-12-22 10:11:54 +00:00
Debezium Builder
b4ea9b46cd
[maven-release-plugin] prepare release v2.1.0.Final
2022-12-22 10:11:53 +00:00
jcechace
351a0d73fe
DBZ-4339 Preferring secondaries to perform incremental snapshot where possible
2022-12-20 15:03:17 +01:00
jcechace
3966a4099a
DBZ-4339 MongoDbIncrementalSnapshotChangeEventSource uses MongoPreferredNode
2022-12-20 15:03:17 +01:00
jcechace
27cea0e5e6
DBZ-4339 Preferring secondaries to perform snapshot
2022-12-20 15:03:17 +01:00
jcechace
44ae4697d7
DBZ-4339 Preferring secondaries for opening mongodb change stream during streaming phase
2022-12-20 15:03:17 +01:00
jcechace
434322256c
DBZ-4339 Generalising MongoPrimary into MongoPreferredNode
2022-12-20 15:03:17 +01:00
harveyyue
03933f61ea
DBZ-5818 field.exclude.list in MongoDB Connector v2.0 doesn't accept * as a wildcard for collectionName
2022-12-19 13:39:37 +01:00
Debezium Builder
177a60a9d3
[maven-release-plugin] prepare for next development iteration
2022-12-16 09:33:38 +00:00
Debezium Builder
f805db8c3c
[maven-release-plugin] prepare release v2.1.0.Beta1
2022-12-16 09:33:38 +00:00
jcechace
60910872b9
DBZ-5857 Using single member replica as default
2022-12-16 08:43:25 +01:00
jcechace
c9ef1136b5
DBZ-5857 Excluding JUnit5 from debezium-testing-testcontainers dependency in mongo module
2022-12-16 08:43:25 +01:00
jcechace
0951f9994f
DBZ-5857 Migrated MongoDB integration tests to test-containers
2022-12-16 08:43:25 +01:00
Vojtech Juranek
9494552a3a
DBZ-5838 Add missing Apicurio converter dependency
2022-12-12 14:44:05 +01:00
Marcelo Avancini
7754163c5d
DBZ-2432: Fixing misuse of assertj
...
- Some usages of assertj was not callig the actual assert of the DSL, fixing it
- Changing the Assertions to be staticlly imported
2022-12-08 13:10:26 +01:00
Vojtech Juranek
e1b65cd6f7
DBZ-5889 Run integration tests in specified order
...
Run the tests alwyas in thr same order to make it more easy to debug
failures. If needed, the order can be changed (e.g. to `random`) by
overriding propeperty `runOrder`.
2022-12-08 13:00:41 +01:00
Jiri Pechanec
97ea30f25d
DBZ-5893 Mask connection string in connector option logging
2022-12-07 08:41:34 +01:00
jcechace
2468d73783
DBZ-5893 Masking credentials in connection string when logging
2022-12-07 05:45:58 +01:00
Debezium Builder
81896ed334
[maven-release-plugin] prepare for next development iteration
2022-11-30 07:52:24 +00:00
Debezium Builder
1aac97b87c
[maven-release-plugin] prepare release v2.1.0.Alpha2
2022-11-30 07:52:23 +00:00
Bobby Tiernay
adb672a87c
DBZ-4339 Migrate from JUnit 4 to 5, rename utility class, fix formatting.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
950a2a8bdf
DBZ-4339 Migrate MongoDB Testcontainer support to debezium-testing-testcontainers
module.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
5c94151b1f
DBZ-4339 Make mongosh
more resilient to various output conditions.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
849347b7ec
DBZ-4339 Remove reliance on MongoDB driver in non-IT code.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
3ced3afa9f
DBZ-4339 Add polish, consistency, Java Doc, and DRY duplicated code elements.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
8f6abd4428
DBZ-4339 Remove need for container names and fixed ports on Linux.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
b6712c7ccc
DBZ-4339 Convert to use IPs for linux MongoClient
s.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
8c8804665f
DBZ-4339 Fix errant shard host names and add additional java docs / assertions.
2022-11-30 07:00:56 +01:00
Bobby Tiernay
6f9ee66678
DBZ-4339 Add MongoDB Testcontainers support for functional testing.
2022-11-30 07:00:56 +01:00
Erdinç Taşkın
97548591d6
[optimization] Prevent unnecessary list creation
...
Prevent unnecessary list creation
2022-11-29 14:08:27 +01:00
Vojtech Juranek
79c47334c2
DBZ-5834 Allow to add updateDescription fields
...
Allow to add `updateDescription` fields as additional fields/headers
during `ExtractNewDocumentState` transformation.
2022-11-16 14:26:51 +01:00