Commit Graph

664 Commits

Author SHA1 Message Date
Debezium Builder
7bd1551c99 [maven-release-plugin] prepare release v2.0.0.Final 2022-10-14 11:26:32 +00:00
Debezium Builder
884254aba1 [maven-release-plugin] prepare for next development iteration 2022-10-07 09:27:52 +00:00
Debezium Builder
a0f3263fa6 [maven-release-plugin] prepare release v2.0.0.CR1 2022-10-07 09:27:51 +00:00
Sage-Pierce
e6776d7382 DBZ-5677: Ensure there is a time to start at before setting one 2022-10-06 11:24:32 +02:00
Sage-Pierce
ff213c9f03 DBZ-5677: Apply fix from DBZ-5371 for MongoDB Snapshot Source to Streaming Source 2022-10-06 11:24:32 +02:00
Jiri Pechanec
75aeac595d DBZ-5628 Fix formatting 2022-10-06 09:38:20 +02:00
Bin Huang
92b3035457 DBZ-5628 Add mongo 6.0 pre-image IT test 2022-10-06 09:38:20 +02:00
Bin Huang
94bdf755d1 DBZ-5628 Add mongo 6.0 version and db version resolver 2022-10-06 09:38:20 +02:00
Xinbin Huang
b4a87d45a7 DBZ-5628 Added support for Mongo pre-image in change stream 2022-10-06 09:38:20 +02:00
harveyyue
f6b72f060c DBZ-5671 Remove logic name parameter from sub connector config 2022-10-03 11:40:17 -04:00
Sergei Morozov
e4d46c81b1 DBZ-5541: Default Debezium schema.name.adjustment.mode to "none" 2022-09-29 09:47:25 +02:00
Vojtech Juranek
97532042b3 DBZ-5657 Allow to use Docker maven plugin external properties
Allow to override Docker maven plugin properties [1] from command line
to be able to change various Docker parameters more easily when starting
the container.

[1] https://dmp.fabric8.io/#combining-property-config
2022-09-29 05:41:09 +02:00
harveyyue
d1153e8ad4 DBZ-5594 Clean up "logical name" config 2022-09-22 14:29:37 +02:00
Debezium Builder
218d414552 [maven-release-plugin] prepare for next development iteration 2022-09-16 06:36:40 +00:00
Debezium Builder
567f57c47b [maven-release-plugin] prepare release v2.0.0.Beta2 2022-09-16 06:36:39 +00:00
Vojtech Juranek
1afcea971c DBZ-5043 Add topic prefix to Mongo config definition 2022-09-14 12:44:38 +02:00
Vojtech Juranek
acc6be6df3 DBZ-5043 Move TOPIC_PREFIX into CommonConnectorConfig
`TOPIC_PREFIX` is now mandatory to all connectors therefore it make
sense to have it in common config. Beside that, it also makes it more
easy to use it in Debezium UI without any workarounds - if the field is
not member of the given connector config, the field has to be
explicitely added into known fields otherwise is invisible for UI.
2022-09-13 09:25:54 +02:00
Anisha Mohanty
f41b6dd458 DBZ-5044 Get rid of schema factory instances 2022-09-08 10:35:20 +02:00
Anisha Mohanty
b498368f82 DBZ-5044 Use SchemaFactory for centralizing MongoDb schemas 2022-09-08 10:35:20 +02:00
Vojtech Juranek
8d7611bfcb DBZ-5043 Remove unused DB_HISTORY_PATH constant 2022-09-07 12:57:31 +02:00
Vojtech Juranek
eb6e2648f4 DBZ-5043 Replace mongodb.name with topic.prefix
To be consistent with specifying topic prefix, replace also MongoDB
`mongodb.name` with `topic.prefix`.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
161a7229b6 DBZ-5043 Use topic prefix as the logical name
With this change the user cannot direcly set connector logical name and
thus in the future it can be remove without breaking user config.
If the turn out that the logical name is useful and user should be able
to configure it, dedicated config option can be added.

Originally it was prposed in the Jira to replace it with connector name,
but it turned out that logical name defaults to `database.server.name`
and is heavily used in the tests and JMX, so it would require another
big refactoring. Thus, use topic prefix for now. Once we know further
direction (remove logical name or add new option), do this refactoring.
2022-09-07 12:57:31 +02:00
ggaborg
664fd9e1b0 DBZ-3293 unit tests for filters 2022-09-01 20:16:47 +02:00
ggaborg
244de2bd96 DBZ-3293 Add signal table automatically to include list 2022-09-01 20:16:47 +02:00
Jiri Pechanec
de53591dba DBZ-5244 Retry all communication exceptions by default 2022-08-31 09:45:09 -04:00
jcechace
5c61bc4341 DBZ-4733 Reordering fileds in MongoDbConnectorConfig 2022-08-19 12:55:58 +02:00
jcechace
b61319feb7 DBZ-4733 Basic integration test for connection string support 2022-08-19 12:55:58 +02:00
jcechace
7670c4ba74 DBZ-4733 working POC for connection String support 2022-08-19 12:55:58 +02:00
jcechace
521156f5c2 DBZ-4733 Refactored MongoUtil.getPrimaryAddress to properly use mongo driver 4.x 2022-08-19 12:55:58 +02:00
jcechace
21ed5f9b9f DBZ-4733 Refactor ReplicaSets.parse into more readable form 2022-08-19 12:55:58 +02:00
jcechace
752062c1a3 DBZ-4733 Fixed thread safety of MongoClients 2022-08-19 12:55:58 +02:00
jcechace
1d9d75754c DBZ-4733 Refactoring ReplicaSetDiscovery to properly use mongodb driver 4.x 2022-08-19 12:55:58 +02:00
jcechace
3a385aa768 DBZ-4733 Refactoring: unified client factory method names 2022-08-19 12:55:58 +02:00
jcechace
cc04a94fd5 DBZ-4733 Ininital connection string filed introduction 2022-08-19 12:55:58 +02:00
Chris Cranford
b7e2112603 DBZ-5497 Align test behavior 2022-08-19 11:05:29 +02:00
Vivek Wassan
d89e71ebeb DBZ-5327 Smart Backfills 2022-08-17 10:05:55 +02:00
nicholas-fwang
58a91cddb4 DBZ-5479 Retry MongoNodeIsRecoveringException when failover 2022-08-05 10:52:46 -04:00
Chris Cranford
90a4c0059a
DBZ-5434 Fix test failure 2022-08-01 14:36:09 -04:00
harveyyue
7c93c597f7 DBZ-5434 Translation from mongodb document to kafka connect schema fails when nested arrays contain no elements 2022-08-01 10:07:28 -04:00
Vojtech Juranek
c7d8353d7b DBZ-4727 Add snapshot paused JXM metrics
Add two JXM objects:
* `snapshotPaused` - detemines if the incremental snapshot is paused
  or not
* `snapshotPausedDurationInSeconds` - overall time when the incremental
  snapshot was paused. The time adds up - if the snapshot was paused
  e.g. two times, the `snapshotPausedDurationInSeconds` is the sum of
  these two paused times.
2022-07-29 12:41:58 -04:00
Vojtech Juranek
ed504173e1 DBZ-4727 Implement incremental snaphost pause/resume
Intorduce two new signals:
* `pause-snapshot` - pauses running incremental snapshot
* `resume-snapshot` - resumes paused incremental snapshot

If the incremental snapshot is running and pause signal is sent,
currently processed chunk is finished and then no other chunk is read.
Snapshot state is kept in running state. Once snapshot is resume,
chunk is reverted to position which was last sent to the broker and
process of reading chunks and emitting `read` events continues until
all tables are read and sent.

This patch handles only table-based signals. MySQL Kafka-based signals
will be addressed later.
2022-07-29 12:41:58 -04:00
harveyyue
5f430f1a8e DBZ-5457 Deprecate legacy topic selector for all connectors 2022-07-28 12:58:59 -04:00
Debezium Builder
1ab51ae8d0 [maven-release-plugin] prepare for next development iteration 2022-07-26 18:13:10 +00:00
Debezium Builder
d247a48b79 [maven-release-plugin] prepare release v2.0.0.Beta1 2022-07-26 18:13:09 +00:00
Chris Cranford
de4d8e17d3 DBZ-5371 Check for $natural not supported, treat as no oplog entry 2022-07-25 10:20:46 -04:00
harveyyue
6ea7abc8fc DBZ-4180 Pluggable topic selector 2022-07-13 14:45:25 +02:00
harveyyue
e8405bc376 DBZ-4846 MongoConnector's field exclusion configuration does not work with fields with the same name but from different collections 2022-07-12 17:26:36 +02:00
Debezium Builder
6fd1441af3 [maven-release-plugin] prepare for next development iteration 2022-07-01 09:21:49 +00:00
Debezium Builder
99402af634 [maven-release-plugin] prepare release v2.0.0.Alpha3 2022-07-01 09:21:48 +00:00
harveyyue
2a2b5462db DBZ-5328 field.exclude.list doesn't accept dashes (-) in database name 2022-06-30 10:47:56 +02:00