Gunnar Morling
dd5c2b8be9
DBZ-1766 Unifying handling of offset commits
2020-02-12 15:44:45 +01:00
Jiri Pechanec
77f9a61b95
DBZ-1751 Centralize envelope schema naming
2020-02-04 20:37:19 +01:00
Jiri Pechanec
ba10a229b6
DBZ-1052 CE support for tx metadata in data messages
2020-02-03 09:29:10 +01:00
Jiri Pechanec
60056b0a94
DBZ-1052 Transaction block moved to envelope
2020-02-03 09:29:10 +01:00
Jiri Pechanec
b0eb571f9f
DBZ-1052 Transaction metadata for SQL Server
2020-02-03 09:29:10 +01:00
Gunnar Morling
d5bbc7a123
DBZ-1717 Misc. clean-up;
...
* Expanding test
* Removing obsolete deprecation warning
* Adding Jan Uyttenhove to COPYRIGHT.txt
2020-01-28 12:41:12 +01:00
Jan Uyttenhove
144936363f
DBZ-1717 Usage of java.util.Date and Kafka Connect Timestamp to create Date(Time) field types
2020-01-28 12:41:12 +01:00
Gunnar Morling
8c33a4e147
DBZ-1292 Catering for the differences in data schema;
...
* MongoDB doesn't have "before", but "patch"
* Also removing some redundancies and making RecordParser immutable
* Reducing log verbosity during tests
2020-01-16 08:01:56 +01:00
Gunnar Morling
7c18a6d8d5
DBZ-1292 Removing static state in CloudEventsConverterTest
2020-01-16 08:00:45 +01:00
GraySmog
d0e4b3c0af
DBZ-1292 Delete data.schema.registry.url config option
2020-01-16 08:00:45 +01:00
GraySmog
915c8cafd9
DBZ-1292 CloudEvents converter for JSON and Avro format
2020-01-16 08:00:45 +01:00
Jiri Pechanec
4102d39aba
DBZ-1168 Suport non-admin authSources
2020-01-06 12:28:49 +01:00
Gunnar Morling
bf25aeb1de
DBZ-1215 Formatting
2019-11-25 11:52:48 +01:00
Gunnar Morling
0fa29a0163
DBZ-1215 Misc. clean-up;
...
* long ms -> Instant
* Using OptionalLong as return value for optional value
2019-11-22 11:42:42 +01:00
Jiri Pechanec
6cadff8c36
DBZ-1215 Added support for transactions
2019-11-22 11:42:42 +01:00
Gunnar Morling
2c569977d7
DBZ-1392 Applying formatter;
...
* Keeping indentation formatting for JSON
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
Co-authored-by: Chris Cranford <chris@hibernate.org>
2019-11-01 13:20:43 +02:00
Chris Cranford
c25bb23a04
DBZ-1583 Propagate Replicator exception to Kafka Connect
2019-10-29 16:17:27 +01:00
Jiri Pechanec
b2a70889f5
DBZ-1563 Collection names can contain dots
2019-10-23 14:40:15 -04:00
Jiri Pechanec
b88d9b06d4
DBZ-1513 Align test names
2019-10-07 18:38:35 +02:00
Jiri Pechanec
2c4cb83259
DBZ-1513 Centralize handling of non-data messages in SMT
2019-10-07 18:38:35 +02:00
Gunnar Morling
92885ebe99
DBZ-1392 Removing trailing whitespace from Java files
2019-10-07 12:11:43 +03:00
Gunnar Morling
8d6f5409b1
DBZ-1392 Moving static imports up and applying this to all files
2019-10-07 12:11:43 +03:00
Chris Cranford
ca5517746a
DBZ-1341 Checkstyle flag missing whitespace
2019-10-03 19:44:43 +02:00
Chris Cranford
7442670692
DBZ-1355 Fix checkstyle problems detected after upgrade to 8.19
2019-10-03 19:44:43 +02:00
Chris Cranford
5ac2318d84
DBZ-1391 Do not allow code after closing brace ( #1065 )
...
* DBZ-1391 Do not allow code after closing brace
2019-10-02 20:44:40 +02:00
jerrinot
3064a4af06
DBZ-1438 Fix InterruptedException handling
...
It appears there is an intent to restore the interruption flag
when catching InterruptedException. However the old code was just
checking the current state of the flag instead of raising it.
2019-09-24 12:51:48 +02:00
Ching Tsai
80210f7f8b
DBZ-1442 Add test for a rewrite delete event
2019-09-13 11:23:26 +02:00
Ching Tsai
b8d565ffe0
DBZ-1442 Add @FixFor
2019-09-13 11:23:26 +02:00
Ching Tsai
486973ad12
DBZ-1442 Porting insert source fields feature to ExtractNewDocumentState
2019-09-13 11:23:26 +02:00
Gunnar Morling
1e0859588f
DBZ-1464 Misc. clean-up;
...
* logger -> LOGGER
* making op literals map unmodifiable
* Adding Lev Zemlyanov to COPYRIGHT.txt
2019-09-06 11:02:45 +02:00
Lev Zemlyanov
8cbb8eb63b
DBZ-1464 Fixed NPE when op is not u/i/d
...
Signed-off-by: Lev Zemlyanov <lev@confluent.io>
2019-09-06 11:02:27 +02:00
Renato Mefi
a9c8a825c3
DBZ-1430 Support heartbeat ExtractNewDocumentState
2019-08-14 23:56:45 +02:00
Gunnar Morling
287551294f
DBZ-1198 Adding comments
2019-07-11 07:43:32 +02:00
Jiri Pechanec
0316ac7fc0
DBZ-1198 Heartbeat for MongoDB connector
2019-07-10 11:16:26 +02:00
Jiri Pechanec
13d638fd43
DBZ-1356 Metrics for schema history ( #947 )
...
* DBZ-1356 Metrics for schema history
2019-07-02 13:26:17 +02:00
Gunnar Morling
a991ea97d7
Handling option for emitting tombstones conistently
2019-07-01 08:27:33 +02:00
Jiri Pechanec
72b751b528
DBZ-1333 Upgrade MongoDb driver to 3.10.1
2019-06-17 20:53:07 +02:00
Gunnar Morling
5d01f409f0
DBZ-1317 Separating update and query methods for MongoDB's SourceInfo;
...
* Avoids access to collection id after events which don't have one
* Pulling up struct() and schema() methods to common base class
2019-06-11 10:07:36 +02:00
Chris Cranford
0aadaa10dc
DBZ-1242 Fixed typo and added negative test checks
2019-06-05 13:31:25 -04:00
Chris Cranford
c53435c26b
DBZ-1242 Checkstyle fix - Remove unused import
2019-06-05 13:31:25 -04:00
Chris Cranford
b3ca33287a
DBZ-1242 Issue warning when filters produce an empty schema
2019-06-05 13:31:25 -04:00
Ching Tsai
f8cb51545f
DBZ-1315 UNDEFINED field causes exception
2019-06-03 13:51:01 +02:00
Jiri Pechanec
d1bc1c218c
DBZ-1295 3state snapshot for MongoDB
2019-05-31 10:56:05 +02:00
Gunnar Morling
b8eede2f41
DBZ-1235 Misc. clean-up;
...
* Putting collection name after replica set name for MongoDB
* Removing redundant method for converting schema to string
* Removing redundant keys from PG SourceInfo
2019-05-28 09:35:15 +02:00
Jiri Pechanec
526e780513
DBZ-1235 Misc fixes
2019-05-28 09:35:15 +02:00
Jiri Pechanec
80afd4975a
DBZ-1235 Changes missed in refactoring
2019-05-28 09:35:15 +02:00
Gunnar Morling
8f28f323e7
DBZ-1235 Using Instant to represent timestamps internally;
...
Only "rendered" as epoch milli-seconds at the system boundaries.
2019-05-28 09:35:15 +02:00
Jiri Pechanec
6f6c8ddf06
DBZ-1235 Refactored to unify the code
2019-05-28 09:35:15 +02:00
Jiri Pechanec
c1bf1752fe
DBZ-1235 Pluggable source maker for MongoDB
2019-05-28 09:35:15 +02:00
Jiri Pechanec
f5757dbd9f
DBZ-1235 Extract serverName to common source
2019-05-28 09:35:15 +02:00
Gunnar Morling
cf4db8c1a1
DBZ-677 Extracting shared configuration options for both new state extraction SMTs
2019-05-27 13:52:58 +02:00
Gunnar Morling
cb9e916616
DBZ-677 Renaming record state SMTs;
...
Keeping old names in deprecated sub-classes for the time being.
2019-05-27 13:52:58 +02:00
Chris Cranford
e2c74111ad
DBZ-1234 Removed deprecated configuration options
2019-05-22 10:19:05 +02:00
jchipmunk
72b06366fe
Removing extra initialisation of "config" variable in PG config
2019-04-30 09:57:11 +02:00
jchipmunk
9fe033ce24
DBZ-1247 Specifying default snapshot fetch size for each connector config
2019-04-30 09:57:11 +02:00
jchipmunk
a5cc83c0d1
DBZ-1247 Adding property to specify snapshot fetch size
2019-04-30 09:57:11 +02:00
jchipmunk
2c79d901c5
DBZ-1247 Using isNonNegativeInteger instead of isPositiveInteger
2019-04-30 09:57:11 +02:00
jchipmunk
3ca47b1055
DBZ-1247 Adding property to specify batch size of MongoDB cursor
...
The "documents.fetch.size" configuration property is an positive integer value that specifies the maximum number of documents that should be read in one go from each collection while taking a snapshot. The connector will read the collection contents in multiple batches of this size. Default to "0", which indicates that the server chooses an appropriate fetch size.
2019-04-30 09:57:11 +02:00
krizhan
70c564b4c2
DBZ-212 Moved SERVER_NAME to RelationalDatabaseConnectorConfig
2019-04-23 12:29:44 +02:00
jchipmunk
7c0ae3ee20
DBZ-1212 SLF4J usage issues
2019-04-04 21:32:12 +02:00
ShubhamRwt
540a951211
DBZ- 362 Adding WhitespaceAfter check to Checkstyle
2019-03-28 09:24:11 +01:00
Jiri Pechanec
9b405aa84d
DBZ-1039 Add missing spaces
2019-03-25 10:23:14 +01:00
shubham
8cca21e969
DBZ-1039 Make all ifs with braces
2019-03-25 10:23:14 +01:00
Renato Mefi
2f187fd1cb
DBZ-354 Improve MongoDB hosts validation
...
The previous statement was never achievable since `ReplicaSets.parse(hosts)` can never return null
Also removed some code duplication for better readability
2019-03-14 16:53:51 +01:00
Gunnar Morling
d2b26d03a7
DBZ-867 Misc clean-up;
...
* Exposing new config via MongoDbConnectorConfig
* Removing superfluous connector restart in the test
2019-02-21 17:00:12 +01:00
krizhan
652f3b1c59
DBZ-867 Added integration test and updated enum names
2019-02-21 17:00:12 +01:00
krizhan
441228a989
DBZ-867 Add snapshot mode NONE for MongoDB connector
2019-02-21 17:00:12 +01:00
Gunnar Morling
2a486ecfff
DBZ-1019 Removing legacy methods from Metronome
2019-02-11 18:11:34 +01:00
Renato Mefi
5810014fda
DBZ-988 Migrate UnwrapFromMongoDbEnvelopeTestIT to new Abstract TestIT
2019-01-07 13:42:40 +01:00
Renato Mefi
8a22b6666f
DBZ-988 Break down Unwrap TestIT Abstract
...
This allows other tests to also take advantages of the testing structure
2019-01-07 13:42:40 +01:00
Renato Mefi
82bd0bb657
DBZ-988 Test all Array Update operators
...
As in the official documentation:
https://docs.mongodb.com/v3.6/reference/operator/update-array/#update-operators
The operations: `$(update)`, `$[]` and `$[<identifier>]` are skipped
since the're Mongo 3.6+ only.
2019-01-07 13:42:40 +01:00
Renato Mefi
6fc010867e
DBZ-988 Test all Bitwise Update operators
...
As in the official documentation:
https://docs.mongodb.com/v3.6/reference/operator/update/bit/#bit
2019-01-07 13:42:40 +01:00
Renato Mefi
20cc350891
DBZ-988 Test all Field Update operators
...
As in the official documentation:
https://docs.mongodb.com/v3.6/reference/operator/update-field/#field-update-operators
2019-01-07 13:42:40 +01:00
Renato Mefi
c43c9ecc54
DBZ-988 MongoDB unwrap Update operators base test
2019-01-07 13:42:40 +01:00
Jiri Pechanec
edec1c3090
DBZ-978 Added last event and captured tables metrics
2018-12-13 20:52:11 +01:00
Gunnar Morling
91b571059e
DBZ-966 Making sure MongoDB connector can be stopped while awaiting initial sync delay
2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
e2942ab72a
DBZ-966 Replace sleep with polling loop
2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
c5fac7d3b2
DBZ-966 Enable to delay initial snapshot in MongoDB connector
2018-12-13 14:29:54 +01:00
Renato Mefi
1f3791cea9
DBZ-1032 Test tombstones have a DELETE header
2018-12-11 13:50:42 +01:00
Renato Mefi
8d04aab6a4
DBZ-1032 Fix issue for tombstone operation headers
...
Since the bomstone has no value it's not possible to imply the header
from the message content, thus if the value is null we set the header to
DELETE
2018-12-11 13:50:42 +01:00
Gunnar Morling
5aeb589932
DBZ-848 Adding Tautvydas Januskevicius to COPYRIGHT.txt
2018-12-11 12:00:59 +01:00
Tautvydas Januskevicius
addf13ca06
DBZ-848 Removed explicit primary node change handling. It is handled by mongoDb driver internally. Explicit primary node change event handling was causing issue that no mongo events were read after primary change. This issue only happened when rs.stepDown(<time s>) was used on mongo node.
2018-12-11 12:00:59 +01:00
Renato Mefi
4ba3213338
DBZ-1020 Refactor UnwrapFromMongoDbEnvelope logic
...
Now it's easier to follow what makes an insert, update or delete within
the transformer, also reduced the amount of npath complexity by doing
early returns and certain extractions.
2018-12-07 09:16:36 +01:00
Renato Mefi
8d8ef26dae
DBZ-563 MongoDB Unwrapper tests for "delete.handling.mode"
2018-11-30 12:06:18 +01:00
Renato Mefi
fcef819dc8
DBZ-563 MongoDB Unwrapper supports "delete.handling.mode"
2018-11-30 12:06:18 +01:00
Renato Mefi
de7c065b43
DBZ-971 Mongo Unwrap can add data operation header
2018-11-30 12:06:18 +01:00
Renato Mefi
4b0a84f587
DBZ-971 Ensure Mongo unwrap propagates headers
2018-11-30 12:06:18 +01:00
Renato Mefi
312b976c0d
DBZ-563 Test MongoDB Unwrapper tombstone drop
2018-11-30 12:06:18 +01:00
Renato Mefi
0d6236c8d8
DBZ-563 Support tombstone drop on MongoDB Unwrapper
2018-11-30 12:06:18 +01:00
Renato Mefi
16ba3764e5
DBZ-989 Support tombstone on MongoDB unwrapper
2018-11-30 12:06:18 +01:00
Renato Mefi
488dfe4f72
DBZ-989 Provide test for the bug confirmation
2018-11-30 12:06:18 +01:00
Renato Mefi
b2aed1f46e
DBZ-987 Allow to processes full document updates
2018-11-30 12:06:18 +01:00
Renato Mefi
c540391db2
DBZ-987 Test Full Document replacement
...
As documented here:
https://docs.mongodb.com/manual/reference/method/db.collection.update/#replace-a-document-entirely
Mongo supports full document updates, currently the Unwrap expects the
operation to always be a patch update, this test proves the bug to be
patched in the next commits.
2018-11-30 12:06:18 +01:00
Renato Mefi
df738a2abd
DBZ-612 Support MongoDB CDC full $unset operation
...
Mongo internally transforms everything into $set and $unset operations
when they are in the oplog, this guarantees that you can have operations
which are $set, $unset or combined.
2018-11-30 12:06:18 +01:00
Renato Mefi
d44a44aee5
DBZ-612 Test Mongo CDC multiple $set operations
2018-11-30 12:06:18 +01:00
Renato Mefi
7a4e1c48f1
DBZ-612 Transform $unset into null values
2018-11-30 12:06:18 +01:00
Renato Mefi
6e115043ca
DBZ-612 Support mongo NULL type
...
Since we can't trace back the original value since mongo is a
schema-less storage we fallback to string as strategy.
2018-11-30 12:06:18 +01:00
Renato Mefi
beb069fc9d
DBZ-974 Replace Mongo deprecations for count usage
...
As in
http://mongodb.github.io/mongo-java-driver/3.8/javadoc/deprecated-list.html
2018-11-16 06:58:09 +01:00
Renato Mefi
7c59023b9a
Fix documentation for Mongo CDC encoding type
2018-11-12 11:47:18 +01:00
Jiri Pechanec
d933c150ad
DBZ-928 NULL field values are not put into struct
2018-10-04 09:24:04 +02:00