Commit Graph

195 Commits

Author SHA1 Message Date
Renato Mefi
13dfa8d9bf DBZ-974 Pair mongo-initiator with Server version
Avoiding client and servers mismatches, that's outside of debezium scope
to test whether the user has this sort of version mismatch.
2018-11-16 06:58:09 +01:00
Renato Mefi
beb069fc9d DBZ-974 Replace Mongo deprecations for count usage
As in
http://mongodb.github.io/mongo-java-driver/3.8/javadoc/deprecated-list.html
2018-11-16 06:58:09 +01:00
Renato Mefi
7c59023b9a Fix documentation for Mongo CDC encoding type 2018-11-12 11:47:18 +01:00
Jenkins user
d952f5dfb0 [maven-release-plugin] prepare for next development iteration 2018-10-04 11:59:14 +00:00
Jenkins user
ff9b70278b [maven-release-plugin] prepare release v0.9.0.Alpha2 2018-10-04 11:59:13 +00:00
Jiri Pechanec
d933c150ad DBZ-928 NULL field values are not put into struct 2018-10-04 09:24:04 +02:00
jchipmunk
3ec0b3be81 DBZ-918 Moving connector identifier from SourceInfo to Module class 2018-10-04 09:00:13 +02:00
jchipmunk
05ba99bb0a DBZ-918 Adding Debezium connector field to source info
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
2018-10-04 09:00:13 +02:00
Jiri Pechanec
bf6b2827c8 DBZ-784 Correct condition for stopped running projection 2018-09-18 13:12:21 +02:00
Gunnar Morling
6f0d11cc64 DBZ-784 Changing var name 2018-09-18 09:55:43 +02:00
Jiri Pechanec
a459c2cc4d DBZ-784 Terminate execute loops during shutdown 2018-09-18 09:54:46 +02:00
jchipmunk
6e9e803e73 DBZ-633 Changes of field.renames format. Misc improvements.
* Changing field.renames delimiter from equal to colon character
* Adding examples of source document structure to tests
* Adding tests for case: two renames have the same target field
* Adding missing return
* Changing Path interface to abstract class
2018-09-14 12:54:11 +02:00
Gunnar Morling
aac24b1ed1 DBZ-633 Misc. improvements around FieldSelector;
* Adding some comments
* Adding missing return
* More expressive names
* Formatting
2018-09-14 12:54:11 +02:00
jchipmunk
b93551f977 DBZ-881 Adding "field.renames" property for MongoDB connector
The "field.renames" configuration property is an optional comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values. Fully-qualified replacements for fields are of the form "databaseName.collectionName.fieldName.nestedFieldName=newNestedFieldName", where "databaseName" and "collectionName" may contain the wildcard (*) which matches any characters, the equal character (=) is used to determine rename mapping of field.
2018-09-14 12:54:11 +02:00
jchipmunk
b0feaf496a DBZ-633 Adding "field.blacklist" property for MongoDB connector
The "field.blacklist" configuration property is an optional comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form "databaseName.collectionName.fieldName.nestedFieldName", where "databaseName" and "collectionName" may contain the wildcard (*) which matches any characters.

Although the "field.blacklist" configuration property allows you to remove fields from the event values, the "_id" field is always included in the event’s key.
2018-09-14 12:54:11 +02:00
jchipmunk
e9ef4f9068 DBZ-908 Removing obsolete config parameters: DATABASE_LIST_NAME and COLLECTION_LIST_NAME 2018-09-13 12:21:29 +02:00
Gunnar Morling
e1bbbd7992 DBZ-865 DBZ-878 Replacing invalid topic name characters with "_"; 2018-09-05 12:56:53 +02:00
Gunnar Morling
72925a3060 DBZ-838 Formatting 2018-07-27 20:33:45 +02:00
jchipmunk
6c18a81faa DBZ-838 Adding condition to interrupt initial sync if task is stopped 2018-07-26 19:53:27 +03:00
Jenkins user
e00dad127f [maven-release-plugin] prepare for next development iteration 2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700 [maven-release-plugin] prepare release v0.9.0.Alpha1 2018-07-26 08:00:12 +00:00
Gunnar Morling
a5b5e7d9eb DBZ-627 Using unfified TopicSelector for MongoDB, too 2018-07-18 12:36:07 +02:00
Jenkins user
f9b8d830a8 [maven-release-plugin] prepare for next development iteration 2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f [maven-release-plugin] prepare release v0.8.0.Final 2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d [maven-release-plugin] prepare for next development iteration 2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0 [maven-release-plugin] prepare release v0.8.0.CR1 2018-07-04 07:07:44 +00:00
Gunnar Morling
e1cf34dc43 DBZ-713 Extracting constaint for expected exception message 2018-07-03 14:41:47 +02:00
Gunnar Morling
c5f9d99644 DBZ-713 Adding overload of execute() that returns a result;
This simplifies interaction with execute(), as we don't need to alter collections, references as a side effect but instead can work with the return value.
2018-07-03 14:41:47 +02:00
Gunnar Morling
d6b897f43d DBZ-713 Applying database and collection filters within MongoPrimary;
This protects against authorization failures when listing collections
from DBs the connector user isn't authorized for. It also simplifies
usage of MongoPrimary#databaseNames() and collections() as consumers
don't need to apply filtering themselves.
2018-07-03 14:41:47 +02:00
Gunnar Morling
d72d3fcbc9 DBZ-782 Propagating authorization failures 2018-07-03 14:41:47 +02:00
Jiri Pechanec
d4c9d24b22 DBZ-779 Detect mongo replicator init completion 2018-07-03 12:24:06 +02:00
Gunnar Morling
dcd6b01134 DBZ-644 Simplifying RelationalDatabaseSchema constructor;
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
2018-07-03 07:03:59 +02:00
Gunnar Morling
bf7a5018ca Setting POM version back to 0.8.0-SNAPSHOT 2018-06-22 12:21:03 +02:00
Jenkins user
db42e4657a [maven-release-plugin] prepare for next development iteration 2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99 [maven-release-plugin] prepare release v0.8.0.Beta1 2018-06-21 14:07:45 +00:00
Gunnar Morling
d7e196a18e DBZ-20 Initial import of Oracle connector based on XStream 2018-06-20 13:05:37 +02:00
echo.xu
15f622f823 DBZ-712. ignore initsync offset during connector start. 2018-06-20 10:41:46 +02:00
Gunnar Morling
3f20c8d8cc DBZ-561 Using less confusing server name in tests 2018-06-11 11:40:15 +02:00
Gunnar Morling
abe7863c28 DBZ-561 More consistent element ordering 2018-06-11 11:40:15 +02:00
Gunnar Morling
bb35d5e4bc DBZ-561 Adjusting test after rebase 2018-06-11 11:40:15 +02:00
Jiri Pechanec
4698dd2745 DBZ-561 Support nested updates 2018-06-11 11:40:15 +02:00
Sairam Polavarapu
4b67b14304 DBZ-561 Provided option to flatten structs in MongoDB unwrapping SMT. 2018-06-11 11:40:15 +02:00
Jiri Pechanec
32d591d392 DBZ-649 Use Avro-compatible field naming 2018-06-05 10:33:40 +02:00
Jiri Pechanec
d1488a7c48 DBZ-650 Fix test 2018-06-05 10:33:39 +02:00
Jiri Pechanec
3eb034f035 DBZ-649 Encoding arrays as struct 2018-06-05 09:57:19 +02:00
Jiri Pechanec
e00dc3beb4 DBZ-649 Graceful failures for heterogenous arrays 2018-06-05 09:57:19 +02:00
Jiri Pechanec
aa5581b524 DBZ-650 Fix the test breaking the build 2018-06-04 14:58:42 +02:00
Gunnar Morling
d556fca852 DBZ-650 Expanding test 2018-05-30 15:27:30 +02:00
Jiri Pechanec
972068129c DBZ-650 Add names to unwrapped MongoDB schemas 2018-05-30 15:02:38 +02:00
Jiri Pechanec
6e1e5f40ce Protect UnwrapFromMongoDbEnvelopeTestIT from inconsistent snapshot 2018-05-15 09:50:54 +02:00
Jiri Pechanec
232e7acde5 DBZ-680 Integration test for MongoDB unwrap SMT 2018-04-13 11:39:34 +02:00
Jiri Pechanec
3e9489741d DBZ-529 Upgrade to MongoDB 3.6, compatibility testing 2018-03-29 15:47:59 +02:00
Saulius Valatka
31c29459f7 DBZ-672 do not use cursor with no timeout 2018-03-29 14:47:57 +02:00
Saulius Valatka
a78c52cbdd DBZ-672 stop tailing the oplog when replicator is stopped 2018-03-29 14:47:57 +02:00
Jenkins user
f4e151b23a [maven-release-plugin] prepare for next development iteration 2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332 [maven-release-plugin] prepare release v0.7.5 2018-03-20 08:14:19 +00:00
Jiri Pechanec
40a527b6ea DBZ-592 Make snapshot/init-sync false instead of null 2018-03-19 13:19:34 +01:00
Jiri Pechanec
2a35eae178 DBZ-648 InterrupedException handled by MongoDB 2018-03-09 20:43:57 +01:00
Jenkins user
daf27207be [maven-release-plugin] prepare for next development iteration 2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928 [maven-release-plugin] prepare release v0.7.4 2018-03-07 08:31:07 +00:00
Gunnar Morling
eef1ad7c2a DBZ-630 Refactoring around MongoDB task context;
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8 DBZ-630 Pulling up getClock() to ConnectorTaskContext 2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6 DBZ-630 Unifying common start-up logic across connectors 2018-02-21 12:04:41 +01:00
Gunnar Morling
7fe45db831 DBZ-626 Renaming AvroValidator to SchemaNameAdjuster;
Making apparent that this functionality doesn't solely validate but returns a new name actually
2018-02-20 09:26:47 +01:00
Gunnar Morling
69df7c4ade DBZ-628 Using common properties for MongoDB connector 2018-02-20 06:34:42 +01:00
Jenkins user
6d0cd88e12 [maven-release-plugin] prepare for next development iteration 2018-02-15 04:15:34 +00:00
Jenkins user
7d1e1a989e [maven-release-plugin] prepare release v0.7.3 2018-02-15 04:15:34 +00:00
Jiri Pechanec
93259e8e11 DBZ-581 Simplified monitor thread shutdown 2018-02-13 14:40:54 +01:00
Jiri Pechanec
dbac1429b2 DBZ-581 Improve error handling for replicators 2018-02-13 14:40:54 +01:00
Gunnar Morling
39e08b7ec6 DBZ-582 Avoiding assignment to test class field from test methods 2018-02-13 10:19:18 +01:00
Gunnar Morling
2e0b9e5e33 DBZ-582 Moving TOMBSTONES_ON_DELETE to CommonsConnectorConfig for re-use 2018-02-13 10:19:18 +01:00
Raf Liwoch
38641f472d DBZ-582 Allowing users to configure whether tombstone events should be emitted for DELETE operations or not;
* Adding new configuration parameter `tombstones.on.delete`
* Using this new option in MongoDB connector
2018-02-13 10:19:18 +01:00
Gunnar Morling
1626f385ad DBZ-580 Using ChangeEventQueue in MongoDB connector 2018-02-11 10:05:06 +01:00
Jiri Pechanec
b3b379a3a3 DBZ-594 Stop copy threads after initial sync is completed 2018-02-05 12:37:10 +01:00
Gunnar Morling
712824e43f DBZ-593 Passing version explicitly from SourceInfos to base class instead of relying specific packaging structure 2018-02-02 13:16:43 +01:00
Jiri Pechanec
7ebee94169 DBZ-593 Add Debezium version to source in envelope 2018-02-02 13:16:43 +01:00
Jiri Pechanec
92740a3626 DBZ-587 Fix thread leak, thread names 2018-02-01 10:04:20 +01:00
Jiri Pechanec
9b592204ac DBZ-587 Centralize and unify thread management 2018-02-01 10:04:20 +01:00
Gunnar Morling
6089d6d051 DBZ-579 Using port 27017 in TestHelper as that's the port used by our Docker set-up 2018-01-29 15:24:16 +01:00
Jiri Pechanec
294a46eeef DBZ-579 Allow repeated running of MongoDB tests from IDE withou config 2018-01-29 15:24:16 +01:00
Jenkins user
04624341f5 [maven-release-plugin] prepare for next development iteration 2018-01-25 09:39:44 +00:00
Jenkins user
898f6884e1 [maven-release-plugin] prepare release v0.7.2 2018-01-25 09:39:44 +00:00
Gunnar Morling
6c0920c701 DBZ-409 Emitting null value for deletes 2018-01-23 14:41:39 +01:00
Gunnar Morling
707421e8e5 DBZ-409 Adding test for complex id type 2018-01-23 14:41:39 +01:00
Gunnar Morling
9fcf670df1 DBZ-409 Adding proper test for UnwrapFromMongoDbEnvelope;
* removing "_id" field
* using more sensible variable names
2018-01-23 14:41:39 +01:00
Gunnar Morling
808da37f48 DBZ-409 Indentation fixes 2018-01-23 14:41:39 +01:00
Gunnar Morling
b7f72bbe3a DBZ-409 Dependency clean-up 2018-01-23 14:41:39 +01:00
Sairam Polavarapu
e1cfa66862 DBZ-409 Added the testcase for UnwrapFromMongoDbEnvelope and also renamed the class names accordingly. 2018-01-23 14:41:39 +01:00
Gunnar Morling
1cfc8c3596 DBZ-409 Indentation fix 2018-01-23 14:41:39 +01:00
Gunnar Morling
d6dbf02f4c DBZ-409 Formatting of test and JSON file; removing unused JSON file 2018-01-23 14:41:39 +01:00
Sairam Polavarapu
8f268993ce DBZ-409 Adding SMT for converting MongoDB CDC messages into traditional flat format 2018-01-23 14:41:39 +01:00
Gunnar Morling
0c4190c493 DBZ-516 Using Duration instead of long in a few more places 2018-01-18 14:13:58 +01:00
Jiri Pechanec
24bdcaf059 DBZ-516 Return control to Connect periodically 2018-01-18 14:13:58 +01:00
Jiri Pechanec
2a7377a833 DBZ-455 Use valueOf instead of constructors 2018-01-05 02:32:37 +01:00
Jenkins user
6bb34b42f9 [maven-release-plugin] prepare for next development iteration 2017-12-20 07:15:12 +00:00
Jenkins user
16dcd4c980 [maven-release-plugin] prepare release v0.7.1 2017-12-20 07:15:12 +00:00
Jenkins user
5e09932cb9 [maven-release-plugin] prepare for next development iteration 2017-12-15 05:10:23 +00:00
Jenkins user
6c1d61e03b [maven-release-plugin] prepare release v0.7.0 2017-12-15 05:10:23 +00:00
Gunnar Morling
2744f54e9d DBZ-438 Renaming BufferedBlockingConsumer#flush() to close() to make clear it's onyl meant to be called once at the end 2017-11-13 14:19:39 +01:00
Gunnar Morling
5fbe742be8 DBZ-285 Specifying scope of dependencies in the individual POMs for the sake of comprehensibility 2017-11-10 16:48:32 +01:00