Commit Graph

93 Commits

Author SHA1 Message Date
jchipmunk
3ec0b3be81 DBZ-918 Moving connector identifier from SourceInfo to Module class 2018-10-04 09:00:13 +02:00
jchipmunk
05ba99bb0a DBZ-918 Adding Debezium connector field to source info
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
2018-10-04 09:00:13 +02:00
Jiri Pechanec
bf6b2827c8 DBZ-784 Correct condition for stopped running projection 2018-09-18 13:12:21 +02:00
Gunnar Morling
6f0d11cc64 DBZ-784 Changing var name 2018-09-18 09:55:43 +02:00
Jiri Pechanec
a459c2cc4d DBZ-784 Terminate execute loops during shutdown 2018-09-18 09:54:46 +02:00
jchipmunk
6e9e803e73 DBZ-633 Changes of field.renames format. Misc improvements.
* Changing field.renames delimiter from equal to colon character
* Adding examples of source document structure to tests
* Adding tests for case: two renames have the same target field
* Adding missing return
* Changing Path interface to abstract class
2018-09-14 12:54:11 +02:00
Gunnar Morling
aac24b1ed1 DBZ-633 Misc. improvements around FieldSelector;
* Adding some comments
* Adding missing return
* More expressive names
* Formatting
2018-09-14 12:54:11 +02:00
jchipmunk
b93551f977 DBZ-881 Adding "field.renames" property for MongoDB connector
The "field.renames" configuration property is an optional comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values. Fully-qualified replacements for fields are of the form "databaseName.collectionName.fieldName.nestedFieldName=newNestedFieldName", where "databaseName" and "collectionName" may contain the wildcard (*) which matches any characters, the equal character (=) is used to determine rename mapping of field.
2018-09-14 12:54:11 +02:00
jchipmunk
b0feaf496a DBZ-633 Adding "field.blacklist" property for MongoDB connector
The "field.blacklist" configuration property is an optional comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form "databaseName.collectionName.fieldName.nestedFieldName", where "databaseName" and "collectionName" may contain the wildcard (*) which matches any characters.

Although the "field.blacklist" configuration property allows you to remove fields from the event values, the "_id" field is always included in the event’s key.
2018-09-14 12:54:11 +02:00
jchipmunk
e9ef4f9068 DBZ-908 Removing obsolete config parameters: DATABASE_LIST_NAME and COLLECTION_LIST_NAME 2018-09-13 12:21:29 +02:00
Gunnar Morling
e1bbbd7992 DBZ-865 DBZ-878 Replacing invalid topic name characters with "_"; 2018-09-05 12:56:53 +02:00
Gunnar Morling
72925a3060 DBZ-838 Formatting 2018-07-27 20:33:45 +02:00
jchipmunk
6c18a81faa DBZ-838 Adding condition to interrupt initial sync if task is stopped 2018-07-26 19:53:27 +03:00
Gunnar Morling
a5b5e7d9eb DBZ-627 Using unfified TopicSelector for MongoDB, too 2018-07-18 12:36:07 +02:00
Gunnar Morling
e1cf34dc43 DBZ-713 Extracting constaint for expected exception message 2018-07-03 14:41:47 +02:00
Gunnar Morling
c5f9d99644 DBZ-713 Adding overload of execute() that returns a result;
This simplifies interaction with execute(), as we don't need to alter collections, references as a side effect but instead can work with the return value.
2018-07-03 14:41:47 +02:00
Gunnar Morling
d6b897f43d DBZ-713 Applying database and collection filters within MongoPrimary;
This protects against authorization failures when listing collections
from DBs the connector user isn't authorized for. It also simplifies
usage of MongoPrimary#databaseNames() and collections() as consumers
don't need to apply filtering themselves.
2018-07-03 14:41:47 +02:00
Gunnar Morling
d72d3fcbc9 DBZ-782 Propagating authorization failures 2018-07-03 14:41:47 +02:00
Gunnar Morling
dcd6b01134 DBZ-644 Simplifying RelationalDatabaseSchema constructor;
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
2018-07-03 07:03:59 +02:00
Gunnar Morling
d7e196a18e DBZ-20 Initial import of Oracle connector based on XStream 2018-06-20 13:05:37 +02:00
echo.xu
15f622f823 DBZ-712. ignore initsync offset during connector start. 2018-06-20 10:41:46 +02:00
Gunnar Morling
3f20c8d8cc DBZ-561 Using less confusing server name in tests 2018-06-11 11:40:15 +02:00
Gunnar Morling
abe7863c28 DBZ-561 More consistent element ordering 2018-06-11 11:40:15 +02:00
Gunnar Morling
bb35d5e4bc DBZ-561 Adjusting test after rebase 2018-06-11 11:40:15 +02:00
Jiri Pechanec
4698dd2745 DBZ-561 Support nested updates 2018-06-11 11:40:15 +02:00
Sairam Polavarapu
4b67b14304 DBZ-561 Provided option to flatten structs in MongoDB unwrapping SMT. 2018-06-11 11:40:15 +02:00
Jiri Pechanec
32d591d392 DBZ-649 Use Avro-compatible field naming 2018-06-05 10:33:40 +02:00
Jiri Pechanec
d1488a7c48 DBZ-650 Fix test 2018-06-05 10:33:39 +02:00
Jiri Pechanec
3eb034f035 DBZ-649 Encoding arrays as struct 2018-06-05 09:57:19 +02:00
Jiri Pechanec
e00dc3beb4 DBZ-649 Graceful failures for heterogenous arrays 2018-06-05 09:57:19 +02:00
Jiri Pechanec
aa5581b524 DBZ-650 Fix the test breaking the build 2018-06-04 14:58:42 +02:00
Gunnar Morling
d556fca852 DBZ-650 Expanding test 2018-05-30 15:27:30 +02:00
Jiri Pechanec
972068129c DBZ-650 Add names to unwrapped MongoDB schemas 2018-05-30 15:02:38 +02:00
Jiri Pechanec
6e1e5f40ce Protect UnwrapFromMongoDbEnvelopeTestIT from inconsistent snapshot 2018-05-15 09:50:54 +02:00
Jiri Pechanec
232e7acde5 DBZ-680 Integration test for MongoDB unwrap SMT 2018-04-13 11:39:34 +02:00
Saulius Valatka
31c29459f7 DBZ-672 do not use cursor with no timeout 2018-03-29 14:47:57 +02:00
Saulius Valatka
a78c52cbdd DBZ-672 stop tailing the oplog when replicator is stopped 2018-03-29 14:47:57 +02:00
Jiri Pechanec
40a527b6ea DBZ-592 Make snapshot/init-sync false instead of null 2018-03-19 13:19:34 +01:00
Jiri Pechanec
2a35eae178 DBZ-648 InterrupedException handled by MongoDB 2018-03-09 20:43:57 +01:00
Gunnar Morling
eef1ad7c2a DBZ-630 Refactoring around MongoDB task context;
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8 DBZ-630 Pulling up getClock() to ConnectorTaskContext 2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6 DBZ-630 Unifying common start-up logic across connectors 2018-02-21 12:04:41 +01:00
Gunnar Morling
7fe45db831 DBZ-626 Renaming AvroValidator to SchemaNameAdjuster;
Making apparent that this functionality doesn't solely validate but returns a new name actually
2018-02-20 09:26:47 +01:00
Gunnar Morling
69df7c4ade DBZ-628 Using common properties for MongoDB connector 2018-02-20 06:34:42 +01:00
Jiri Pechanec
93259e8e11 DBZ-581 Simplified monitor thread shutdown 2018-02-13 14:40:54 +01:00
Jiri Pechanec
dbac1429b2 DBZ-581 Improve error handling for replicators 2018-02-13 14:40:54 +01:00
Gunnar Morling
39e08b7ec6 DBZ-582 Avoiding assignment to test class field from test methods 2018-02-13 10:19:18 +01:00
Gunnar Morling
2e0b9e5e33 DBZ-582 Moving TOMBSTONES_ON_DELETE to CommonsConnectorConfig for re-use 2018-02-13 10:19:18 +01:00
Raf Liwoch
38641f472d DBZ-582 Allowing users to configure whether tombstone events should be emitted for DELETE operations or not;
* Adding new configuration parameter `tombstones.on.delete`
* Using this new option in MongoDB connector
2018-02-13 10:19:18 +01:00
Gunnar Morling
1626f385ad DBZ-580 Using ChangeEventQueue in MongoDB connector 2018-02-11 10:05:06 +01:00