alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
7,090 Commits 3 Branches 0 Tags 51 MiB
46753e90e5
Commit Graph

406 Commits

Author SHA1 Message Date
Chris Cranford
151f0777ec DBZ-3788 Remove superfluous sort 2021-08-05 12:25:37 -04:00
Chris Cranford
3e72fb7e79 DBZ-3788 Avoid oplog collection scan 2021-08-05 12:25:37 -04:00
Chris Cranford
ed31ea9ffb DBZ-3668 Adjust ServiceLoader for SPI change 2021-07-03 16:31:52 +07:00
Gunnar Morling
b84c3c870d DBZ-3668 Extracting SPI package 2021-07-03 16:31:52 +07:00
Chris Cranford
7f7e23b110 DBZ-3668 Add ServiceLoader hook CloudEvents converter 2021-07-03 16:31:52 +07:00
Sergei Morozov
db105baeba DBZ-2975: Add partition awareness to source task components 
Co-authored-by: Mike Kamornikov <mikekamornikov@gmail.com>
 2021-07-03 10:56:34 +02:00
Sergei Morozov
657a6d2542 DBZ-2975: Extract offset context from object states to method signatures 2021-06-09 16:02:33 +02:00
chriscollingwood
6225e124bb DBZ-3522 Ensure that there is an active session on the replica set mongo client before using clusterDescription and ServerDescriptions to determine primary node. 2021-06-03 08:28:58 +02:00
Chris Cranford
d31d562376 DBZ-3331 Do not stream if offset no longer in the MongoDB oplog 2021-05-14 09:07:32 +02:00
Gunnar Morling
42a7b74725 DBZ-3403 Misc. improvements 2021-04-14 11:14:07 +02:00
Hossein Torabi
ba048cf47d DBZ-3403 Semi Optimisation on MongoDB and MySQL connector for skipped.operations 2021-04-14 11:14:07 +02:00
JLDLaughlin
06b0475f17 DBZ-2911 Add additional LSN information to all sources 
Adds a new "sequence" field to all sources via the AbstractSourceInfo
class. Specifically, this field is required to deduplicate records
from PostgreSQL sources in O(1) time. The sequence field is a stringified
list of metadata. For PostgreSQL sources, this will include the last
committed LSN and the current LSN. For other sources, this will be NULL
until implemented. A new integration test was added for PostgreSQL.
 2021-03-26 09:19:17 +01:00
Gunnar Morling
c7b7d5303a DBZ-3218 Avoiding duplicated validation (again ;) 2021-03-19 15:04:20 +01:00
Gunnar Morling
178140ae9c DBZ-3218 Removing superfluous method parameter 2021-03-19 15:04:20 +01:00
Anisha Mohanty
c761d92b2a DBZ-3218 Add separate method for logging warnings 2021-03-19 15:04:20 +01:00
Jiri Pechanec
495be2659b DBZ-2511 Dedicated logger for schema name adjuster 2021-03-18 14:42:34 +01:00
Frank Koornstra
cdc7abdf28 DBZ-3278 Add timeout exceptions to MongoDBErrorHandler 
When connecting to the server or executing some queries, due to
networking issues, timeouts sometimes occur which would crash the whole
connector. By including a few extra exceptions in the error handler we
make sure it will retry in these situations.
 2021-03-15 12:27:03 +01:00
Thomas Prelle
15887077ff DBZ-2680 Sanitize field name do not santize sub struct field 
Their are an issue during the sanitisation to avro on mongodb connector.
The sanitization failed on sub struct like
{​
"_id": "222",
"metrics": {​
"metric::fct":

{​"min": 0, "max": 1 }
​
}
 2021-02-17 09:09:00 +01:00
ani-sha
5f8a22a5df DBZ-2858 Make field descriptions consistent for time values 2021-02-16 09:55:35 +01:00
Gunnar Morling
6d441bc538 DBZ-2938 Centralizing listener handling; avoiding some empty methods 2021-01-19 15:16:46 +01:00
rkerner
15064f42b1 DBZ-2831 allow non default Mongo DB ports 2020-12-10 05:33:00 +01:00
Gunnar Morling
9bb53b9e2d DBZ-2138 Static loggers 2020-11-30 10:33:05 +01:00
Jiri Pechanec
83df96ff70 DBZ-2138 Add defensive checks 2020-11-30 10:33:05 +01:00
Jiri Pechanec
349fcdad07 DBZ-2138 Handle lsid as Binary for MongoDB 4.2 2020-11-30 10:33:05 +01:00
Jiri Pechanec
e8aecd7b45 DBZ-2138 Use MongoDB 4.x driver 2020-11-30 10:33:05 +01:00
Jiri Pechanec
ea019c3b2c DBZ-1322 Removed dependency on deprecated MongoDB driver code 2020-11-27 17:37:57 +01:00
vanhoale
17a424d9ac DBZ-2662 Handle ChangeEventQueue by the size in bytes 2020-11-16 08:39:33 +01:00
Chris Cranford
aea26a2949 DBZ-2742 Migrate initial.sync.max.threads to snapshot.max.threads 2020-11-11 09:08:05 +01:00
Faizan
f5f6abfc7b DBZ-2496 Corrections to code formatting and improving doc text for consistency 2020-10-22 06:30:40 +02:00
Faizan
0adaed07d2 DBZ-2496 Updating documentation and test case as per review comment 2020-10-22 06:30:40 +02:00
Faizan
24f375aa3d DBZ-2496 Corrections to code formatting 2020-10-22 06:30:40 +02:00
Faizan
925c7e35f5 DBZ-2496 Added snapshot override behavior for MongoDB using filter query 2020-10-22 06:30:40 +02:00
Kaushik Iyer
7d9e8fdc46 DBZ-2456 fixes after rebase 2020-10-16 10:13:48 +02:00
Kaushik Iyer
d758491dba DBZ-2456 style change fixes 2020-10-16 10:13:48 +02:00
Kaushik Iyer
e08cea175f DBZ-2456 style change fixes 2020-10-16 10:13:48 +02:00
Kaushik Iyer
e2e6dc0932 DBZ-2456 selective snapshot implementation 2020-10-16 10:13:48 +02:00
Chris Cranford
85821b4975 DBZ-2597 Suggested changes 2020-10-03 13:36:17 +02:00
Chris Cranford
0126e44ad1 DBZ-2597 Fix ConcurrentModificationException when schema modified in sharded cluster 2020-10-03 13:36:17 +02:00
Gunnar Morling
eeec1e7638 DBZ-2585 Simpler boolean condition 2020-10-01 10:27:43 +02:00
Thomas Prelle
d4ad6d525d DBZ-2585 Skipping empty arrays when preparing values for Avro 2020-10-01 10:27:43 +02:00
Gunnar Morling
1dd6293325 DBZ-2569 Adding some line breaks 2020-09-30 09:16:23 +02:00
Thomas Prelle
3951540082 DBZ-2569 : MongoDB ExtractNewDocumentState can not extract array of array 2020-09-30 09:14:34 +02:00
Chris Cranford
0040eca740 DBZ-2316 Throw DataException with non-uniform array elements 
While using array.encoding=array, array elements that do not have
uniform structures should throw a DataException with a meaningful
message rather than throw a NullPointerException.
 2020-09-29 22:12:43 +02:00
Ganesh Ramasubramanian
6ca025b1bb DBZ-2504: Add support for field and header prefix 
Added support for field and header prefix.
Updated the documentation to include the options
 'add.fields.prefix' and 'add.headers.prefix'.
 2020-09-16 11:31:05 +02:00
Chris Cranford
e5c01c6b09 DBZ-137 Fix compilation problems 2020-09-14 12:33:43 +02:00
AndreyIg
77662a0dde DBZ-137 contribution to the core, Oracle related 2020-09-14 12:33:43 +02:00
Gunnar Morling
193541800d DBZ-2455 Adding constant for some MongoDB specific field names 2020-09-02 17:41:49 +02:00
Chris Cranford
642fda71d9 DBZ-2455 Fix test compatibility with MongoDB 3.2 2020-09-02 17:41:49 +02:00
Chris Cranford
2b5c2e3ec7 DBZ-2455 ExtractNewDocumentState support patch in add.fields 2020-09-02 17:41:49 +02:00
Chris Cranford
f74da055b8 DBZ-2429 Clarify Field descriptions for include/exclude for MongoDB 
The include/exclude Field descriptions do not explicitly state if
the input argument is a comma-separated list of regex values and
should since that the expected format of the option.
 2020-09-01 09:50:34 +02:00
Chris Cranford
2e800756b8 DBZ-2464 Add debezium.test.records.waittime support 2020-09-01 07:49:10 +02:00
Chris Cranford
94a4019f9a DBZ-2464 Fix intermittent failures with FieldRenamesIT 2020-09-01 07:49:10 +02:00
rkerner
2c5c093587 DBZ-2171 avoid divisive language 2020-08-21 09:29:04 +02:00
Chris Cranford
f3dbfdef3c DBZ-2400 Fix test failures 2020-08-05 09:13:43 +02:00
Chris Cranford
e9e274ede1 DBZ-2400 Change precedence 2020-08-05 09:13:43 +02:00
Chris Cranford
cc4e78bbff DBZ-2400 Deprecate mongodb.poll.internval.sec for mongodb.poll.interval.ms 2020-08-05 09:13:43 +02:00
Chris Cranford
931fecd216 DBZ-2225 Introduce MongoDB timeout configuration options 
This adds configurable timeout options for:
* server.selection.timeout.ms
* connect.timeout.ms
* socket.timeout.ms
 2020-08-05 09:13:43 +02:00
jgraf50
a7faa7f42f DBZ-2141 Code formatting 2020-07-09 11:38:32 -04:00
jgraf50
a7d66ed4e4 DBZ-2141 Fix formatting 2020-07-09 11:38:32 -04:00
jgraf50
ce7a44cb71 DBZ-2141 Narrow to only MongoSocketException errors 2020-07-09 11:38:32 -04:00
jgraf50
5ccec1d96e DBZ-2141 Add custom MongoDB connector error handler to do retries on connection errors 2020-07-09 11:38:32 -04:00
Chris Cranford
86b4e3f5b6 DBZ-2286 Fix test slowness with FieldRenamesIT 2020-07-09 09:13:27 +02:00
Chris Cranford
c718127faa DBZ-2264 Fix compatibility with MongoDB 3.x 2020-07-09 09:11:28 +02:00
Gunnar Morling
324eed10a3 DBZ-2216 Reducing log level to stay below on max output threshold on CI 2020-06-18 18:20:52 +02:00
Jiri Pechanec
31f03881e5 DBZ-2216 Formatting 2020-06-18 18:20:52 +02:00
Chris Cranford
26044a478d DBZ-2216 Fix transaction-id resolution logic 2020-06-18 18:20:52 +02:00
Chris Cranford
8b439a4cd9 DBZ-2216 Fix backward compatibility failure with V1 source block 2020-06-18 18:20:52 +02:00
Chris Cranford
4f561600a4 DBZ-2216 Fix transaction-id resolution with MongoDB 4.2 2020-06-18 18:20:52 +02:00
Gunnar Morling
9479f356d9 DBZ-2116 Snapshot fix for documents with a field named "op" 2020-06-09 15:19:49 -04:00
Chris Cranford
11f5c59c10 DBZ-1791 Formatting and suggested changes 2020-05-13 21:36:12 +02:00
Chris Cranford
2eb2245f07 DBZ-1828 Align ExtractNewDocumentState SMT with deprecations 2020-05-13 21:36:12 +02:00
Chris Cranford
5ae8313e2e DBZ-1791 Add add.fields and add.headers options to MongoDB flattening SMT 2020-05-13 21:36:12 +02:00
Jiri Pechanec
e350d74b17 DBZ-1904 Public facing schema history topic 2020-05-04 12:31:05 +02:00
Chris Cranford
230ddaebdc DBZ-1953 Generalized SkipWhenDatabaseVersion functionality 2020-04-29 12:27:38 +02:00
Gunnar Morling
03ce403972 DBZ-1750 Separating config definition editor and immutable config definitions 2020-04-24 15:35:28 +02:00
Jiri Pechanec
37f0dbcaf8 DBZ-1750 Limit access to parent defs; dont share the parent defs 2020-04-24 15:35:28 +02:00
Jiri Pechanec
af90507fa6 DBZ-1750 Centralize shared config definitions 2020-04-24 15:35:28 +02:00
Gunnar Morling
905629973e DBZ-1968 Removing SMTs deprecated in 0.10 2020-04-15 15:11:32 +02:00
Chris Cranford
f4edc018d4 DBZ-1859 Cleaned up test 2020-04-15 08:04:23 +02:00
Chris Cranford
f5a4d80416 DBZ-1859 Moved metrics to subpackage 2020-04-15 08:04:23 +02:00
Chris Cranford
e87bfefdf7 DBZ-1859 Renamed MetadataEvent to ConnectorEvent 2020-04-15 08:04:23 +02:00
Chris Cranford
57af80afd5 DBZ-1859 Enhance MongoDB connector metrics 2020-04-15 08:04:23 +02:00
Chris Cranford
65c8564d66 DBZ-1925 Introduced SkipWhenKafkaVersion annotation support 2020-04-07 09:01:20 +02:00
Chris Cranford
d5a3bea4bd DBZ-1925 Fix incompatible tests with Kafak 2.4.1 2020-04-07 09:01:20 +02:00
rk3rn3r
7f36b3c08f DBZ-1531 added headers for primary key update events to reference the original key 2020-03-25 12:33:38 +01:00
Chris Cranford
2f6308bee9 DBZ-1874 Resolve metrics test failures 2020-03-20 09:37:40 +01:00
Chris Cranford
ddfcdcc20d DBZ-1874 Use seconds to get timestamp from legacy source info blocks 2020-03-20 09:37:40 +01:00
Chris Cranford
6590c7d6c2 DBZ-1880 Fix compatibility with event metadata and V1 source info block 2020-03-20 09:37:40 +01:00
Chris Cranford
afdad564e2 DBZ-1874 Recommended changes 2020-03-20 09:37:40 +01:00
Chris Cranford
c4fb8b67d3 DBZ-1874 Add check for tx support in test. 2020-03-20 09:37:40 +01:00
Chris Cranford
9e31c4fb04 DBZ-1874 Stablize MongoDB transaction metadata support 
* Added TransactionMetadataIT use test
* Correctly resolve transaction metadata and field emission
 2020-03-20 09:37:40 +01:00
Chris Cranford
d6114bb99f DBZ-1879 Introduce JMX tests 2020-03-17 08:18:00 +01:00
Chris Cranford
04370f2bf2 DBZ-1871 Reduce log verbosity for Travis CI 2020-03-12 22:23:14 +01:00
Chris Cranford
c8f95d853d DBZ-1867 Fix failing MongoDB tests on MongoDB 3.2 2020-03-12 22:23:14 +01:00
Gunnar Morling
449154beaa DBZ-1863 Centralizing coordinator shutdown 2020-03-12 11:28:08 +01:00
Gunnar Morling
3437da00d7 DBZ-1723 Simplifying shutdown handling; 
doStop() is already called from stop(), no need to do it again via the error handler.
 2020-03-11 13:57:23 +01:00
Gunnar Morling
c156d30048 DBZ-1723 Centralizing stopping() logic in BaseSourceTask 2020-03-11 13:57:23 +01:00
Gunnar Morling
8ce69dc597 DBZ-1723 Encapsulating common poll() and stop() logic in BaseSourceTask 2020-03-11 13:57:23 +01:00
Gunnar Morling
6318cf0f2b DBZ-1831 Pulling up connector option retrieval to CommonConnectorConfig; 
* Expanding test
* Adding Sayed to COPYRIGHT.txt
 2020-03-11 12:43:25 +01:00
Sayed Mohammad Hossein Torabi
63a462dbc0 DBZ-1831 Support MongoDB Oplog operations as config 2020-03-11 12:43:25 +01:00
Chris Cranford
baffcbdfd9 DBZ-1726 Added TransactionMonitor hooks 2020-03-11 10:15:00 +01:00
Chris Cranford
6f1eb49b32 DBZ-1726 Fix formatting failures 2020-03-11 10:15:00 +01:00
Chris Cranford
ab8359b9cc DBZ-1726 Implemented getting transaction-id metadata 2020-03-11 10:15:00 +01:00
Chris Cranford
3a4788cc33 DBZ-1726 Introduced AbstractSnapshotChangeEventSource and refactor implementations 2020-03-11 10:15:00 +01:00
Gunnar Morling
8f1edb0881 DBZ-1726 Removing some unused fields 2020-03-11 10:15:00 +01:00
Chris Cranford
38cf4ceae1 DBZ-1726 Introduce DataCollectionId#identifier() 2020-03-11 10:15:00 +01:00
Chris Cranford
4b6f0588c9 DBZ-1726 Fix oplog cursor filter 2020-03-11 10:15:00 +01:00
Chris Cranford
71bf17b24f DBZ-1726 Fix formatting 2020-03-11 10:15:00 +01:00
Chris Cranford
4e27c90028 DBZ-1726 Move MongoDB connector to common framework 2020-03-11 10:15:00 +01:00
Chris Cranford
4752078ee0 DBZ-1848 Skip adding rename field if source field does not exist. 2020-03-04 17:54:03 +01:00
Gunnar Morling
8744bb6a16 DBZ-1781 Making idObjToJson() method more specific 2020-03-04 09:10:49 +01:00
jgraf50
c84a8a28b4 DBZ-1781 Added support for sharded MongoDB collections. 2020-03-04 09:10:49 +01:00
Jiri Pechanec
a998db9caa DBZ-1767 Support field sanitization for MongoDB unwrap 2020-02-12 18:49:32 +01:00
Gunnar Morling
dd5c2b8be9 DBZ-1766 Unifying handling of offset commits 2020-02-12 15:44:45 +01:00
Jiri Pechanec
77f9a61b95 DBZ-1751 Centralize envelope schema naming 2020-02-04 20:37:19 +01:00
Jiri Pechanec
ba10a229b6 DBZ-1052 CE support for tx metadata in data messages 2020-02-03 09:29:10 +01:00
Jiri Pechanec
60056b0a94 DBZ-1052 Transaction block moved to envelope 2020-02-03 09:29:10 +01:00
Jiri Pechanec
b0eb571f9f DBZ-1052 Transaction metadata for SQL Server 2020-02-03 09:29:10 +01:00
Gunnar Morling
d5bbc7a123 DBZ-1717 Misc. clean-up; 
* Expanding test
* Removing obsolete deprecation warning
* Adding Jan Uyttenhove to COPYRIGHT.txt
 2020-01-28 12:41:12 +01:00
Jan Uyttenhove
144936363f DBZ-1717 Usage of java.util.Date and Kafka Connect Timestamp to create Date(Time) field types 2020-01-28 12:41:12 +01:00
Gunnar Morling
8c33a4e147 DBZ-1292 Catering for the differences in data schema; 
* MongoDB doesn't have "before", but "patch"
* Also removing some redundancies and making RecordParser immutable
* Reducing log verbosity during tests
 2020-01-16 08:01:56 +01:00
Gunnar Morling
7c18a6d8d5 DBZ-1292 Removing static state in CloudEventsConverterTest 2020-01-16 08:00:45 +01:00
GraySmog
d0e4b3c0af DBZ-1292 Delete data.schema.registry.url config option 2020-01-16 08:00:45 +01:00
GraySmog
915c8cafd9 DBZ-1292 CloudEvents converter for JSON and Avro format 2020-01-16 08:00:45 +01:00
Jiri Pechanec
4102d39aba DBZ-1168 Suport non-admin authSources 2020-01-06 12:28:49 +01:00
Gunnar Morling
bf25aeb1de DBZ-1215 Formatting 2019-11-25 11:52:48 +01:00
Gunnar Morling
0fa29a0163 DBZ-1215 Misc. clean-up; 
* long ms -> Instant
* Using OptionalLong as return value for optional value
 2019-11-22 11:42:42 +01:00
Jiri Pechanec
6cadff8c36 DBZ-1215 Added support for transactions 2019-11-22 11:42:42 +01:00
Gunnar Morling
2c569977d7 DBZ-1392 Applying formatter; 
* Keeping indentation formatting for JSON


Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
Co-authored-by: Chris Cranford <chris@hibernate.org>
 2019-11-01 13:20:43 +02:00
Chris Cranford
c25bb23a04 DBZ-1583 Propagate Replicator exception to Kafka Connect 2019-10-29 16:17:27 +01:00
Jiri Pechanec
b2a70889f5 DBZ-1563 Collection names can contain dots 2019-10-23 14:40:15 -04:00
Jiri Pechanec
b88d9b06d4 DBZ-1513 Align test names 2019-10-07 18:38:35 +02:00
Jiri Pechanec
2c4cb83259 DBZ-1513 Centralize handling of non-data messages in SMT 2019-10-07 18:38:35 +02:00
Gunnar Morling
92885ebe99 DBZ-1392 Removing trailing whitespace from Java files 2019-10-07 12:11:43 +03:00
Gunnar Morling
8d6f5409b1 DBZ-1392 Moving static imports up and applying this to all files 2019-10-07 12:11:43 +03:00
Chris Cranford
ca5517746a DBZ-1341 Checkstyle flag missing whitespace 2019-10-03 19:44:43 +02:00
Chris Cranford
7442670692 DBZ-1355 Fix checkstyle problems detected after upgrade to 8.19 2019-10-03 19:44:43 +02:00
Chris Cranford
5ac2318d84 DBZ-1391 Do not allow code after closing brace (#1065) 
* DBZ-1391 Do not allow code after closing brace
 2019-10-02 20:44:40 +02:00
jerrinot
3064a4af06 DBZ-1438 Fix InterruptedException handling 
It appears there is an intent to restore the interruption flag
when catching InterruptedException. However the old code was just
checking the current state of the flag instead of raising it.
 2019-09-24 12:51:48 +02:00
Ching Tsai
80210f7f8b DBZ-1442 Add test for a rewrite delete event 2019-09-13 11:23:26 +02:00
Ching Tsai
b8d565ffe0 DBZ-1442 Add @FixFor 2019-09-13 11:23:26 +02:00
Ching Tsai
486973ad12 DBZ-1442 Porting insert source fields feature to ExtractNewDocumentState 2019-09-13 11:23:26 +02:00
Gunnar Morling
1e0859588f DBZ-1464 Misc. clean-up; 
* logger -> LOGGER
* making op literals map unmodifiable
* Adding Lev Zemlyanov to COPYRIGHT.txt
 2019-09-06 11:02:45 +02:00
Lev Zemlyanov
8cbb8eb63b DBZ-1464 Fixed NPE when op is not u/i/d 
Signed-off-by: Lev Zemlyanov <lev@confluent.io>
 2019-09-06 11:02:27 +02:00
Renato Mefi
a9c8a825c3 DBZ-1430 Support heartbeat ExtractNewDocumentState 2019-08-14 23:56:45 +02:00
Gunnar Morling
287551294f DBZ-1198 Adding comments 2019-07-11 07:43:32 +02:00
Jiri Pechanec
0316ac7fc0 DBZ-1198 Heartbeat for MongoDB connector 2019-07-10 11:16:26 +02:00
Jiri Pechanec
13d638fd43 DBZ-1356 Metrics for schema history (#947) 
* DBZ-1356 Metrics for schema history
 2019-07-02 13:26:17 +02:00
Gunnar Morling
a991ea97d7 Handling option for emitting tombstones conistently 2019-07-01 08:27:33 +02:00
Jiri Pechanec
72b751b528 DBZ-1333 Upgrade MongoDb driver to 3.10.1 2019-06-17 20:53:07 +02:00