alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
7,296 Commits 3 Branches 0 Tags 51 MiB
3cdcf795d8
Commit Graph

1478 Commits

Author SHA1 Message Date
harveyyue
0842e5cd0e DBZ-5366 MySQL Connector column hash v2 does not work 2022-07-06 14:23:57 -04:00
Paweł Malon
707a43d3bc DBZ-5367 Fixed the problem where JSON expanding failed for Outbox event payloads that contained nested arrays where first array was empty and further array contained a Struct 
With proposed changes empty arrays are skipped, as it allows us to delay defining the schema for them until an array with elements is found, as it seems that there is no way to modify a schema that was already defined for a given field
 2022-07-06 13:47:54 -04:00
Debezium Builder
6fd1441af3 [maven-release-plugin] prepare for next development iteration 2022-07-01 09:21:49 +00:00
Debezium Builder
99402af634 [maven-release-plugin] prepare release v2.0.0.Alpha3 2022-07-01 09:21:48 +00:00
Vojtech Juranek
ac06623459 DBZ-5240 Move table types into separate method 
This will allow connectors to override the default values and return
types the connector supports.
 2022-06-30 10:46:54 +02:00
Gunnar Morling
cb53332561 DBZ-5335 Formatting 2022-06-30 09:54:13 +02:00
Mickael Maison
5ff71aa992 DBZ-5335 Fix formatting issues in the connectors configdefs 2022-06-30 09:35:54 +02:00
harveyyue
20dc353262 DBZ-5170 Mysql Commit Timestamp 2022-06-27 15:50:52 +02:00
Naveen Kumar KR
b75c9e635b DBZ-5277 ChangeEventQueue - Support Multiple Writer (doEnqueue) and Reader (poll) Threads 
* ReentrantLock based write and read coordination
* Condition based Inter Thread communication for isFull and isNotFull conditions
 2022-06-27 15:37:00 +02:00
Naveen Kumar KR
823a49fff4 DBZ-5277 ChangeEventQueue Test with Multiple Writer and Reader Threads 
Assertion would fail if one or both of the scenario occurs

1) Deadlock between Writer and Reader Threads
2) Thread Starvation between Writer and Reader Threads
 2022-06-27 15:37:00 +02:00
Jiri Pechanec
9ada2f28ce DBZ-4902 Change for exponential strategy 2022-06-27 14:29:51 +02:00
kanha gupta
1b17372d2d DBZ-4902 Replace long durations with Duration 2022-06-27 14:29:51 +02:00
Chris Cranford
ef99595d91 DBZ-5270 Reduce table schema capture logging verbosity 
When enabling the `database.history.store.only.captured.tables.ddl`,
the logging in RelationalSnapshotChangeEventSource does not take into
account this setting when logging, leading to a false impression of
what tables will actually be captured in the schema history.
 2022-06-27 08:47:34 +02:00
Jiri Pechanec
5e1ae24e92 DBZ-5272 Notify all threads in case of mutliple writers 2022-06-21 07:08:39 +02:00
Chris Cranford
3fb5bf1339 DBZ-4451 Introduce SchemaChangeEvent factory methods 2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081 DBZ-4451 Store previous table id in alter schema change events 
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
 2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5 DBZ-4834 Fix incremental snapshot of table added to include list 2022-06-16 08:39:16 +02:00
Vojtech Juranek
25eba9749d DBZ-5198 Add support for SQL server specific table delimiters 
Add suppoprt for `TableId` delimiters and provide implementation for
SQL server. SQL server allows to use reserved words in table names or
names with spaces, but they have to wrapped by `[]` characters, e.g.
`[dbname].[table name]`.

Debezium can handle spaces e.g. in table include list, but fails when
parsing snapshot select, therefore the parsing with predicates is used
only for parsing spanshot select for now. If useful, it can be used
later on on other places as well.
 2022-06-13 10:57:15 +02:00
Vojtech Juranek
796e17b2c6 DBZ-5198 Move predicates for parsing TableId into dedicate class 
Moving the predicates into parsing context decouples the predicates from
the private `TableIdTokenizer` class. Moving the predicates into
separate class would allow us to provide database specific
implementations.
 2022-06-13 10:57:15 +02:00
Debezium Builder
6fff3a2876 [maven-release-plugin] prepare for next development iteration 2022-06-09 11:49:26 +00:00
Debezium Builder
b2e2e351df [maven-release-plugin] prepare release v2.0.0.Alpha2 2022-06-09 11:49:26 +00:00
Vojtech Juranek
4a48eb33df DBZ-5045 Remove legacy V1 source info 2022-06-08 09:11:00 +02:00
Vojtech Juranek
250e160b81 DBZ-5045 Removed unused methods 2022-06-08 09:11:00 +02:00
Vojtech Juranek
ef8d7f7af7 DBZ-5045 Remove deprecated OBSOLETE_NAME_FOR_SKIP_FAILURE_HANDLING option value 2022-06-08 09:11:00 +02:00
Vojtech Juranek
ff5ddd7959 DBZ-5045 Remove deprecated FIELD_PAYLOAD_ID 
`table.field.event.payload.id` is replaced by `table.field.event.key`.
Also fix default value of `table.field.event.key` which was missing.
 2022-06-08 09:11:00 +02:00
Vojtech Juranek
0530dc2de9 DBZ-5045 Remove deprecated STORE_ONLY_MONITORED_TABLES_DDL 2022-06-08 09:11:00 +02:00
Vojtech Juranek
4e7b30b718 DBZ-5045 Remove deprecated DEBEZIUM_OPERATION_INVALID_BEHAVIOR 2022-06-08 09:11:00 +02:00
Vojtech Juranek
b0b3066ed5 DBZ-5045 Remove deprecated COLUMN_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
5031594b75 DBZ-5045 Remove deprecated COLUMN_WHITELIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
b3a6131cee DBZ-5045 Remove deprecated DATABASE_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
0cbb57f4bf DBZ-5045 Remove deprecated DATABASE_WHITELIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
f84893a8b4 DBZ-5045 Remove deprecated SCHEMA_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
261188482f DBZ-5045 Remove deprecated SCHEMA_WHITELIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
6abbcee6a3 DBZ-5045 Remove deprecated TABLE_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
db08e894fc DBZ-5045 Remove deprecated TABLE_WHITELIST 2022-06-08 09:11:00 +02:00
harveyyue
acb743bd34 DBZ-5206 Remove the deprecated monitored tables metrics 2022-06-07 16:13:29 -04:00
Chris Cranford
3e820c4047 DBZ-4251 Adjust semantics with collection removal 2022-06-02 07:30:07 +02:00
Chris Cranford
187e3250a3 DBZ-4251 Add regex support for stop signal 2022-06-02 07:30:07 +02:00
Chris Cranford
9d71296b5c DBZ-4251 Support stopping incremental snapshots 2022-06-02 07:30:07 +02:00
Jiri Pechanec
88b7ae2050 DBZ-5136 Move sourceInfo to common class 2022-06-01 16:56:29 +02:00
Mark Bereznitsky
c64b8af31a DBZ-5136 reintroduce the CommonOffsetContext class and refactor common functionality into it 2022-06-01 16:56:29 +02:00
Mark Bereznitsky
2d72dae649 DBZ-5136 further refining of the algorithm for determining position in snapshot and IT test fixes 2022-06-01 16:56:29 +02:00
Jiri Pechanec
08e8c3d795 DBZ-5136 Move common class to common package 2022-06-01 16:56:29 +02:00
Andrew Walker
d0759461a4 DBZ-5136: Convert RelationalOffsetContext to CommonOffsetContext 2022-06-01 16:56:29 +02:00
Andrew Walker
01a81f8bc1 DBZ-5136: Refactored SnapshotContext setters 2022-06-01 16:56:29 +02:00
Andrew Walker
b6222d42bf DBZ-5136: Add first flag and refactor OffsetContext 2022-06-01 16:56:29 +02:00
Andrew Walker
593c2769e6 DBZ-5136: Add last in table flag 2022-06-01 16:56:29 +02:00
Chris Cranford
15e9231c30 DBZ-5179 Use LRUCacheMap to avoid high memory usage 2022-06-01 15:53:55 +02:00
Chris Cranford
9fa5a48274 DBZ-5011 Register MongoDB metrics with task id when more than one task 2022-06-01 09:41:36 +02:00
harveyyue
9a3afac243 DBZ-5180 Put the MysqlPartition values to MDC getting NPE in logging context phase 2022-06-01 09:17:04 +02:00
Vojtech Juranek
5550057034 DBZ-5042 Add abstract partition with DB name 
DB names is used by logging facilities. As the DB name is required only
for relational DBs, also this abstract class is provided only for
relational DB connectors.

Use this abstract class for SQL server connector, where DB name logging
is already implemented.
 2022-05-27 14:20:30 -04:00
Jiri Pechanec
b85cb275b1 DBZ-5089 Use atomic reference 2022-05-27 12:46:39 +02:00
connorszczepaniak-wk
8693b4f008 DBZ-5089: add partition to the outbox event router in additional fields 
DBZ-5089: undo format change

DBZ-5089: add tests for the partition routing

DBZ-5089: remove nonsensical comment

DBZ-5089: partition fieldname is a string; test properly

DBZ-5089: fix one of the failing tests

DBZ-5089: pass args through

DBZ-5089: fix oracle implementation; add configs for mongo connector

DBZ-5089: put partition in additional placement

DBZ-5089: restore formatting
 2022-05-27 12:46:39 +02:00
harveyyue
e45e8bd43b DBZ-5172 Expose more useful exception info with building the field default value schema 2022-05-27 12:41:12 +02:00
Rotem Adhoh
8ef7224543 DBZ-5166 Fixed ExtractNewRecordState SMT to pass all columns' data as is, i.e: without potential unwanted values manipulation. 
Prior to this fix, when inserting an explicit NULL value to a nullable column which has a default value, resulted in a record which contains the corresponding field with the default value (instead of NULL) after applying ExtractNewRecordState transformation.
A new unit test was added as well.
 2022-05-27 12:33:23 +02:00
harveyyue
3ada04b522 DBZ-3865 Debezium MySql connector does not capture floating point numbers with the right precision 2022-05-25 12:54:42 -04:00
Chris Cranford
7d8c9486d6 DBZ-5119 Move Heartbeat creation to ConnectorConfig & descendants 2022-05-25 09:59:59 +02:00
Chris Cranford
7ff7826743 DBZ-5119 Pull heartbeat.action.query support into debezium-core 2022-05-24 12:54:40 +02:00
Chris Lee
2a8456d21b DBZ-5123 data-collection regex snapshot signal 2022-05-24 12:46:28 +02:00
harveyyue
52ba58ccc3 DBZ-5098 Introduce a new field "ts_ms" to identify the process time for schema change event 2022-05-18 14:21:39 +02:00
harveyyue
f66b07cd80 DBZ-5141 Should be sleep with REGISTRATION_RETRY_DELAY when registry MBean failed 2022-05-18 12:19:31 +02:00
Jiri Pechanec
8643ad3a22 DBZ-5138 Unify happy/unhappy common path 2022-05-18 07:42:33 +02:00
Chris Cranford
e075d6b2eb DBZ-5138 Fix tracking MBean registration 2022-05-17 20:02:11 -04:00
Jiri Pechanec
d56f6ef6e0 DBZ-5138 Retry failed metrics registration; allow starting without metrics 2022-05-17 20:02:11 -04:00
Sergei Morozov
1ad97b9d66 DBZ-5130: Log history recover progress every two seconds 2022-05-16 09:17:24 +02:00
Sergei Morozov
65e97796e7 DBZ-5129: Use buffered reader when reading history from file 2022-05-13 09:38:43 +02:00
Jiri Pechanec
4396fc3ec3 DBZ-4600 Remove deprecated class 2022-05-10 15:25:35 +02:00
Jiri Pechanec
5e94c1d498 DBZ-4600 Fix formatting 2022-05-10 15:08:46 +02:00
Nathan Bradshaw
c76e697f2f DBZ-4600 - Add BinaryDataConverter to support byte[] and ByteBuffer source data 2022-05-10 15:08:46 +02:00
Nathan Bradshaw
b8b8dffac4 DBZ-4600 - Deprecate ByteBufferConverter from 1.9 2022-05-10 15:08:46 +02:00
Nathan Bradshaw
4bf01a95a2 DBZ-4600 - Start to rough-out ByteArrayConverter 
- Add delegator support to ByteArrayConverter. Add missing tests
- Add required headers, correct typos
 2022-05-10 15:08:46 +02:00
Mark Bereznitsky
ae00729aa2 DBZ-5087 Added a decorator for result sets that can cancel the parent statements 2022-05-06 07:52:00 +02:00
Sergei Morozov
f6546e4d87 DBZ-5075: Remove unused legacy MySQL code 2022-05-03 13:46:23 +02:00
harveyyue
d03e555144 DBZ-5072 Update the two properties to topic-routing doc 2022-05-03 13:44:58 +02:00
harveyyue
25205b233c DBZ-5072 Config the cache size property for ByLogicalTableRouter caches 2022-05-03 13:44:58 +02:00
Debezium Builder
901825810a [maven-release-plugin] prepare for next development iteration 2022-04-28 13:51:13 +00:00
Debezium Builder
05dcb45647 [maven-release-plugin] prepare release v2.0.0.Alpha1 2022-04-28 13:51:12 +00:00
Chris Cranford
09e58c5a5c DBZ-5006 Allow "/" characters in Oracle table names 2022-04-27 08:52:29 -04:00
Sergei Morozov
fd46a670ad DBZ-5033: Ignore null offsets when recovering database history 2022-04-26 11:40:21 +02:00
Sergei Morozov
e79305cf93 DBZ-5033: Update Javadocs in DatabaseHistory 2022-04-26 11:40:21 +02:00
Vojtech Juranek
da5378d095 DBZ-4949 Adjust one test to use features > Java 11 
Adjust one test to use multiline strings, which is a feature added in
Java 15 and thus verify Java source level > 11 works for tests.
 2022-04-22 05:53:51 +02:00
Brad Morgan
8e25c52585 DBZ-4989 Passthrough payload schema in EventRouterDelegate 2022-04-14 12:17:53 +02:00
Jiri Pechanec
053bf602af DBZ-4979 Guard against NPE for non-table related DDLs 2022-04-12 09:36:14 -04:00
Debezium Builder
4a6de64c25 [maven-release-plugin] prepare for next development iteration 2022-04-05 14:49:52 +00:00
Debezium Builder
683851fd90 [maven-release-plugin] prepare release v1.9.0.Final 2022-04-05 14:49:51 +00:00
Debezium Builder
ab7fe3b96a [maven-release-plugin] prepare for next development iteration 2022-03-25 16:45:06 +00:00
Debezium Builder
d19fc234ee [maven-release-plugin] prepare release v1.9.0.CR1 2022-03-25 16:45:06 +00:00
Sergei Morozov
010f8c9d50 DBZ-3535: Rework CloudEventsConverter 2022-03-24 09:37:05 +01:00
Sergei Morozov
854186e846 DBZ-3535: Rework ByLogicalTableRouter 2022-03-24 09:37:05 +01:00
Sergei Morozov
139b129cf5 DBZ-3535: Rework TransactionMonitor 2022-03-24 09:37:05 +01:00
Sergei Morozov
fccc31c35b DBZ-3535: Rework ConnectTableChangeSerializer 2022-03-24 09:37:05 +01:00
Sergei Morozov
b258674782 DBZ-3535: Introduce schema.name.adjustment.mode 2022-03-24 09:37:05 +01:00
Sergei Morozov
3a03ff0186 DBZ-3535: Remove unused EventDispatcher constructor parameter 2022-03-24 09:37:05 +01:00
Vojtech Juranek
0163c07800 DBZ-4786 Avoid NPE when removing column which doesn't exists 2022-03-23 07:53:06 +01:00
Vojtech Juranek
fdfc7a6647 DBZ-3762 Filter out MySQL inline comments by default 2022-03-21 16:57:43 +01:00
kate
4b2d5c29be DBZ-4878 Re-read incremental snapshot chunk on DDL event 2022-03-18 07:55:43 +01:00
harveyyue
044e454e0a DBZ-4832 Don't set truststore/keystore parameters to system variables 2022-03-15 11:25:28 +01:00
Josh Ribera
020e845f44 DBZ-4809 Add task id and partition to the logging context for 
multi-partition connectors.

Co-authored-by: Sergei Morozov <morozov@tut.by>
 2022-03-09 20:46:38 +01:00
Gunnar Morling
d20a6f00e7 DBZ-4801 Using JdbcConfiguration in JdbcConnection 2022-03-07 17:53:45 +01:00
Gunnar Morling
c1c322ecf0 DBZ-4801 Simplifying JdbcConnection constructors 2022-03-07 17:53:45 +01:00