alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
7,683 Commits 3 Branches 0 Tags 51 MiB
1aac97b87c
Commit Graph

1544 Commits

Author SHA1 Message Date
Anisha Mohanty
91ccb7578f DBZ-5044 Centralize datatype schemas 2022-09-08 10:35:20 +02:00
Jiri Pechanec
77efd818da DBZ-5044 More descriptive names for schema history schemas 2022-09-08 10:35:20 +02:00
Jiri Pechanec
b8bcac1aef DBZ-5044 Ignore versions for schema comparison 2022-09-08 10:35:20 +02:00
Jiri Pechanec
fa2a7ba60c DBZ-5044 Fix transaction metadata schema 2022-09-08 10:35:20 +02:00
Anisha Mohanty
aea7c1ffa6 DBZ-5044 Use SchemaFactory for ConnectTable & EvenyDispatcher schemas 2022-09-08 10:35:20 +02:00
Anisha Mohanty
8fb2ffd70a DBZ-5044 Add review suggestions 2022-09-08 10:35:20 +02:00
Jiri Pechanec
cd07b8a06d DBZ-5044 Rename and reorder schema attributes 2022-09-08 10:35:20 +02:00
Anisha Mohanty
1a55869f5b DBZ-5044 Reword schema names 2022-09-08 10:35:20 +02:00
Anisha Mohanty
867ff0a117 DBZ-5044 Use seperate version for each schema 2022-09-08 10:35:20 +02:00
Anisha Mohanty
ec0ac3fea5 DBZ-5044 Centralize creation of Message Schemas 2022-09-08 10:35:20 +02:00
Vojtech Juranek
402df78381 DBZ-5043 Rename schema.history to schema.history.internal 2022-09-07 12:57:31 +02:00
Vojtech Juranek
b6c31f3da5 DBZ-5043 Replace 'database history' with 'database schema history' 
Reflect renaming database history to schema history in comments,
configuration descriptions and log and error messages.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
5bffd7839a DBZ-5043 Change default schema history suffix 
Change default schema history name from `$LOGICAL_NAME-dbhistory` to
`$LOGICAL_NAME-schemahistory`.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
0494a16a79 DBZ-5043 Rename DatabaseHistory to SchemaHistory 
Rename also all related classes and varibales.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
0823cedf25 DBZ-5043 Replace database.server.name with topic.prefix 2022-09-07 12:57:31 +02:00
Vojtech Juranek
a7a293c5c8 DBZ-5043 Move driver pass-through parameters into dedicated namespace 
Currently we mix database parameters like DB hostname, username or
password with other JDBC driver parameters which are passed to JDBC
driver as it is. Move these pass-through parameters into dedicated
namespace `driver.*` to separate them from common database parameters.
JDBC parameters which are defined on the connector level still stay
in `database` namespace.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
5d1b72e95f DBZ-5043 Simplify code after renaming database.history namespace 
Filtering `database.history` namespace from pass-through parameters is now
not needed as it doesn't interfere with other `database.*` parameters.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
25aa6c4acc DBZ-5043 Change namespace to database.history to schema.history 2022-09-07 12:57:31 +02:00
harveyyue
48d37857b6 DBZ-5588 Field validation errors are misleading for positive, non-zero expectations 2022-09-07 09:18:38 +02:00
harveyyue
58f4d3d912 DBZ-5480 Augment a uniqueness key filed/value in regex topic naming strategy 2022-09-06 14:58:57 +02:00
Claus Ibsen
1f48bfc515
[docs] Typo in description (remove space before dot) 2022-09-06 09:48:57 +02:00
Alexey Loubyansky
c4b95f4ea2 DBZ-5561 Use TCCL to load configuration classes by default 2022-09-06 08:42:45 +02:00
Vadzim Ramanenka
917de37a01 DBZ-5557: Supply partition when comiting offsets with source database 
Keep track of partitions offsets being committed belong to in
`BaseSourceTask`. Supply the partition along with the offset in calls to
`commitOffset`.
 2022-09-05 14:04:23 +02:00
Jiri Pechanec
6e53798407 DBZ-5538 Remove duplicated formatting message 2022-09-05 10:35:55 +02:00
harveyyue
f4c7b52451 DBZ-5538 Unclear validation error when required field is missing 2022-09-05 10:35:55 +02:00
ggaborg
664fd9e1b0 DBZ-3293 unit tests for filters 2022-09-01 20:16:47 +02:00
ggaborg
244de2bd96 DBZ-3293 Add signal table automatically to include list 2022-09-01 20:16:47 +02:00
thangdc94
57210095d2 DBZ-5439 Fix testcase 2022-09-01 09:32:17 +02:00
Jiri Pechanec
de53591dba DBZ-5244 Retry all communication exceptions by default 2022-08-31 09:45:09 -04:00
harveyyue
7ccd40129c DBZ-5569 Oracle connector's max.queue.size.in.bytes is invalid 2022-08-31 09:39:29 -04:00
Jiri Pechanec
1c291d75fd DBZ-5568 Use only Java 11 for tests 2022-08-31 09:19:23 +02:00
pkgonan
fd309193c6 DBZ-5544 Add document 2022-08-24 09:50:23 +02:00
pkgonan
b031770300 DBZ-5544 Support BASE64_URL_SAFE in BinaryHandlingMode 2022-08-24 09:50:23 +02:00
tjwornjs
ca009d8492 DBZ-5489 add column comment to schema 2022-08-24 09:46:01 +02:00
Anisha Mohanty
669024ed46 DBZ-5484 Better error reporting in SKIP/WARN cases 2022-08-23 10:02:29 -04:00
harveyyue
0308a1cbb6 DBZ-5535 Disable preferring DDL before logical schema in history recovery 2022-08-23 14:16:35 +02:00
Jiri Pechanec
34b78e6bce DBZ-2766 Make queue more generic 2022-08-23 13:43:41 +02:00
Jiri Pechanec
f770e0afb1 DBZ-2766 Remove Guava dependency 2022-08-23 13:43:41 +02:00
ggaborg
bf20a9c70f DBZ-5533 Unit test fails on Windows 2022-08-19 09:44:26 -04:00
Sergei Morozov
1c99437890 DBZ-5470: Remove the dependency of JdbcConnection on DatabaseSchema 2022-08-19 11:06:59 +02:00
Chris Cranford
37b97970a6 DBZ-5497 Skip truncate operations by default 
* Promote truncate skip default from PostgreSQL to core library
* Adjust behavior for Oracle with truncates skipped by default
 2022-08-19 11:05:29 +02:00
Vivek Wassan
d89e71ebeb DBZ-5327 Smart Backfills 2022-08-17 10:05:55 +02:00
harveyyue
a08420b21c DBZ-5511 Keep consistence with method serialize of Connect/Json Table Change Serializer 2022-08-16 16:58:07 +02:00
Vojtech Juranek
8e0ad245e5 DBZ-5475 Ensure Outbox JSON NUMBER arrays are of the same type 2022-08-05 11:00:46 -04:00
Vojtech Juranek
7256f62f6b DBZ-5475 Remove unneeded checks from Outbox payload conversion 
* remove float check as we return `float64` by default anyway
* simplify/narrow the checks when detemining array type, also add a test
  for array having the first element `null`
 2022-08-05 11:00:46 -04:00
Chris Cranford
d4261bd2fd DBZ-5476 Fix record accumulator during statistics log entry 2022-08-04 15:28:46 -04:00
harveyyue
2481172309 DBZ-5471 Missing the regex properties validation before start connector of DefaultRegexTopicNamingStrategy 2022-08-04 10:49:29 -04:00
Vojtech Juranek
c7d8353d7b DBZ-4727 Add snapshot paused JXM metrics 
Add two JXM objects:
* `snapshotPaused` - detemines if the incremental snapshot is paused
  or not
* `snapshotPausedDurationInSeconds` - overall time when the incremental
  snapshot was paused. The time adds up - if the snapshot was paused
  e.g. two times, the `snapshotPausedDurationInSeconds` is the sum of
  these two paused times.
 2022-07-29 12:41:58 -04:00
Vojtech Juranek
ed504173e1 DBZ-4727 Implement incremental snaphost pause/resume 
Intorduce two new signals:
* `pause-snapshot` - pauses running incremental snapshot
* `resume-snapshot` - resumes paused incremental snapshot

If the incremental snapshot is running and pause signal is sent,
currently processed chunk is finished and then no other chunk is read.
Snapshot state is kept in running state. Once snapshot is resume,
chunk is reverted to position which was last sent to the broker and
process of reading chunks and emitting `read` events continues until
all tables are read and sent.

This patch handles only table-based signals. MySQL Kafka-based signals
will be addressed later.
 2022-07-29 12:41:58 -04:00
harveyyue
5f430f1a8e DBZ-5457 Deprecate legacy topic selector for all connectors 2022-07-28 12:58:59 -04:00
ggaborg
2c8d71a36f DBZ-5452 Fix unit test failing on Windows platform 2022-07-26 15:50:28 -04:00
Debezium Builder
1ab51ae8d0 [maven-release-plugin] prepare for next development iteration 2022-07-26 18:13:10 +00:00
Debezium Builder
d247a48b79 [maven-release-plugin] prepare release v2.0.0.Beta1 2022-07-26 18:13:09 +00:00
harveyyue
7730610320 DBZ-5425 Remove the duplicated SimpleDdlParserListener from mysql connector 2022-07-20 14:35:34 +02:00
Chris Cranford
75ad893d67 DBZ-5396 Scope attributes inside table document 2022-07-19 15:41:02 -04:00
Chris Cranford
76dafedea7 DBZ-5396 Add multi-type accessors for attribute values 2022-07-19 15:41:02 -04:00
Chris Cranford
f412c2e6a4 DBZ-5396 Add support for connector-specific relational model attributes 2022-07-19 15:41:02 -04:00
harveyyue
daf329bc06 DBZ-5413 Update the topic naming strategy doc to all connectors 2022-07-18 10:18:12 -04:00
harveyyue
77c7db163d DBZ-5414 Validate topic naming strategy relative topic name properties 2022-07-18 09:58:51 -04:00
Chris Cranford
3acf317668 DBZ-5404 Restore build compatibility with Kafka 2.x and 3.0.x 2022-07-14 08:40:42 +02:00
harveyyue
6ea7abc8fc DBZ-4180 Pluggable topic selector 2022-07-13 14:45:25 +02:00
Jiri Pechanec
d8baac7c46 DBZ-5229 Reorganize code for inheritance use 2022-07-12 17:31:19 +02:00
Hossein Torabi
1ea02be9c2 DBZ-5229 define centralized and modular aproach for debezium storage 2022-07-12 17:31:19 +02:00
Chris Cranford
d2a76a7ffc DBZ-5295 Reselect LOB columns on primary key update 2022-07-12 11:59:34 +02:00
Chris Cranford
7b08fb8d3d DBZ-5356 Rewrite loop 2022-07-12 11:01:10 +02:00
Chris Cranford
deaa34d71e DBZ-5356 Avoid NPE during index evaluation 2022-07-12 11:01:10 +02:00
harveyyue
0842e5cd0e DBZ-5366 MySQL Connector column hash v2 does not work 2022-07-06 14:23:57 -04:00
Paweł Malon
707a43d3bc DBZ-5367 Fixed the problem where JSON expanding failed for Outbox event payloads that contained nested arrays where first array was empty and further array contained a Struct 
With proposed changes empty arrays are skipped, as it allows us to delay defining the schema for them until an array with elements is found, as it seems that there is no way to modify a schema that was already defined for a given field
 2022-07-06 13:47:54 -04:00
Debezium Builder
6fd1441af3 [maven-release-plugin] prepare for next development iteration 2022-07-01 09:21:49 +00:00
Debezium Builder
99402af634 [maven-release-plugin] prepare release v2.0.0.Alpha3 2022-07-01 09:21:48 +00:00
Vojtech Juranek
ac06623459 DBZ-5240 Move table types into separate method 
This will allow connectors to override the default values and return
types the connector supports.
 2022-06-30 10:46:54 +02:00
Gunnar Morling
cb53332561 DBZ-5335 Formatting 2022-06-30 09:54:13 +02:00
Mickael Maison
5ff71aa992 DBZ-5335 Fix formatting issues in the connectors configdefs 2022-06-30 09:35:54 +02:00
harveyyue
20dc353262 DBZ-5170 Mysql Commit Timestamp 2022-06-27 15:50:52 +02:00
Naveen Kumar KR
b75c9e635b DBZ-5277 ChangeEventQueue - Support Multiple Writer (doEnqueue) and Reader (poll) Threads 
* ReentrantLock based write and read coordination
* Condition based Inter Thread communication for isFull and isNotFull conditions
 2022-06-27 15:37:00 +02:00
Naveen Kumar KR
823a49fff4 DBZ-5277 ChangeEventQueue Test with Multiple Writer and Reader Threads 
Assertion would fail if one or both of the scenario occurs

1) Deadlock between Writer and Reader Threads
2) Thread Starvation between Writer and Reader Threads
 2022-06-27 15:37:00 +02:00
Jiri Pechanec
9ada2f28ce DBZ-4902 Change for exponential strategy 2022-06-27 14:29:51 +02:00
kanha gupta
1b17372d2d DBZ-4902 Replace long durations with Duration 2022-06-27 14:29:51 +02:00
Chris Cranford
ef99595d91 DBZ-5270 Reduce table schema capture logging verbosity 
When enabling the `database.history.store.only.captured.tables.ddl`,
the logging in RelationalSnapshotChangeEventSource does not take into
account this setting when logging, leading to a false impression of
what tables will actually be captured in the schema history.
 2022-06-27 08:47:34 +02:00
Jiri Pechanec
5e1ae24e92 DBZ-5272 Notify all threads in case of mutliple writers 2022-06-21 07:08:39 +02:00
Chris Cranford
3fb5bf1339 DBZ-4451 Introduce SchemaChangeEvent factory methods 2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081 DBZ-4451 Store previous table id in alter schema change events 
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
 2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5 DBZ-4834 Fix incremental snapshot of table added to include list 2022-06-16 08:39:16 +02:00
Vojtech Juranek
25eba9749d DBZ-5198 Add support for SQL server specific table delimiters 
Add suppoprt for `TableId` delimiters and provide implementation for
SQL server. SQL server allows to use reserved words in table names or
names with spaces, but they have to wrapped by `[]` characters, e.g.
`[dbname].[table name]`.

Debezium can handle spaces e.g. in table include list, but fails when
parsing snapshot select, therefore the parsing with predicates is used
only for parsing spanshot select for now. If useful, it can be used
later on on other places as well.
 2022-06-13 10:57:15 +02:00
Vojtech Juranek
796e17b2c6 DBZ-5198 Move predicates for parsing TableId into dedicate class 
Moving the predicates into parsing context decouples the predicates from
the private `TableIdTokenizer` class. Moving the predicates into
separate class would allow us to provide database specific
implementations.
 2022-06-13 10:57:15 +02:00
Debezium Builder
6fff3a2876 [maven-release-plugin] prepare for next development iteration 2022-06-09 11:49:26 +00:00
Debezium Builder
b2e2e351df [maven-release-plugin] prepare release v2.0.0.Alpha2 2022-06-09 11:49:26 +00:00
Vojtech Juranek
4a48eb33df DBZ-5045 Remove legacy V1 source info 2022-06-08 09:11:00 +02:00
Vojtech Juranek
250e160b81 DBZ-5045 Removed unused methods 2022-06-08 09:11:00 +02:00
Vojtech Juranek
ef8d7f7af7 DBZ-5045 Remove deprecated OBSOLETE_NAME_FOR_SKIP_FAILURE_HANDLING option value 2022-06-08 09:11:00 +02:00
Vojtech Juranek
ff5ddd7959 DBZ-5045 Remove deprecated FIELD_PAYLOAD_ID 
`table.field.event.payload.id` is replaced by `table.field.event.key`.
Also fix default value of `table.field.event.key` which was missing.
 2022-06-08 09:11:00 +02:00
Vojtech Juranek
0530dc2de9 DBZ-5045 Remove deprecated STORE_ONLY_MONITORED_TABLES_DDL 2022-06-08 09:11:00 +02:00
Vojtech Juranek
4e7b30b718 DBZ-5045 Remove deprecated DEBEZIUM_OPERATION_INVALID_BEHAVIOR 2022-06-08 09:11:00 +02:00
Vojtech Juranek
b0b3066ed5 DBZ-5045 Remove deprecated COLUMN_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
5031594b75 DBZ-5045 Remove deprecated COLUMN_WHITELIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
b3a6131cee DBZ-5045 Remove deprecated DATABASE_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
0cbb57f4bf DBZ-5045 Remove deprecated DATABASE_WHITELIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
f84893a8b4 DBZ-5045 Remove deprecated SCHEMA_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
261188482f DBZ-5045 Remove deprecated SCHEMA_WHITELIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
6abbcee6a3 DBZ-5045 Remove deprecated TABLE_BLACKLIST 2022-06-08 09:11:00 +02:00