Jiri Pechanec
1ec41fa120
DBZ-5879 Added logging; removed commented out code
2023-01-18 14:46:14 +01:00
Jeremy Ford
553cb9b1dd
DBZ-5879 Add wait to tests in order to maintain expectations of existing integration tests
2023-01-18 14:46:14 +01:00
Jeremy Ford
f2a6f0b8db
DBZ-5879 Support retrying database connection failures during connector start
...
Reworked when task requests start from subclasses
Add support for restarting
2023-01-18 14:46:14 +01:00
Jiri Pechanec
e8729bb0eb
DBZ-5996 Properly handle zoned timestamp width in Oracle
2023-01-18 10:42:36 +01:00
Jiri Pechanec
bcd6787cdc
DBZ-6008 Revert "DBZ-3653 Introduce incremental.snapshot.isolation.mode config"
...
This reverts commit 6d8bcd42fd
.
2023-01-17 10:40:20 +01:00
Jiri Pechanec
8738e62459
DBZ-6008 Revert "DBZ-3653 Remove extra modifiers"
...
This reverts commit bdfc1e6861
.
2023-01-17 10:40:20 +01:00
Jiri Pechanec
0c5e4f0051
DBZ-6002 Fix tests now requiring fractions
2023-01-13 12:00:00 +01:00
harveyyue
bea65acc0e
DBZ-6002 Run PostgresConnectorIT.shouldReceiveChangesForChangeColumnDefault() failed
2023-01-13 12:00:00 +01:00
akshansh
444dab3ffa
DBZ-5996 Handled case where fractionalWidth was -1
2023-01-11 12:09:26 +01:00
akshansh
b87d80f462
DBZ-5996 Implemented custom DateTimeFormatter to parse ZonedDateTime while preserving length of column.
2023-01-11 12:09:26 +01:00
harveyyue
5e56ad75f8
DBZ-5985 Disable to print invalid estimated row count when config "snapshot.select.statement.overrides"
2023-01-11 09:20:15 +01:00
Jiri Pechanec
bdfc1e6861
DBZ-3653 Remove extra modifiers
2023-01-11 08:55:33 +01:00
govi20
6d8bcd42fd
DBZ-3653 Introduce incremental.snapshot.isolation.mode config
2023-01-11 08:55:33 +01:00
Jiri Pechanec
8ca5e0a9b5
DBZ-4669 Reformat after plugin version consolidation
2023-01-11 06:44:53 +01:00
Mark Lambert
f0a1fa8a67
DBZ-4669 Use the newer version of plugins to consolidate remaining duplicates plugin versions
2023-01-11 06:44:53 +01:00
Mark Lambert
e5d16b14a6
DBZ-2439 Remove redundant modifiers
2023-01-11 06:44:53 +01:00
harveyyue
d941bc928b
DBZ-5966 Truncate records incompatible with ExtractNewRecordState
2023-01-05 13:51:35 +01:00
Jiri Pechanec
26bcb67cdc
DBZ-5967 Provide additional checks
2023-01-05 08:12:21 +01:00
mfvitale
c2d7d82d30
DBZ-5967: Fix negative partition number in ComputePartition SMT
...
Also improved check on configuration consistency to avoid 0 as partition number.
2023-01-05 08:12:21 +01:00
Debezium Builder
cf1720e4cb
[maven-release-plugin] prepare for next development iteration
2022-12-22 10:11:54 +00:00
Debezium Builder
b4ea9b46cd
[maven-release-plugin] prepare release v2.1.0.Final
2022-12-22 10:11:53 +00:00
harveyyue
6d70bf7b9d
DBZ-5344 Add complex json array test case to EventRouterTest
2022-12-19 12:24:24 +01:00
harveyyue
348e77d334
DBZ-5946 Print the readable data class name in JdbcValueConverters.handleUnknownData
2022-12-19 11:02:38 +01:00
Debezium Builder
177a60a9d3
[maven-release-plugin] prepare for next development iteration
2022-12-16 09:33:38 +00:00
Debezium Builder
f805db8c3c
[maven-release-plugin] prepare release v2.1.0.Beta1
2022-12-16 09:33:38 +00:00
Vojtech Juranek
1c0300cf2a
DBZ-5838 Disable schema caching
...
While documentation says that schema caching is disabled by default,
which would everyone sane assume anyway, it's actually turned of and
clean result in very strange test failures.
See also https://github.com/Apicurio/apicurio-registry/issues/2997
2022-12-12 14:44:05 +01:00
mfvitale
4c941b63ed
DBZ-5847: Add support for all connectors
...
Support for qualified table names
2022-12-12 14:40:56 +01:00
Marcelo Avancini
7754163c5d
DBZ-2432: Fixing misuse of assertj
...
- Some usages of assertj was not callig the actual assert of the DSL, fixing it
- Changing the Assertions to be staticlly imported
2022-12-08 13:10:26 +01:00
Jiri Pechanec
97ea30f25d
DBZ-5893 Mask connection string in connector option logging
2022-12-07 08:41:34 +01:00
jcechace
2468d73783
DBZ-5893 Masking credentials in connection string when logging
2022-12-07 05:45:58 +01:00
Debezium Builder
81896ed334
[maven-release-plugin] prepare for next development iteration
2022-11-30 07:52:24 +00:00
Debezium Builder
1aac97b87c
[maven-release-plugin] prepare release v2.1.0.Alpha2
2022-11-30 07:52:23 +00:00
Jiri Pechanec
f50ed10377
DBZ-5796 Fix typo in test description
2022-11-16 14:55:01 +01:00
harveyyue
b124d91a93
DBZ-5796 For outbox transformation, when 'table.expand.json.payload' is set to true null values are not correctly deserialized
2022-11-16 14:55:01 +01:00
Debezium Builder
e311dd1a67
[maven-release-plugin] prepare for next development iteration
2022-11-10 13:29:06 +00:00
Debezium Builder
e3bf53440a
[maven-release-plugin] prepare release v2.1.0.Alpha1
2022-11-10 13:29:05 +00:00
Anil Dasari
c781832f41
DBZ-5822 Change visibility of BaseSourceTask#logStatistics method to protected
2022-11-10 08:02:48 +01:00
Chris Cranford
1d9a21bb85
DBZ-5759 Log per topic record counts
2022-11-10 07:31:28 +01:00
Vojtech Juranek
96e4326d35
DBZ-2131 Use system property to determine if Apicurio should be used
2022-11-09 08:44:45 +01:00
Vojtech Juranek
41d3d7d155
DBZ-2131 Run tests with Apicurio registry
...
When Apicurio profile is used and container with Apicurio started, use
Apicurio registry in the tests. Fall back to in-memory Confluent
registry otherwise.
2022-11-09 08:44:45 +01:00
harveyyue
e6f0551451
DBZ-5801 The merge method of configuration is not work
2022-11-08 14:19:51 +01:00
jcechace
f46c127d11
DBZ-5779 Migrated debezium-core to AssertJ
2022-11-02 08:50:26 +01:00
Chris Cranford
dc5de5ae87
DBZ-5760 Only log full row data at TRACE level
2022-10-26 08:13:34 +02:00
Chris Cranford
c9084dd3f7
DBZ-5615 Mask apicurio registry credentials
2022-10-25 05:48:54 +02:00
Chris Cranford
a0299ec594
DBZ-5615 Mask schema registry credentials
2022-10-25 05:48:54 +02:00
Jiri Pechanec
ad2733eb1a
DBZ-5727 Simplify checking of column filter presence
2022-10-20 12:47:47 +02:00
harveyyue
0e9bd53629
DBZ-5727 Columns are not excluded when doing incremental snapshots
2022-10-20 12:47:47 +02:00
Jiri Pechanec
8a02907576
DBZ-5610 Truncate events are not emitted by every connector
2022-10-20 08:57:28 +02:00
harveyyue
a37d3332df
DBZ-5610 MySQL Connector capture TRUNCATE command as message in table topic
2022-10-20 08:57:28 +02:00
Debezium Builder
4d35fb8011
[maven-release-plugin] prepare for next development iteration
2022-10-14 11:26:32 +00:00
Debezium Builder
7bd1551c99
[maven-release-plugin] prepare release v2.0.0.Final
2022-10-14 11:26:32 +00:00
harveyyue
299f93afd7
DBZ-5709 The DDL_FILTER of SchemaHistory doesn't work for including break lines ddl statement
2022-10-14 09:21:40 +02:00
Vojtech Juranek
820b9d10d5
DBZ-5329 Add LSN to Postgres transation ID
...
Add `EventDispatcher` constructor which accepts `TransactionMonior`
instance as a parameter and in case of Postgres pass into
`EventDispatcher` custom `PostgresTransactionMonitor` which adjusts
transactions IDs by adding LSN, i.e. Postgres transaction is now of
form `txId:LSN`.
2022-10-11 15:38:33 +02:00
Vojtech Juranek
4554a63afe
DBZ-5329 Allow override transaction message key/value
...
Allow subclasses override transaction metadata keys and values.
2022-10-11 15:38:33 +02:00
Debezium Builder
884254aba1
[maven-release-plugin] prepare for next development iteration
2022-10-07 09:27:52 +00:00
Debezium Builder
a0f3263fa6
[maven-release-plugin] prepare release v2.0.0.CR1
2022-10-07 09:27:51 +00:00
Jiri Pechanec
99dff302f1
DBZ-5612 Support for formatted messages
2022-10-04 13:45:43 +02:00
rajdangwal
58ab632ea3
DBZ-5612 log records only at trace level
2022-10-04 13:45:43 +02:00
rajdangwal
c03bcf8606
DBZ-5612 Remove records from being logged at all levels
2022-10-04 13:45:43 +02:00
harveyyue
f6b72f060c
DBZ-5671 Remove logic name parameter from sub connector config
2022-10-03 11:40:17 -04:00
Jiri Pechanec
7d2cf85279
DBZ-5541 Change default in the description
2022-09-29 09:47:25 +02:00
Sergei Morozov
e4d46c81b1
DBZ-5541: Default Debezium schema.name.adjustment.mode to "none"
2022-09-29 09:47:25 +02:00
harveyyue
14b08c1748
DBZ-5654 Outbox pattern nested payload leads to connector crash
2022-09-28 10:20:22 -04:00
harveyyue
d1153e8ad4
DBZ-5594 Clean up "logical name" config
2022-09-22 14:29:37 +02:00
Jiri Pechanec
70d5b2db50
DBZ-5639 Inform about snapshot reexcution
2022-09-22 13:14:28 +02:00
harveyyue
3b4d19c174
DBZ-5639 Logging enhancement for non-incremental snapshot in postgres connector
2022-09-22 13:14:28 +02:00
Ismail Simsek
b966f4b65b
DBZ-5591 Snapshot process setting source.ts_ms
2022-09-20 17:12:54 -04:00
Jiri Pechanec
29fcb0e047
DBZ-5585 Add last resort classloader
2022-09-19 17:15:29 +02:00
harveyyue
288c93434f
DBZ-5585 Remove classloader parameter from JdbcConnection
2022-09-19 17:15:29 +02:00
harveyyue
20d56734d4
DBZ-5585 Modify the Instantiator to not require classloader
2022-09-19 17:15:29 +02:00
Vojtech Juranek
2a0e2a7e81
DBZ-5603 Move test containers to quay.io
2022-09-19 09:12:46 +02:00
Debezium Builder
218d414552
[maven-release-plugin] prepare for next development iteration
2022-09-16 06:36:40 +00:00
Debezium Builder
567f57c47b
[maven-release-plugin] prepare release v2.0.0.Beta2
2022-09-16 06:36:39 +00:00
harveyyue
09b99840d8
DBZ-4617 Deprecate internal key/value converter options
2022-09-14 12:49:30 +02:00
Vojtech Juranek
8b8d4828f3
DBZ-5043 Make topic prefix field manadatory and with proper group
2022-09-14 12:44:38 +02:00
harveyyue
665edab844
DBZ-5589 Mysql connector can't handle the case sensitive of rename/change column statement
2022-09-13 09:36:25 +02:00
Vojtech Juranek
acc6be6df3
DBZ-5043 Move TOPIC_PREFIX into CommonConnectorConfig
...
`TOPIC_PREFIX` is now mandatory to all connectors therefore it make
sense to have it in common config. Beside that, it also makes it more
easy to use it in Debezium UI without any workarounds - if the field is
not member of the given connector config, the field has to be
explicitely added into known fields otherwise is invisible for UI.
2022-09-13 09:25:54 +02:00
Anisha Mohanty
f41b6dd458
DBZ-5044 Get rid of schema factory instances
2022-09-08 10:35:20 +02:00
Anisha Mohanty
b0dc7e557a
DBZ-5044 Update schemaHistoryColumnSchema to add new fields
2022-09-08 10:35:20 +02:00
Anisha Mohanty
ec792527e9
DBZ-5044 Use SchemaFactory for centralizing Postgres schemas
2022-09-08 10:35:20 +02:00
Anisha Mohanty
cfd0242e28
DBZ-5044 Sort imports & fix test failures
2022-09-08 10:35:20 +02:00
Anisha Mohanty
91ccb7578f
DBZ-5044 Centralize datatype schemas
2022-09-08 10:35:20 +02:00
Jiri Pechanec
77efd818da
DBZ-5044 More descriptive names for schema history schemas
2022-09-08 10:35:20 +02:00
Jiri Pechanec
b8bcac1aef
DBZ-5044 Ignore versions for schema comparison
2022-09-08 10:35:20 +02:00
Jiri Pechanec
fa2a7ba60c
DBZ-5044 Fix transaction metadata schema
2022-09-08 10:35:20 +02:00
Anisha Mohanty
aea7c1ffa6
DBZ-5044 Use SchemaFactory for ConnectTable & EvenyDispatcher schemas
2022-09-08 10:35:20 +02:00
Anisha Mohanty
8fb2ffd70a
DBZ-5044 Add review suggestions
2022-09-08 10:35:20 +02:00
Jiri Pechanec
cd07b8a06d
DBZ-5044 Rename and reorder schema attributes
2022-09-08 10:35:20 +02:00
Anisha Mohanty
1a55869f5b
DBZ-5044 Reword schema names
2022-09-08 10:35:20 +02:00
Anisha Mohanty
867ff0a117
DBZ-5044 Use seperate version for each schema
2022-09-08 10:35:20 +02:00
Anisha Mohanty
ec0ac3fea5
DBZ-5044 Centralize creation of Message Schemas
2022-09-08 10:35:20 +02:00
Vojtech Juranek
402df78381
DBZ-5043 Rename schema.history to schema.history.internal
2022-09-07 12:57:31 +02:00
Vojtech Juranek
b6c31f3da5
DBZ-5043 Replace 'database history' with 'database schema history'
...
Reflect renaming database history to schema history in comments,
configuration descriptions and log and error messages.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
5bffd7839a
DBZ-5043 Change default schema history suffix
...
Change default schema history name from `$LOGICAL_NAME-dbhistory` to
`$LOGICAL_NAME-schemahistory`.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0494a16a79
DBZ-5043 Rename DatabaseHistory to SchemaHistory
...
Rename also all related classes and varibales.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0823cedf25
DBZ-5043 Replace database.server.name with topic.prefix
2022-09-07 12:57:31 +02:00
Vojtech Juranek
a7a293c5c8
DBZ-5043 Move driver pass-through parameters into dedicated namespace
...
Currently we mix database parameters like DB hostname, username or
password with other JDBC driver parameters which are passed to JDBC
driver as it is. Move these pass-through parameters into dedicated
namespace `driver.*` to separate them from common database parameters.
JDBC parameters which are defined on the connector level still stay
in `database` namespace.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
5d1b72e95f
DBZ-5043 Simplify code after renaming database.history namespace
...
Filtering `database.history` namespace from pass-through parameters is now
not needed as it doesn't interfere with other `database.*` parameters.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
25aa6c4acc
DBZ-5043 Change namespace to database.history
to schema.history
2022-09-07 12:57:31 +02:00
harveyyue
48d37857b6
DBZ-5588 Field validation errors are misleading for positive, non-zero expectations
2022-09-07 09:18:38 +02:00
harveyyue
58f4d3d912
DBZ-5480 Augment a uniqueness key filed/value in regex topic naming strategy
2022-09-06 14:58:57 +02:00
Claus Ibsen
1f48bfc515
[docs] Typo in description (remove space before dot)
2022-09-06 09:48:57 +02:00
Alexey Loubyansky
c4b95f4ea2
DBZ-5561 Use TCCL to load configuration classes by default
2022-09-06 08:42:45 +02:00
Vadzim Ramanenka
917de37a01
DBZ-5557: Supply partition when comiting offsets with source database
...
Keep track of partitions offsets being committed belong to in
`BaseSourceTask`. Supply the partition along with the offset in calls to
`commitOffset`.
2022-09-05 14:04:23 +02:00
Jiri Pechanec
6e53798407
DBZ-5538 Remove duplicated formatting message
2022-09-05 10:35:55 +02:00
harveyyue
f4c7b52451
DBZ-5538 Unclear validation error when required field is missing
2022-09-05 10:35:55 +02:00
ggaborg
664fd9e1b0
DBZ-3293 unit tests for filters
2022-09-01 20:16:47 +02:00
ggaborg
244de2bd96
DBZ-3293 Add signal table automatically to include list
2022-09-01 20:16:47 +02:00
thangdc94
57210095d2
DBZ-5439 Fix testcase
2022-09-01 09:32:17 +02:00
Jiri Pechanec
de53591dba
DBZ-5244 Retry all communication exceptions by default
2022-08-31 09:45:09 -04:00
harveyyue
7ccd40129c
DBZ-5569 Oracle connector's max.queue.size.in.bytes is invalid
2022-08-31 09:39:29 -04:00
Jiri Pechanec
1c291d75fd
DBZ-5568 Use only Java 11 for tests
2022-08-31 09:19:23 +02:00
pkgonan
fd309193c6
DBZ-5544 Add document
2022-08-24 09:50:23 +02:00
pkgonan
b031770300
DBZ-5544 Support BASE64_URL_SAFE in BinaryHandlingMode
2022-08-24 09:50:23 +02:00
tjwornjs
ca009d8492
DBZ-5489 add column comment to schema
2022-08-24 09:46:01 +02:00
Anisha Mohanty
669024ed46
DBZ-5484 Better error reporting in SKIP/WARN cases
2022-08-23 10:02:29 -04:00
harveyyue
0308a1cbb6
DBZ-5535 Disable preferring DDL before logical schema in history recovery
2022-08-23 14:16:35 +02:00
Jiri Pechanec
34b78e6bce
DBZ-2766 Make queue more generic
2022-08-23 13:43:41 +02:00
Jiri Pechanec
f770e0afb1
DBZ-2766 Remove Guava dependency
2022-08-23 13:43:41 +02:00
ggaborg
bf20a9c70f
DBZ-5533 Unit test fails on Windows
2022-08-19 09:44:26 -04:00
Sergei Morozov
1c99437890
DBZ-5470: Remove the dependency of JdbcConnection on DatabaseSchema
2022-08-19 11:06:59 +02:00
Chris Cranford
37b97970a6
DBZ-5497 Skip truncate operations by default
...
* Promote truncate skip default from PostgreSQL to core library
* Adjust behavior for Oracle with truncates skipped by default
2022-08-19 11:05:29 +02:00
Vivek Wassan
d89e71ebeb
DBZ-5327 Smart Backfills
2022-08-17 10:05:55 +02:00
harveyyue
a08420b21c
DBZ-5511 Keep consistence with method serialize of Connect/Json Table Change Serializer
2022-08-16 16:58:07 +02:00
Vojtech Juranek
8e0ad245e5
DBZ-5475 Ensure Outbox JSON NUMBER arrays are of the same type
2022-08-05 11:00:46 -04:00
Vojtech Juranek
7256f62f6b
DBZ-5475 Remove unneeded checks from Outbox payload conversion
...
* remove float check as we return `float64` by default anyway
* simplify/narrow the checks when detemining array type, also add a test
for array having the first element `null`
2022-08-05 11:00:46 -04:00
Chris Cranford
d4261bd2fd
DBZ-5476 Fix record accumulator during statistics log entry
2022-08-04 15:28:46 -04:00
harveyyue
2481172309
DBZ-5471 Missing the regex properties validation before start connector of DefaultRegexTopicNamingStrategy
2022-08-04 10:49:29 -04:00
Vojtech Juranek
c7d8353d7b
DBZ-4727 Add snapshot paused JXM metrics
...
Add two JXM objects:
* `snapshotPaused` - detemines if the incremental snapshot is paused
or not
* `snapshotPausedDurationInSeconds` - overall time when the incremental
snapshot was paused. The time adds up - if the snapshot was paused
e.g. two times, the `snapshotPausedDurationInSeconds` is the sum of
these two paused times.
2022-07-29 12:41:58 -04:00
Vojtech Juranek
ed504173e1
DBZ-4727 Implement incremental snaphost pause/resume
...
Intorduce two new signals:
* `pause-snapshot` - pauses running incremental snapshot
* `resume-snapshot` - resumes paused incremental snapshot
If the incremental snapshot is running and pause signal is sent,
currently processed chunk is finished and then no other chunk is read.
Snapshot state is kept in running state. Once snapshot is resume,
chunk is reverted to position which was last sent to the broker and
process of reading chunks and emitting `read` events continues until
all tables are read and sent.
This patch handles only table-based signals. MySQL Kafka-based signals
will be addressed later.
2022-07-29 12:41:58 -04:00
harveyyue
5f430f1a8e
DBZ-5457 Deprecate legacy topic selector for all connectors
2022-07-28 12:58:59 -04:00
ggaborg
2c8d71a36f
DBZ-5452 Fix unit test failing on Windows platform
2022-07-26 15:50:28 -04:00
Debezium Builder
1ab51ae8d0
[maven-release-plugin] prepare for next development iteration
2022-07-26 18:13:10 +00:00
Debezium Builder
d247a48b79
[maven-release-plugin] prepare release v2.0.0.Beta1
2022-07-26 18:13:09 +00:00
harveyyue
7730610320
DBZ-5425 Remove the duplicated SimpleDdlParserListener from mysql connector
2022-07-20 14:35:34 +02:00
Chris Cranford
75ad893d67
DBZ-5396 Scope attributes inside table document
2022-07-19 15:41:02 -04:00
Chris Cranford
76dafedea7
DBZ-5396 Add multi-type accessors for attribute values
2022-07-19 15:41:02 -04:00
Chris Cranford
f412c2e6a4
DBZ-5396 Add support for connector-specific relational model attributes
2022-07-19 15:41:02 -04:00
harveyyue
daf329bc06
DBZ-5413 Update the topic naming strategy doc to all connectors
2022-07-18 10:18:12 -04:00
harveyyue
77c7db163d
DBZ-5414 Validate topic naming strategy relative topic name properties
2022-07-18 09:58:51 -04:00
Chris Cranford
3acf317668
DBZ-5404 Restore build compatibility with Kafka 2.x and 3.0.x
2022-07-14 08:40:42 +02:00
harveyyue
6ea7abc8fc
DBZ-4180 Pluggable topic selector
2022-07-13 14:45:25 +02:00
Jiri Pechanec
d8baac7c46
DBZ-5229 Reorganize code for inheritance use
2022-07-12 17:31:19 +02:00
Hossein Torabi
1ea02be9c2
DBZ-5229 define centralized and modular aproach for debezium storage
2022-07-12 17:31:19 +02:00
Chris Cranford
d2a76a7ffc
DBZ-5295 Reselect LOB columns on primary key update
2022-07-12 11:59:34 +02:00
Chris Cranford
7b08fb8d3d
DBZ-5356 Rewrite loop
2022-07-12 11:01:10 +02:00
Chris Cranford
deaa34d71e
DBZ-5356 Avoid NPE during index evaluation
2022-07-12 11:01:10 +02:00
harveyyue
0842e5cd0e
DBZ-5366 MySQL Connector column hash v2 does not work
2022-07-06 14:23:57 -04:00
Paweł Malon
707a43d3bc
DBZ-5367 Fixed the problem where JSON expanding failed for Outbox event payloads that contained nested arrays where first array was empty and further array contained a Struct
...
With proposed changes empty arrays are skipped, as it allows us to delay defining the schema for them until an array with elements is found, as it seems that there is no way to modify a schema that was already defined for a given field
2022-07-06 13:47:54 -04:00
Debezium Builder
6fd1441af3
[maven-release-plugin] prepare for next development iteration
2022-07-01 09:21:49 +00:00
Debezium Builder
99402af634
[maven-release-plugin] prepare release v2.0.0.Alpha3
2022-07-01 09:21:48 +00:00
Vojtech Juranek
ac06623459
DBZ-5240 Move table types into separate method
...
This will allow connectors to override the default values and return
types the connector supports.
2022-06-30 10:46:54 +02:00
Gunnar Morling
cb53332561
DBZ-5335 Formatting
2022-06-30 09:54:13 +02:00
Mickael Maison
5ff71aa992
DBZ-5335 Fix formatting issues in the connectors configdefs
2022-06-30 09:35:54 +02:00
harveyyue
20dc353262
DBZ-5170 Mysql Commit Timestamp
2022-06-27 15:50:52 +02:00
Naveen Kumar KR
b75c9e635b
DBZ-5277 ChangeEventQueue - Support Multiple Writer (doEnqueue) and Reader (poll) Threads
...
* ReentrantLock based write and read coordination
* Condition based Inter Thread communication for isFull and isNotFull conditions
2022-06-27 15:37:00 +02:00
Naveen Kumar KR
823a49fff4
DBZ-5277 ChangeEventQueue Test with Multiple Writer and Reader Threads
...
Assertion would fail if one or both of the scenario occurs
1) Deadlock between Writer and Reader Threads
2) Thread Starvation between Writer and Reader Threads
2022-06-27 15:37:00 +02:00
Jiri Pechanec
9ada2f28ce
DBZ-4902 Change for exponential strategy
2022-06-27 14:29:51 +02:00
kanha gupta
1b17372d2d
DBZ-4902 Replace long durations with Duration
2022-06-27 14:29:51 +02:00
Chris Cranford
ef99595d91
DBZ-5270 Reduce table schema capture logging verbosity
...
When enabling the `database.history.store.only.captured.tables.ddl`,
the logging in RelationalSnapshotChangeEventSource does not take into
account this setting when logging, leading to a false impression of
what tables will actually be captured in the schema history.
2022-06-27 08:47:34 +02:00
Jiri Pechanec
5e1ae24e92
DBZ-5272 Notify all threads in case of mutliple writers
2022-06-21 07:08:39 +02:00
Chris Cranford
3fb5bf1339
DBZ-4451 Introduce SchemaChangeEvent factory methods
2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081
DBZ-4451 Store previous table id in alter schema change events
...
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5
DBZ-4834 Fix incremental snapshot of table added to include list
2022-06-16 08:39:16 +02:00
Vojtech Juranek
25eba9749d
DBZ-5198 Add support for SQL server specific table delimiters
...
Add suppoprt for `TableId` delimiters and provide implementation for
SQL server. SQL server allows to use reserved words in table names or
names with spaces, but they have to wrapped by `[]` characters, e.g.
`[dbname].[table name]`.
Debezium can handle spaces e.g. in table include list, but fails when
parsing snapshot select, therefore the parsing with predicates is used
only for parsing spanshot select for now. If useful, it can be used
later on on other places as well.
2022-06-13 10:57:15 +02:00
Vojtech Juranek
796e17b2c6
DBZ-5198 Move predicates for parsing TableId into dedicate class
...
Moving the predicates into parsing context decouples the predicates from
the private `TableIdTokenizer` class. Moving the predicates into
separate class would allow us to provide database specific
implementations.
2022-06-13 10:57:15 +02:00
Debezium Builder
6fff3a2876
[maven-release-plugin] prepare for next development iteration
2022-06-09 11:49:26 +00:00
Debezium Builder
b2e2e351df
[maven-release-plugin] prepare release v2.0.0.Alpha2
2022-06-09 11:49:26 +00:00
Vojtech Juranek
4a48eb33df
DBZ-5045 Remove legacy V1 source info
2022-06-08 09:11:00 +02:00
Vojtech Juranek
250e160b81
DBZ-5045 Removed unused methods
2022-06-08 09:11:00 +02:00
Vojtech Juranek
ef8d7f7af7
DBZ-5045 Remove deprecated OBSOLETE_NAME_FOR_SKIP_FAILURE_HANDLING option value
2022-06-08 09:11:00 +02:00
Vojtech Juranek
ff5ddd7959
DBZ-5045 Remove deprecated FIELD_PAYLOAD_ID
...
`table.field.event.payload.id` is replaced by `table.field.event.key`.
Also fix default value of `table.field.event.key` which was missing.
2022-06-08 09:11:00 +02:00
Vojtech Juranek
0530dc2de9
DBZ-5045 Remove deprecated STORE_ONLY_MONITORED_TABLES_DDL
2022-06-08 09:11:00 +02:00
Vojtech Juranek
4e7b30b718
DBZ-5045 Remove deprecated DEBEZIUM_OPERATION_INVALID_BEHAVIOR
2022-06-08 09:11:00 +02:00
Vojtech Juranek
b0b3066ed5
DBZ-5045 Remove deprecated COLUMN_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
5031594b75
DBZ-5045 Remove deprecated COLUMN_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
b3a6131cee
DBZ-5045 Remove deprecated DATABASE_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
0cbb57f4bf
DBZ-5045 Remove deprecated DATABASE_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
f84893a8b4
DBZ-5045 Remove deprecated SCHEMA_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
261188482f
DBZ-5045 Remove deprecated SCHEMA_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
6abbcee6a3
DBZ-5045 Remove deprecated TABLE_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
db08e894fc
DBZ-5045 Remove deprecated TABLE_WHITELIST
2022-06-08 09:11:00 +02:00
harveyyue
acb743bd34
DBZ-5206 Remove the deprecated monitored tables metrics
2022-06-07 16:13:29 -04:00
Chris Cranford
3e820c4047
DBZ-4251 Adjust semantics with collection removal
2022-06-02 07:30:07 +02:00
Chris Cranford
187e3250a3
DBZ-4251 Add regex support for stop signal
2022-06-02 07:30:07 +02:00
Chris Cranford
9d71296b5c
DBZ-4251 Support stopping incremental snapshots
2022-06-02 07:30:07 +02:00
Jiri Pechanec
88b7ae2050
DBZ-5136 Move sourceInfo to common class
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
c64b8af31a
DBZ-5136 reintroduce the CommonOffsetContext class and refactor common functionality into it
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
2d72dae649
DBZ-5136 further refining of the algorithm for determining position in snapshot and IT test fixes
2022-06-01 16:56:29 +02:00
Jiri Pechanec
08e8c3d795
DBZ-5136 Move common class to common package
2022-06-01 16:56:29 +02:00
Andrew Walker
d0759461a4
DBZ-5136: Convert RelationalOffsetContext to CommonOffsetContext
2022-06-01 16:56:29 +02:00
Andrew Walker
01a81f8bc1
DBZ-5136: Refactored SnapshotContext setters
2022-06-01 16:56:29 +02:00
Andrew Walker
b6222d42bf
DBZ-5136: Add first flag and refactor OffsetContext
2022-06-01 16:56:29 +02:00
Andrew Walker
593c2769e6
DBZ-5136: Add last in table flag
2022-06-01 16:56:29 +02:00
Chris Cranford
15e9231c30
DBZ-5179 Use LRUCacheMap
to avoid high memory usage
2022-06-01 15:53:55 +02:00
Chris Cranford
9fa5a48274
DBZ-5011 Register MongoDB metrics with task id when more than one task
2022-06-01 09:41:36 +02:00
harveyyue
9a3afac243
DBZ-5180 Put the MysqlPartition values to MDC getting NPE in logging context phase
2022-06-01 09:17:04 +02:00
Vojtech Juranek
5550057034
DBZ-5042 Add abstract partition with DB name
...
DB names is used by logging facilities. As the DB name is required only
for relational DBs, also this abstract class is provided only for
relational DB connectors.
Use this abstract class for SQL server connector, where DB name logging
is already implemented.
2022-05-27 14:20:30 -04:00
Jiri Pechanec
b85cb275b1
DBZ-5089 Use atomic reference
2022-05-27 12:46:39 +02:00
connorszczepaniak-wk
8693b4f008
DBZ-5089: add partition to the outbox event router in additional fields
...
DBZ-5089: undo format change
DBZ-5089: add tests for the partition routing
DBZ-5089: remove nonsensical comment
DBZ-5089: partition fieldname is a string; test properly
DBZ-5089: fix one of the failing tests
DBZ-5089: pass args through
DBZ-5089: fix oracle implementation; add configs for mongo connector
DBZ-5089: put partition in additional placement
DBZ-5089: restore formatting
2022-05-27 12:46:39 +02:00
harveyyue
e45e8bd43b
DBZ-5172 Expose more useful exception info with building the field default value schema
2022-05-27 12:41:12 +02:00
Rotem Adhoh
8ef7224543
DBZ-5166 Fixed ExtractNewRecordState SMT to pass all columns' data as is, i.e: without potential unwanted values manipulation.
...
Prior to this fix, when inserting an explicit NULL value to a nullable column which has a default value, resulted in a record which contains the corresponding field with the default value (instead of NULL) after applying ExtractNewRecordState transformation.
A new unit test was added as well.
2022-05-27 12:33:23 +02:00
harveyyue
3ada04b522
DBZ-3865 Debezium MySql connector does not capture floating point numbers with the right precision
2022-05-25 12:54:42 -04:00
Chris Cranford
7d8c9486d6
DBZ-5119 Move Heartbeat creation to ConnectorConfig & descendants
2022-05-25 09:59:59 +02:00
Chris Cranford
7ff7826743
DBZ-5119 Pull heartbeat.action.query
support into debezium-core
2022-05-24 12:54:40 +02:00
Chris Lee
2a8456d21b
DBZ-5123 data-collection regex snapshot signal
2022-05-24 12:46:28 +02:00
harveyyue
52ba58ccc3
DBZ-5098 Introduce a new field "ts_ms" to identify the process time for schema change event
2022-05-18 14:21:39 +02:00
harveyyue
f66b07cd80
DBZ-5141 Should be sleep with REGISTRATION_RETRY_DELAY when registry MBean failed
2022-05-18 12:19:31 +02:00
Jiri Pechanec
8643ad3a22
DBZ-5138 Unify happy/unhappy common path
2022-05-18 07:42:33 +02:00
Chris Cranford
e075d6b2eb
DBZ-5138 Fix tracking MBean registration
2022-05-17 20:02:11 -04:00
Jiri Pechanec
d56f6ef6e0
DBZ-5138 Retry failed metrics registration; allow starting without metrics
2022-05-17 20:02:11 -04:00
Sergei Morozov
1ad97b9d66
DBZ-5130: Log history recover progress every two seconds
2022-05-16 09:17:24 +02:00
Sergei Morozov
65e97796e7
DBZ-5129: Use buffered reader when reading history from file
2022-05-13 09:38:43 +02:00
Jiri Pechanec
4396fc3ec3
DBZ-4600 Remove deprecated class
2022-05-10 15:25:35 +02:00
Jiri Pechanec
5e94c1d498
DBZ-4600 Fix formatting
2022-05-10 15:08:46 +02:00
Nathan Bradshaw
c76e697f2f
DBZ-4600 - Add BinaryDataConverter to support byte[] and ByteBuffer source data
2022-05-10 15:08:46 +02:00
Nathan Bradshaw
b8b8dffac4
DBZ-4600 - Deprecate ByteBufferConverter from 1.9
2022-05-10 15:08:46 +02:00
Nathan Bradshaw
4bf01a95a2
DBZ-4600 - Start to rough-out ByteArrayConverter
...
- Add delegator support to ByteArrayConverter. Add missing tests
- Add required headers, correct typos
2022-05-10 15:08:46 +02:00
Mark Bereznitsky
ae00729aa2
DBZ-5087 Added a decorator for result sets that can cancel the parent statements
2022-05-06 07:52:00 +02:00
Sergei Morozov
f6546e4d87
DBZ-5075: Remove unused legacy MySQL code
2022-05-03 13:46:23 +02:00
harveyyue
d03e555144
DBZ-5072 Update the two properties to topic-routing doc
2022-05-03 13:44:58 +02:00
harveyyue
25205b233c
DBZ-5072 Config the cache size property for ByLogicalTableRouter caches
2022-05-03 13:44:58 +02:00
Debezium Builder
901825810a
[maven-release-plugin] prepare for next development iteration
2022-04-28 13:51:13 +00:00
Debezium Builder
05dcb45647
[maven-release-plugin] prepare release v2.0.0.Alpha1
2022-04-28 13:51:12 +00:00
Chris Cranford
09e58c5a5c
DBZ-5006 Allow "/" characters in Oracle table names
2022-04-27 08:52:29 -04:00
Sergei Morozov
fd46a670ad
DBZ-5033: Ignore null offsets when recovering database history
2022-04-26 11:40:21 +02:00
Sergei Morozov
e79305cf93
DBZ-5033: Update Javadocs in DatabaseHistory
2022-04-26 11:40:21 +02:00
Vojtech Juranek
da5378d095
DBZ-4949 Adjust one test to use features > Java 11
...
Adjust one test to use multiline strings, which is a feature added in
Java 15 and thus verify Java source level > 11 works for tests.
2022-04-22 05:53:51 +02:00
Brad Morgan
8e25c52585
DBZ-4989 Passthrough payload schema in EventRouterDelegate
2022-04-14 12:17:53 +02:00
Jiri Pechanec
053bf602af
DBZ-4979 Guard against NPE for non-table related DDLs
2022-04-12 09:36:14 -04:00
Debezium Builder
4a6de64c25
[maven-release-plugin] prepare for next development iteration
2022-04-05 14:49:52 +00:00
Debezium Builder
683851fd90
[maven-release-plugin] prepare release v1.9.0.Final
2022-04-05 14:49:51 +00:00
Debezium Builder
ab7fe3b96a
[maven-release-plugin] prepare for next development iteration
2022-03-25 16:45:06 +00:00
Debezium Builder
d19fc234ee
[maven-release-plugin] prepare release v1.9.0.CR1
2022-03-25 16:45:06 +00:00
Sergei Morozov
010f8c9d50
DBZ-3535: Rework CloudEventsConverter
2022-03-24 09:37:05 +01:00
Sergei Morozov
854186e846
DBZ-3535: Rework ByLogicalTableRouter
2022-03-24 09:37:05 +01:00
Sergei Morozov
139b129cf5
DBZ-3535: Rework TransactionMonitor
2022-03-24 09:37:05 +01:00
Sergei Morozov
fccc31c35b
DBZ-3535: Rework ConnectTableChangeSerializer
2022-03-24 09:37:05 +01:00
Sergei Morozov
b258674782
DBZ-3535: Introduce schema.name.adjustment.mode
2022-03-24 09:37:05 +01:00
Sergei Morozov
3a03ff0186
DBZ-3535: Remove unused EventDispatcher constructor parameter
2022-03-24 09:37:05 +01:00
Vojtech Juranek
0163c07800
DBZ-4786 Avoid NPE when removing column which doesn't exists
2022-03-23 07:53:06 +01:00
Vojtech Juranek
fdfc7a6647
DBZ-3762 Filter out MySQL inline comments by default
2022-03-21 16:57:43 +01:00
kate
4b2d5c29be
DBZ-4878 Re-read incremental snapshot chunk on DDL event
2022-03-18 07:55:43 +01:00
harveyyue
044e454e0a
DBZ-4832 Don't set truststore/keystore parameters to system variables
2022-03-15 11:25:28 +01:00
Josh Ribera
020e845f44
DBZ-4809 Add task id and partition to the logging context for
...
multi-partition connectors.
Co-authored-by: Sergei Morozov <morozov@tut.by>
2022-03-09 20:46:38 +01:00
Gunnar Morling
d20a6f00e7
DBZ-4801 Using JdbcConfiguration in JdbcConnection
2022-03-07 17:53:45 +01:00
Gunnar Morling
c1c322ecf0
DBZ-4801 Simplifying JdbcConnection constructors
2022-03-07 17:53:45 +01:00
harveyyue
856ab85cef
DBZ-4824 Expect the null value with snapshot CapturedTables metric when skipping snapshotting
2022-03-07 12:19:13 +01:00
Debezium Builder
fece16039c
[maven-release-plugin] prepare for next development iteration
2022-03-03 17:00:08 +00:00
Debezium Builder
e0d8a0d809
[maven-release-plugin] prepare release v1.9.0.Beta1
2022-03-03 17:00:08 +00:00
yingyingtang-brex
eb3f02b9c0
DBZ-4813 indicating message id and key in outbox log messages
2022-03-03 10:47:20 +01:00
Josh Ribera
fc6d55e935
DBZ-4783: Add task id label to metrics
2022-03-01 14:21:39 +01:00
Mike Kamornikov
37a67d5dcf
DBZ-4783: Manage change tables per partition
2022-03-01 14:21:39 +01:00
Chris Cranford
e472a06da4
DBZ-4773 Fix formatting
2022-02-23 08:49:31 +01:00
Gunnar Morling
a144d84257
DBZ-4773 Using right constructor signature based on Kafka client version
2022-02-23 08:49:31 +01:00
Chris Cranford
9103a220b7
DBZ-4744 Correctly parse signal table name when using domain name
2022-02-22 15:27:21 +01:00
Jiri Pechanec
d13a1635d7
DBZ-3952 Store schema for tables not in schema collection list
2022-02-17 07:41:00 +01:00
Sergei Morozov
2d14b5b09f
DBZ-4478: Multi-partition metrics for SQL Server
2022-02-16 11:31:38 +01:00
Sergei Morozov
6d0aa9408c
DBZ-4478: Add Partition parameter to io.debezium.pipeline.source.spi.*Listener interfaces
2022-02-16 11:31:38 +01:00
Sergei Morozov
af8ea4d73d
DBZ-4478: Move logging from metrics to coordinator
2022-02-16 11:31:38 +01:00
Sergei Morozov
24ff3c9eba
DBZ-4478: Remove throws InterruptedException from DataChangeEventListener::onEvent
2022-02-16 11:31:38 +01:00
Jiri Pechanec
54ba2cd598
DBZ-4736 Handle string NULL default values correctly
2022-02-11 12:06:55 +01:00
harveyyue
8431a614a2
DBZ-4730 Expect plain value instead of scientific exponential notation when using decimal string mode
2022-02-10 17:40:12 +01:00
Debezium Builder
7a47f478a4
[maven-release-plugin] prepare for next development iteration
2022-02-09 07:23:10 +00:00
Debezium Builder
b1dd35025e
[maven-release-plugin] prepare release v1.9.0.Alpha2
2022-02-09 07:23:10 +00:00
Farid Uyar
a559e51b08
DBZ-4547 Create History Topic with optional RF
2022-02-08 16:58:13 +01:00
Paweł Malon
7f7e71b36a
DBZ-4704 Replaced hasNotField
with hasField
and negated the result of it
2022-02-07 20:39:52 +01:00
Paweł Malon
abd8c4202f
DBZ-4704 Fixed the problem with expanding JSON payload that contains arrays of objects
2022-02-07 20:39:52 +01:00
Chris Cranford
055d5ba474
DBZ-4701 executeWithoutCommitting throw exception when auto-commit is enabled
2022-02-07 06:21:11 +01:00
Snigdhajyoti Ghosh
0ab45ecab0
DBZ-4518 Make kafka query timeout configurable
2022-02-04 14:08:15 +01:00
Chris Cranford
3980210c0c
DBZ-4648 Relax case requirement for SnapshotType
2022-02-03 18:15:23 +01:00
nathan
d5dcde8f7a
DBZ-4646 Changing exists method for FileDataBaseHistory to test if file is empty.
2022-02-03 16:47:22 +01:00
Mark Drilling
8a5279d304
DBZ-4314 Define Oracle category grouping and ordering ( #3151 )
2022-02-02 15:33:01 -06:00
Jiri Pechanec
74ea147152
DBZ-3679 Clarify log messages
2022-02-02 14:33:41 -05:00
Chris Cranford
088df2db8e
DBZ-4584 Fix test failures
2022-01-31 09:14:46 +01:00
Chris Cranford
dcd8607108
DBZ-4584 Support case-sensitive Oracle incremental snapshots
2022-01-31 09:14:46 +01:00
Debezium Builder
33e2f5cd0f
[maven-release-plugin] prepare for next development iteration
2022-01-26 19:00:20 +00:00
Debezium Builder
a77b6f8b8e
[maven-release-plugin] prepare release v1.9.0.Alpha1
2022-01-26 19:00:20 +00:00
Jiri Pechanec
55894e57c2
DBZ-4610 Upgrade to Kafka 3.1.0
2022-01-25 15:26:42 +01:00
Chris Cranford
bd604a23cd
DBZ-4419 Add "none" option to skipped.operations
2022-01-22 00:33:21 +01:00
Chris Cranford
773debc340
DBZ-4419 Deprecate TruncateHandlingMode, favor skipped.operations
2022-01-22 00:33:21 +01:00
Jiri Pechanec
664d989e42
DBZ-4577 Provide custom error message handling
2022-01-21 10:24:19 +01:00
Jiri Pechanec
8f5bb31eab
DBZ-3700 Strings a bytes only when converters present
...
When database was used with non-UTF8 charset the snapshot was sending
byte array with UTF-8 bytes but MySQL converter was using database based
charset to convert it to String leading to malformed data.
It is necessary to convert strings using the database charset but it
might be a problem in case of non-charset matching.
The current solution
1) Uses strings by default
2) Switches to byte array only when converters are present to cover
original use case
3) If the conversion fails it switches back to string
2022-01-19 16:20:32 +01:00
Jiri Pechanec
8fd3243a86
DBZ-2224 Test logging based on logback
2022-01-18 11:24:21 +01:00
Sergei Morozov
17fba7258b
DBZ-4541: Promote failure to register metrics to exception
2022-01-14 19:50:57 +01:00
Sergei Morozov
6d97433d18
DBZ-4541: Remove logger from Metrics register/unregister signatures
2022-01-14 19:50:57 +01:00
Chris Cranford
610e8eba73
DBZ-4548 Fix DatabaseVersionResolver comparison logic
2022-01-13 13:29:34 -05:00
Sergei Morozov
0329859cda
DBZ-4459: Introduce interfaces and default implementations for change event source metrics
2022-01-12 14:32:19 +01:00
Gunnar Morling
f0b4a5ba9f
DBZ-4351 Misc. clean-up
2022-01-06 19:03:46 +01:00
harveyyue
291e8ac221
DBZ-4351 Add create/update/delete event seen metrics for monitor upstream dml operation
2022-01-06 19:03:46 +01:00
harveyyue
831217053c
DBZ-4503 Drop the primary key column getting exception
2022-01-03 10:57:07 +01:00
harveyyue
3670b0c6ef
DBZ-4497 Parse multiple signed/unsigned keyword from ddl statement failed
2022-01-03 10:47:03 +01:00
Gunnar Morling
92665f1799
DBZ-4472 Improving log output
2021-12-20 15:58:14 +01:00
Jiri Pechanec
d13e6f546c
DBZ-4467 Skip unsupported test for Db2
2021-12-20 11:29:30 +01:00
Debezium Builder
67c9663c98
[maven-release-plugin] prepare for next development iteration
2021-12-16 10:11:29 +00:00
Debezium Builder
8d532d895f
[maven-release-plugin] prepare release v1.8.0.Final
2021-12-16 10:11:28 +00:00
Chris Cranford
f99ad7512f
DBZ-3940 Add connector-driven EventRouter configuration
2021-12-13 13:52:31 +01:00
Jiri Pechanec
740b6b21f6
DBZ-4298 Support non-JDBC logical values in keys for incremental snapshots
...
Incremental snapshot source gets the PK values after they are processed
by value converter. This is usally correct solution. The values are
passed to chunk queries. There are few datatypes like unconstrained
NUMERIC in PostgreSQL that must be converted to a Debezium specific
class to keep and enrich the value. Such class is not known to JDBC
driver. This commit intorduce a marker interface that enables the code
to identify the value not recognized by JDBC and provides a method to
obtain the original value.
2021-12-13 12:29:21 +01:00
Gunnar Morling
0023cb10a5
DBZ-4393 Create a Debezium Schema Generator for Debezium connectors
...
* added an API generator for Debezium connectors and static API definitions for connectors in a separate module
* added Maven plug-in
* added GH workflow for debezium-schema-generator
Co-authored-by: rkerner <rkerner.mobil@gmail.com>
Co-authored-by: Anisha Mohanty <anishamohanty23@gmail.com>
2021-12-10 12:35:09 +01:00
Gunnar Morling
73cfe71342
DBZ-4423 Micro-optimizing Strings::duration()
2021-12-10 11:30:18 +01:00
Debezium Builder
f53e6c00df
[maven-release-plugin] prepare for next development iteration
2021-12-09 20:16:04 +00:00
Debezium Builder
826837701a
[maven-release-plugin] prepare release v1.8.0.CR1
2021-12-09 20:16:04 +00:00
Jiri Pechanec
b1dcfcb0b8
DBZ-3342 Incremental snapshot support for MongoDB
2021-12-09 10:03:41 +01:00
Tom Billiet
ec9f122e0a
DBZ-4385 Truncate support for Oracle
2021-12-08 15:00:53 -05:00
Gunnar Morling
1f8fd22685
DBZ-4341 Improving logging;
...
Also fixing incorrect JavaDoc.
2021-12-08 13:12:54 +01:00
Chris Cranford
8f007e08a2
DBZ-4388 Don't set default value if resolution fails
2021-12-07 16:38:22 +01:00
Debezium Builder
b5439b54bf
[maven-release-plugin] prepare for next development iteration
2021-11-30 09:38:47 +00:00
Debezium Builder
125f6e5aa2
[maven-release-plugin] prepare release v1.8.0.Beta1
2021-11-30 09:38:47 +00:00
Jiri Pechanec
2789dd610a
DBZ-4272 Throw exception for non matching schema
2021-11-30 09:28:44 +01:00
Jiri Pechanec
5e40e1cdf5
DBZ-4272 Skip connector test using annotations
2021-11-30 09:28:44 +01:00
Jiri Pechanec
358eb0379b
DBZ-4272 Disable tests for SQL Server
2021-11-30 09:28:44 +01:00
Chris Cranford
0a10d5b484
DBZ-4272 Trigger schema refresh before snapshot; only for PostgreSQL
2021-11-30 09:28:44 +01:00
Chris Cranford
3a8d1ff838
DBZ-4272 Incrementally snapshot after schema change without NPE
2021-11-30 09:28:44 +01:00
Gunnar Morling
2157256398
DBZ-4364 Awaiting table metadata to become queryable
2021-11-29 09:23:18 +01:00
Jiri Pechanec
1c5f3b2330
DBZ-4364 Make byte[] and ByteBuffer comparable
2021-11-29 09:23:18 +01:00
Jiri Pechanec
4a2a60db74
DBZ-4364 Fix missing method call
2021-11-29 09:23:18 +01:00
Gunnar Morling
560ef2babd
DBZ-2363 Misc. adjustments;
...
* Adding Strings::isBlank()
* Simplifying test
* Using long in cases where we know it's present
* Removing some unused code
* Testing binary handling mode
* Avoiding TX id value overflow
* Assigning names to schemas to avoid issues as in DBZ-650
2021-11-26 14:29:15 +01:00
lairen
a0898ed440
DBZ-2363 Adding support for pg_logical_emit messages in Postgres connector for the pgoutput plugin
2021-11-26 14:29:15 +01:00
Jacob Gminder
f7f8748062
DBZ-2975: Move the poll interval logic to the ChangeEventSourceCoordinator.
2021-11-22 12:30:31 +01:00
Gunnar Morling
a7c6d18489
DBZ-4309 Removing legacy logger
2021-11-19 08:04:44 +01:00
Andrey Yegorov
b6d2c41fb0
DBZ-4309 Avoid calling 'LoggerFactory.getLogger(getClass())' for each new instance of the RelationalChangeRecordEmitter
2021-11-18 22:37:51 +01:00
Chris Cranford
2730187a4b
DBZ-4012 Deprecate table.field.event.payload.id
2021-11-18 21:45:41 +01:00
Chris Cranford
ab86cd18a9
DBZ-4012 Suggested changes
2021-11-18 21:45:41 +01:00
Chris Cranford
349b9690f4
DBZ-4012 Suggested changes
2021-11-18 21:45:41 +01:00
Chris Cranford
0890202539
DBZ-4012 Remove incubating from EventRouter
2021-11-18 21:45:41 +01:00
Chris Cranford
0df5c6219b
DBZ-4012 Bubble up SMT validation failures to Kafka Connect
2021-11-18 21:45:41 +01:00
Chris Cranford
970792bc40
DBZ-4012 Deprecate/Rename debezium namespace config options
2021-11-18 21:45:41 +01:00
Chris Cranford
7a4140c6aa
DBZ-4012 ActivateTracingSpan is pass-through when OpenTracing unavailable
2021-11-18 21:45:41 +01:00
Gunnar Morling
ca95b424fb
DBZ-3528 Misc. clean-up
2021-11-17 16:02:06 +01:00
Sungho Hwang
78c92c3772
DBZ-3528 turned access modifier of jsonNodeToSchemaBuilder() back to private
2021-11-17 16:02:06 +01:00
Gunnar Morling
03433b663b
DBZ-3528 Misc. fixes
2021-11-17 16:02:06 +01:00
Sungho Hwang
7c95636668
DBZ-3528 Chaged variable naming for better readability
2021-11-17 16:02:06 +01:00
Sungho Hwang
c9eb0f9157
DBZ-3528 Refactored several parts according to the feedbacks
...
A delegate class for common logic between two Outbox Event Routers has been created.
Documentation has been fixed.
The configuration properties for MongoDB Outbox Event Router have been changed to use existing Outbox Event Router's.
2021-11-17 16:02:06 +01:00
Sungho Hwang
0b9fc88ea6
DBZ-3528 Applied the code style rules
2021-11-17 16:02:06 +01:00
Sungho Hwang
2c279b07f6
DBZ-3528 Wrote MongoDB Outbox Event Router with documentation and its unit & integration tests
2021-11-17 16:02:06 +01:00
Chris Cranford
e3da449303
DBZ-4276 Expose unavailable value placeholder for PG and Oracle only
2021-11-16 20:29:59 +01:00
Chris Cranford
ca17352221
DBZ-4276 Unify unavailable value placeholder handling
...
Deprecate the `toasted.value.placeholder` option in PostgreSQL in favor
of the new `unavailable.value.placeholder` option that is shared by all
relational connectors.
2021-11-16 20:29:59 +01:00
Kate
44073cf7d8
DBZ-4196 Support schema changes during incremental snapshot
2021-11-16 14:56:06 +01:00
camilesing
dfabe8cda8
DBZ-4275 fix AbstractDatabaseHistory typo
2021-11-14 19:21:37 +01:00
Debezium Builder
f93e901dce
[maven-release-plugin] prepare for next development iteration
2021-11-11 09:21:59 +00:00
Debezium Builder
895a16bf19
[maven-release-plugin] prepare release v1.8.0.Alpha2
2021-11-11 09:21:59 +00:00
Chris Cranford
85101f0ad7
DBZ-3611 Avoid NPE with database include/exclude lists
2021-11-11 09:24:05 +01:00
Kate
992ae537c8
DBZ-4244 Quote signal table name in MySQL connector
2021-11-10 10:18:26 +01:00
Chris Cranford
5afda302ec
DBZ-3710 Javadoc updates
2021-11-10 09:43:25 +01:00
Chris Cranford
5ec9381702
DBZ-3710 Unify DefaultValueMapper (Oracle/SQL Server/PostgreSQL)
2021-11-10 09:43:25 +01:00
Chris Cranford
343318b5d4
DBZ-3710 Support Oracle column default values
2021-11-10 09:43:25 +01:00
harveyyue
aa3ae05e22
DBZ-4107 Incremental snapshot doesn't work without primary key
2021-11-05 13:31:26 +01:00
Gunnar Morling
1226c3b2ab
DBZ-4029 Misc. clean-up;
...
* Exposing heartbeatInterval() on CommonConnectorConfig
* Closing heartbeat connection
* Making test more resilient
* Correct member ordering and visibility
* Doc fixes
2021-11-05 13:27:09 +01:00
ahodavdekar
91b7ef2c36
DBZ-4029 Adding support for heartbeat action queries to MySQL connector
2021-11-05 13:27:09 +01:00
Gunnar Morling
b78e318399
DBZ-3966 Expanding test
2021-11-04 10:47:24 +01:00
jiabao.sun
2987e96c44
DBZ-3966 Serializing hasDefaultValue in JsonTableChangeSerializer
2021-11-04 10:47:17 +01:00
Gunnar Morling
d1cae3f3b7
DBZ-3966 Making defaultValueExpression() return Optional<String> from editor too
2021-11-04 10:47:17 +01:00
jiabao.sun
24ead16c24
DBZ-3966 Remove defaultValue and keep defaultValueExpression
2021-11-04 10:47:17 +01:00
jiabao.sun
77c4113697
DBZ-3966 JsonTableChangeSerializer support serialization for defaultValue and enumValues
2021-11-04 10:47:17 +01:00
Kate
a645d0bc7e
DBZ-4197 Process transaction started/committed in R/O snapshot
2021-11-04 06:19:58 +01:00
ahodavdekar
1d81b6db19
DBZ-4077 Exposing the transaction metadata topic name as a config option
2021-11-02 06:07:05 +01:00
Gunnar Morling
fd06a4e2a4
DBZ-2975 Using Offsets in recover() contract;
...
Keeping legacy method until the legacy MySQL connector implementation has been removed.
2021-11-01 10:17:49 +01:00
Mike Kamornikov
16c07e915b
DBZ-2975: Recover database schema from multiple partitions
2021-11-01 10:17:49 +01:00
Debezium Builder
2026a48302
[maven-release-plugin] prepare for next development iteration
2021-10-27 06:47:27 +00:00
Debezium Builder
97106c1fd9
[maven-release-plugin] prepare release v1.8.0.Alpha1
2021-10-27 06:47:26 +00:00
Jiri Pechanec
a1084360da
DBZ-435 Too fast delete can prevent from obtaining the full document
2021-10-26 18:13:16 +02:00
Jiri Pechanec
cb36b575db
DBZ-435 Support for Change Streams capture
2021-10-26 18:13:16 +02:00
Sungho Hwang
54d5c16b69
DBZ-4191 Fixed a variable name in an unit test of Outbox Event Router to enhance readability.
2021-10-22 07:57:24 +02:00
harveyyue
90e9bea93f
DBZ-4000 Support parse table and columns comment
2021-10-20 15:19:31 +02:00
Anisha Mohanty
2eae7a742c
DBZ-4045 Use listener props for deprecated host & port props
2021-10-20 11:51:34 +02:00
Gunnar Morling
3d093e2a3b
DBZ-4164 Import ordering
2021-10-18 14:45:00 +02:00
lujiefsi
12ccc24e17
DBZ-4164 Fixing resource leak due to Files::lines
2021-10-18 14:31:39 +02:00
Juan Fiallo
346e0f3af2
DBZ-3517 Making database.user optional for SQL Server.
...
SQL Server doesn't require username/password when using Kerberos authentication, which can be configured using pass-through database properties. Changing the database.user config to be optional for SQL Server Connector to reflect that.
2021-10-18 12:06:56 +02:00
Juan Fiallo
5ac93ac20e
DBZ-3517 Making database.user optional for SQL Server.
...
SQL Server doesn't require username/password when using Kerberos authentication, which can be configured using pass-through database properties. Changing the database.user config to be optional for SQL Server Connector to reflect that.
2021-10-18 12:06:56 +02:00
Juan Fiallo
e041b599d8
DBZ-3517 Making database.user optional for SQL Server.
...
SQL Server doesn't require username/password when using Kerberos authentication, which can be configured using pass-through database properties. Changing the database.user config to be optional for SQL Server Connector to reflect that.
2021-10-18 12:06:56 +02:00
Jiri Pechanec
0ccc83db21
DBZ-3860 Provide correct boundaries for multi PK tables
2021-10-14 17:26:50 +02:00
Gunnar Morling
26f13d83d5
DBZ-3993 Reflectively invoking Runtime.version();
...
As that API is only available from Java 9 onwards, calling it will cause
a compilation failure when compiling with --release.
2021-10-14 09:46:06 -04:00
Gunnar Morling
6b3d2c3b76
DBZ-3993 Formatting fix
2021-10-13 10:59:23 +02:00
Gunnar Morling
c3eed6051b
DBZ-3993 Skipping object size based test on Java 16+
2021-10-13 10:59:23 +02:00
Sergei Morozov
4a03c5df17
DBZ-2975: Include database name into topic and schema names
...
Co-authored-by: Mike Kamornikov <mikekamornikov@gmail.com>
2021-10-12 15:40:02 +02:00
Jiri Pechanec
657c5540b3
DBZ-4083 Add filter to MongoDB cloud converter
2021-10-11 11:19:30 +02:00
Jiri Pechanec
aec2080415
DBZ-3895 Config returns Patterns not Strings
2021-10-11 10:57:33 +02:00
Jiri Pechanec
68f148c3f3
DBZ-3895 Make snapshot collection list matching case insensitive
2021-10-11 10:57:33 +02:00
Chris Cranford
abc3bb42d6
DBZ-4106 Don't continuously reconfigure DatabaseHistory
2021-10-05 20:15:58 +02:00
Gunnar Morling
62715c4e1d
DBZ-4033 Doc update
2021-10-05 10:54:01 +02:00
unalsurmeli
9a13d80074
DBZ-4033 A new hash version has been added so that it can be used to have the same hash value in different places;
...
In V2, the byte array only has the actual value, unlike the serialization based V1 implementation which exposed Java's internal serialization format.
2021-10-05 10:52:30 +02:00
Vivek Wassan
82dad32e2a
DBZ-4078 Checking connection is valid before commit in readChunk()
2021-10-04 10:00:27 +02:00
Laurent Broudoux
77eaa0a293
DBZ-1297 Add boolean type in test
2021-10-01 22:29:53 +02:00
Laurent Broudoux
7b7a1f3064
DBZ-1297 Expand JSON escaped string as JSON
2021-10-01 22:29:53 +02:00
Jiri Pechanec
f910fc930b
[release] Switch to 1.8 snapshot
2021-10-01 14:50:16 +02:00
Debezium Builder
12b0ae9046
[maven-release-plugin] prepare for next development iteration
2021-10-01 13:26:33 +02:00
Debezium Builder
652162fd85
[maven-release-plugin] prepare release v1.7.0.Final
2021-10-01 13:26:33 +02:00
Jiri Pechanec
a2c6058404
DBZ-4057 Initialize Postgres schema before incremental snapshot restart
2021-09-30 09:10:49 +02:00
Debezium Builder
bec0b6b67e
[maven-release-plugin] prepare for next development iteration
2021-09-23 06:31:25 +00:00
Debezium Builder
cf8da39725
[maven-release-plugin] prepare release v1.7.0.CR2
2021-09-23 06:31:25 +00:00
Naveen Kumar KR
62045f6430
DBZ-3887 1) Replace notifyAll with notify as atmost only one thread would be in wait state at any time
...
2) Remove synchronized from jmh metric methods as it doesn't strictly require lock due to nature of metrics
3) Always wait by predefined timeout to gurantee that thread would never stuck
2021-09-22 19:12:21 +02:00
Naveen Kumar KR
858557d184
DBZ-3887 Major improvements for Throughput Bottleneck and Inefficient Batching in ChangeEventQueue
...
* Replaced naive metronome.pause and Thread.sleep with efficient wait() and notifyAll()
* Replaced LinkedBlockingDeque with ArrayDeque for better performance
* Replaced Atomic and Map operation related to maxQueueSize with primitive long and ArrayDeque for better performance
* x10 - x100 Improvements in overall throughput
* Efficient batching based on maxBatchSize
2021-09-22 19:12:21 +02:00
Kate
0099a50c53
DBZ-3948 Support kafka topic based signals
2021-09-22 13:08:45 +02:00
Vivek Wassan
b6e891ec7d
DBZ-3951 Moving preReadChunk() to AbstractIncrementalSnapshotChangeEventSource
2021-09-22 07:01:29 +02:00
Vivek Wassan
65ac2387dc
DBZ-3951 Adding a separate isValid method
2021-09-22 07:01:29 +02:00
Vivek Wassan
5f37e6630f
DBZ-3951 Adding a safety check, isValid, for connection
2021-09-22 07:01:29 +02:00
Naveen Kumar KR
b5ccd249c9
DBZ-4015 1) TableIdParser - Initialize ArrayList with optimal capacity of 3 (at max three parts in an identifier - catalog.schema.table)
...
2) TableId - Replace .stream().toArray() with .toArray() to eliminate stream overhead
2021-09-21 10:35:08 +02:00
Naveen Kumar KR
4c5264f7d2
DBZ-4015 Efficient String Replacement using String.replace in TableIdParser (improvement of x4-x5 times)
...
(https://medium.com/javarevisited/micro-optimizations-in-java-string-replaceall-c6d0edf2ef6 )
2021-09-21 10:35:08 +02:00
Jiri Pechanec
26d0e78023
DBZ-4013 Skip invalid table
2021-09-17 14:20:42 +02:00
Debezium Builder
c15e0b303b
[maven-release-plugin] prepare for next development iteration
2021-09-16 08:10:45 +00:00