Jiri Pechanec
1c9f7a1c1c
DBZ-3099 Document non-volatile fields accessed by a single thread
2021-02-16 10:02:17 +01:00
Jiri Pechanec
745bb6c11d
DBZ-3099 Address review comments; extract history comparator for testability
2021-02-16 10:02:17 +01:00
Jiri Pechanec
cb1476ce18
DBZ-3099 Address review comments
2021-02-16 10:02:17 +01:00
Jiri Pechanec
21b15facb2
DBZ-3099 Snapshot buffering to handle config corner cases
2021-02-16 10:02:17 +01:00
Jiri Pechanec
6a9ca7d544
DBZ-3099 Fix schema recovery; Add table level locking
2021-02-16 10:02:17 +01:00
Jiri Pechanec
08e4988c72
DBZ-3093 Unify snapshot and streaming parsing
2021-02-16 10:02:17 +01:00
Jiri Pechanec
6607310f13
DBZ-3093 Separate parsing and change applying phase in MySQL schema
2021-02-16 10:02:17 +01:00
ani-sha
5f8a22a5df
DBZ-2858 Make field descriptions consistent for time values
2021-02-16 09:55:35 +01:00
Gunnar Morling
b5b73b4409
DBZ-3057 Formatting fix
2021-02-16 09:40:13 +01:00
Chris Cranford
09de77cf7d
DBZ-3057 Add some TableId helper methods
2021-02-16 09:40:13 +01:00
harveyyue
3a4bcb104b
DBZ-3096 Getting RejectedExecutionException when checking topic settings from KafkaDatabaseHistory.checkTopicSettings
2021-02-12 08:37:16 +01:00
Kevin Pullin
2426de849a
DBZ-3074 PgOutputMessageDecoder - Fix unordered keys
...
Fix issue where `PgOutputMessageDecoder` does not define primary key metadata in key-order.
`PgOutputMessageDecoder.getTablePrimaryKeyColumnNamesFromDatabase()` returns a `Set`, leading to unordered
primary keys in the metadata. Switch the logic to call `JdbcConnection.readPrimaryKeyNames()` which
_does_ correctly return an ordered primary key list.
2021-02-10 10:58:26 +01:00
Yilong Chang
0e63a5a651
DBZ-3075 Close Kafka Database History producer with a timeout of 30s.
2021-02-10 09:08:02 +01:00
Debezium Builder
e08df0a177
[maven-release-plugin] prepare for next development iteration
2021-02-06 01:25:23 -05:00
Debezium Builder
da0411bf70
[maven-release-plugin] prepare release v1.5.0.Alpha1
2021-02-06 01:25:23 -05:00
Jiri Pechanec
f57343dc3b
DBZ-1865 Use OptionaLong; add comments
2021-02-04 09:07:55 +01:00
Jiri Pechanec
2374b6b045
DBZ-1865 First draft of MySQL rewrite
2021-02-04 09:07:55 +01:00
Gunnar Morling
6c5fb793af
DBZ-2575 Marking option as private
2021-02-04 08:28:55 +01:00
Martin Sillence
164ab1330c
DBZ-2575 Improve performance where tables we fetch are much less than total tables
2021-02-04 08:28:49 +01:00
Naveen Kumar KR
75882f35ac
DBZ-2382 Support emitting TRUNCATE events in PG11+ PGOUTPUT plugin
...
default - TruncateHandlingMode.SKIP
supported configs - TruncateHandlingMode.SKIP / TruncateHandlingMode.INCLUDE
2021-02-03 13:53:01 +01:00
ani-sha
d4003c9172
DBZ-2420 Define database configs under RelationalDatabaseConnectorConfig
2021-02-03 08:05:42 +01:00
Debezium Builder
237b4ac0f1
[maven-release-plugin] prepare for next development iteration
2021-01-28 07:38:55 -05:00
Debezium Builder
06b1d0cbad
[maven-release-plugin] prepare release v1.4.1.Final
2021-01-28 07:38:55 -05:00
Gunnar Morling
f76d8e0869
DBZ-2938 Calling snapshot listener only after complete() hook
2021-01-19 15:16:46 +01:00
Gunnar Morling
6d441bc538
DBZ-2938 Centralizing listener handling; avoiding some empty methods
2021-01-19 15:16:46 +01:00
rkerner
d7ac0f7e6b
DBZ-2572 Fix docs for message.key.columns and skipped.operations
2021-01-12 14:08:21 +01:00
Debezium Builder
3010f99c32
[maven-release-plugin] prepare for next development iteration
2021-01-07 03:38:13 -05:00
Debezium Builder
54534538d9
[maven-release-plugin] prepare release v1.4.0.Final
2021-01-07 03:38:13 -05:00
Gunnar Morling
21c3003187
DBZ-2889 Don't access payload id field if not present
2021-01-06 14:46:58 +01:00
Chris Cranford
d32e4b3720
DBZ-2849 Added test
2021-01-05 12:42:08 +01:00
Chris Cranford
8409282739
DBZ-2849 Fix value converter field lookups with sanitized field names
2021-01-05 12:42:08 +01:00
Jiri Pechanec
d178365529
DBZ-2880 Guava bumped to 30.0
2021-01-05 12:35:32 +01:00
Jiri Pechanec
58713599ba
DBZ-2167 Remove legacy parser completely; tests depending on parser move to MySQL
2020-12-21 16:30:15 +01:00
Jiri Pechanec
00f9e3b261
DBZ-2167 Use legacy parser for testing purposes only
2020-12-21 16:30:15 +01:00
Debezium Builder
b1ab996c3e
[maven-release-plugin] prepare for next development iteration
2020-12-16 14:11:31 -05:00
Debezium Builder
8fc864b278
[maven-release-plugin] prepare release v1.4.0.CR1
2020-12-16 14:11:31 -05:00
Debezium Builder
cf921f56b4
[maven-release-plugin] prepare for next development iteration
2020-12-10 00:27:40 -05:00
Debezium Builder
c49a377487
[maven-release-plugin] prepare release v1.4.0.Beta1
2020-12-10 00:27:39 -05:00
Jiri Pechanec
8e333d9b1d
DBZ-1818 DBZ-559 Changes after review
...
OpenTracing not used in names
Span for event time
Span SMT embedded in outbox
Handling of missing tracing libs in Connect runtime
Aligned tracing tag names with the standard ones
2020-12-08 16:09:09 +01:00
Jiri Pechanec
318340078b
DBZ-1818 DBZ-559 Integration with OpenTracing
2020-12-08 16:09:09 +01:00
Jiri Pechanec
b7fede7d53
DBZ-735 Verify Avro serialization
2020-11-27 17:33:23 +01:00
Debezium Builder
4535549dcd
[maven-release-plugin] prepare for next development iteration
2020-11-16 15:18:43 -05:00
Debezium Builder
3c70c5f7d5
[maven-release-plugin] prepare release v1.4.0.Alpha2
2020-11-16 15:18:43 -05:00
vanhoale
17a424d9ac
DBZ-2662 Handle ChangeEventQueue by the size in bytes
2020-11-16 08:39:33 +01:00
rkerner
a611979268
DBZ-2580 apply PR feedback
2020-11-12 07:42:48 +01:00
rkerner
9c5db0cd2f
DBZ-2580 Fix "The primary key cannot reference a non-existant column" error from MySQL DDL parser when CREATE TABLE statement starts with a primary key definition like "CREATE TABLE Products (PRIMARY KEY (id), ..." where the referenced primary key column is not yet defined
2020-11-12 07:42:48 +01:00
Chris Cranford
aea26a2949
DBZ-2742 Migrate initial.sync.max.threads to snapshot.max.threads
2020-11-11 09:08:05 +01:00
Ganesh Ramasubramanian
4716194ff9
DBZ-2606: Map new name for the fields and headers
...
Added ability to map new name for the fields and headers.
The "new name" mapping is completely optional and we will not
be affecting the existing functionality.
The optional "new name" is case-sensitive.
2020-11-04 08:57:16 +01:00
ani-sha
fd6c8f9502
DBZ-2690 get oracle specific changes into debezium-core
2020-11-04 07:21:10 +01:00
rkerner
c274c2b39b
DBZ-2617 refactor to proper HeartbeatErrorHandler injection
2020-11-03 07:23:02 +01:00
rkerner
2a3f44ebe5
DBZ-2617 failing heartbeats should put Postgres connector into FAILED state to allow proper shutdown of Postgres when shutdown was triggered with shutdown mode "fast"
2020-11-03 07:23:02 +01:00
Sergei Morozov
344acb55c0
DBZ-2682: Mapper converters should leave empty dates as zero
2020-10-27 16:21:53 +01:00
Debezium Builder
26883dad92
[maven-release-plugin] prepare for next development iteration
2020-10-22 03:45:59 -04:00
Debezium Builder
9adbc2d67e
[maven-release-plugin] prepare release v1.4.0.Alpha1
2020-10-22 03:45:59 -04:00
Jiri Pechanec
2cded66d30
DBZ-2632 Minor improvements
2020-10-20 16:28:23 +02:00
Arik Cohen
2e8c43bd60
DBZ-2632: Forcibly abort the JDBC connection if failed to close it gracefully.
2020-10-20 16:28:23 +02:00
Jiri Pechanec
ee5353ef6f
DBZ-2653 Process commits even with tx monitoring
2020-10-20 10:42:10 +02:00
Kaushik Iyer
7d9e8fdc46
DBZ-2456 fixes after rebase
2020-10-16 10:13:48 +02:00
Kaushik Iyer
f752b3ab39
DBZ-2456 updated common connector config with generalized naming
2020-10-16 10:13:48 +02:00
Kaushik Iyer
e2e6dc0932
DBZ-2456 selective snapshot implementation
2020-10-16 10:13:48 +02:00
Debezium Builder
95b8ed3877
[maven-release-plugin] prepare for next development iteration
2020-10-01 09:08:51 -04:00
Debezium Builder
19ad73beb2
[maven-release-plugin] prepare release v1.3.0.Final
2020-10-01 09:08:50 -04:00
Gunnar Morling
b792ac3274
DBZ-2495 Misc. fixes
2020-09-30 12:13:45 +02:00
Gunnar Morling
6b5a3100e9
DBZ-2495 Misc. clean-up
2020-09-30 12:13:45 +02:00
James Gormley
9e780ac481
DBZ-2495 Updating per PR comments
2020-09-30 12:13:45 +02:00
James Gormley
1b845535ab
DBZ-2495 Updating tables to utilize Table.editor for creating new tables
2020-09-30 12:13:45 +02:00
James Gormley
c7b53e1992
DBZ-2495 Updating docs and comments
2020-09-30 12:13:45 +02:00
James Gormley
0d46ec47da
DBZ-2495 Updating to handle captured_column_list without specify column.exclude.list
2020-09-30 12:13:45 +02:00
Gunnar Morling
c1b057f792
DBZ-2588 Introducing dedicated functional interface for key/value generators;
...
Also keys are always KC Struct instead of generic j.l.Object.
2020-09-25 06:01:59 +02:00
Debezium Builder
b146b69154
[maven-release-plugin] prepare for next development iteration
2020-09-24 04:56:42 -04:00
Debezium Builder
c51cfe8eb9
[maven-release-plugin] prepare release v1.3.0.CR1
2020-09-24 04:56:41 -04:00
Chris Cranford
df41cd8bfe
DBZ-2396 Suggested changes
2020-09-24 07:41:36 +02:00
Chris Cranford
0db4997102
DBZ-2396 Support Converter delegation in ByteBufferConverter
...
In the even that the value supplied to the ByteBufferConverter
is not supported, e.g. not BYTES, the converter should then
support delegation to a configured converter.
2020-09-24 07:41:36 +02:00
Jiri Pechanec
73c74189fa
DBZ-2549 Move scripting code into a separate module
2020-09-18 11:17:17 +02:00
Debezium Builder
837669890a
[maven-release-plugin] prepare for next development iteration
2020-09-16 12:18:08 -04:00
Debezium Builder
086dbd960b
[maven-release-plugin] prepare release v1.3.0.Beta2
2020-09-16 12:18:08 -04:00
Ganesh Ramasubramanian
6ca025b1bb
DBZ-2504: Add support for field and header prefix
...
Added support for field and header prefix.
Updated the documentation to include the options
'add.fields.prefix' and 'add.headers.prefix'.
2020-09-16 11:31:05 +02:00
rkerner
011c3be948
DBZ-2527 Add missing ListOfRegex validator to all regex list fields and fix dependent fields to not include @Deprecated whitelist/blacklist fields
2020-09-15 10:05:04 +02:00
rkerner
385a873172
DBZ-2526 add DisplayName for MASK and TRUNCATE COLUMN fields
2020-09-15 10:05:04 +02:00
rkerner
5c5fb5dc2c
DBZ-2516 add config fields for datatype.propagate.source.type
and column.propagate.source.type
properties
2020-09-15 10:05:04 +02:00
Gunnar Morling
73ba6d42a0
DBZ-137 Docs update
2020-09-14 12:33:43 +02:00
Chris Cranford
a686c1c875
DBZ-137 Suggested changes
2020-09-14 12:33:43 +02:00
Chris Cranford
5ca36a84ad
DBZ-137 Reimagine stream.fetch.size as query.fetch.size
2020-09-14 12:33:43 +02:00
Chris Cranford
8584831f51
DBZ-137 Use Duration for CONNECTION_TIMEOUT_MS
2020-09-14 12:33:43 +02:00
Chris Cranford
43440ca76f
DBZ-137 Remove snapshot.skip.locks for SQL Server
2020-09-14 12:33:43 +02:00
Chris Cranford
72d59348ed
DBZ-137 Fix rebase
2020-09-14 12:33:43 +02:00
AndreyIg
77662a0dde
DBZ-137 contribution to the core, Oracle related
2020-09-14 12:33:43 +02:00
cooksey
8bcbdb639d
DBZ-2094 Allow Postgres snapshotter to set streaming start on resume
...
When a connector resumes after previously streaming and takes a
snapshot, through a new method on the snapshotter interface,
shouldStreamEventsStartingFromSnapshot, can choose whether
to resume streaming from the last known streaming position or the
beginning of the snapshot. This is helpful for snapshotters that
may not want to resnapshot every table in the whitelist/blacklist
but not miss event on the tables that are skipped.
2020-09-09 10:54:11 +02:00
Grzegorz Kołakowski
c74579c1df
DBZ-1068 Forward port the work by @grzegorz8 to master
...
Co-authored-by: GuyIEX <gpascarella@ironeaglex.com>
2020-09-04 12:06:57 +02:00
Nathan Mills
6e915b7cfd
DBZ-2473 Decrement remaining for skipped tables
2020-09-04 07:50:34 +02:00
Debezium Builder
9979db648f
[maven-release-plugin] prepare for next development iteration
2020-08-28 05:04:26 -04:00
Debezium Builder
5c58b23756
[maven-release-plugin] prepare release v1.3.0.Beta1
2020-08-28 05:04:26 -04:00
Jiri Pechanec
d9f616caed
DBZ-2303 Set source when new table detected
2020-08-24 13:08:09 +02:00
rkerner
2c5c093587
DBZ-2171 avoid divisive language
2020-08-21 09:29:04 +02:00
Gunnar Morling
4c3cf9f93e
DBZ-2427 Logging inability to unregister metrics MBean as WARN
2020-08-13 14:05:11 -04:00
Gunnar Morling
dac7f3998c
DBZ-2412 Housekeeping;
...
* Static logger
* Protecting against empty topic name
* Removing unused logger
2020-08-13 13:53:50 -04:00
Gunnar Morling
19ae707a18
DBZ-2412 Ignoring non change events in logical router
2020-08-13 13:53:50 -04:00
Gunnar Morling
44efd1bc6b
DBZ-2423 Propagating port default values
2020-08-12 13:50:42 +02:00
Gunnar Morling
26b51e02cb
DBZ-2418 Misc. clean-up
2020-08-12 09:28:35 +02:00
Jiri Pechanec
0b95df28d5
DBZ-2411 Use Scala 2.12
2020-08-11 09:20:53 +02:00
Jiri Pechanec
391d086529
DBZ-2411 Upgrade to Kafka 2.6.0
2020-08-11 09:20:53 +02:00
Debezium Builder
c4c50f5fc6
[maven-release-plugin] prepare for next development iteration
2020-08-06 00:46:18 -04:00
Debezium Builder
94cbeb95fa
[maven-release-plugin] prepare release v1.3.0.Alpha1
2020-08-06 00:46:17 -04:00
rkerner
cad6cf62d3
DBZ-2315 fix BIT VARYING handling in Postgres connector
...
* removed little endian padding for BIT types in JdbcValueConverters (only used for Postgres yet)
* removed legacy format handling fot BIT related types in JdbcValueConverters#convertBits which were leftovers from PR #1408
* removed unnecessary zero-ing of a newly created byte array to improve performance for huge byte arrays
* updated Postgres connector docs
2020-08-03 14:58:31 +02:00
Cory Harper
724fc7ac98
DBZ-2362 add configurable restart wait time
2020-07-30 18:08:26 +02:00
Jiri Pechanec
5fcb221993
DBZ-1491 Database connection provides default value
2020-07-24 09:07:09 +02:00
Jiri Pechanec
45a6f9bef1
DBZ-1491 Refactor after review
2020-07-24 09:07:09 +02:00
Grzegorz Kołakowski
825ffe3848
DBZ-1491 Add SqlServerDefaultValueConverter class
2020-07-24 09:07:07 +02:00
Grzegorz Kołakowski
234341a471
DBZ-1491 Parse temporal values using queries on database
...
* Add tests for altering column default value
* Add end-to-end test
2020-07-24 09:06:30 +02:00
Grzegorz Kołakowski
a8ea5e2256
DBZ-1491 Extract default values from schema
...
The goal is to enrich schema metadata with column default value.
2020-07-24 09:05:58 +02:00
Gunnar Morling
3ebfbcefdb
DBZ-2329 Moving isNewTransaction() to SQL Server implementation
2020-07-24 07:57:50 +02:00
Lukasz Korzeniowski
1ada5c4b23
DBZ-2329 Do not skip events after restart when stream is not monotonic
2020-07-21 14:21:29 +02:00
Debezium Builder
e1ab38de84
[maven-release-plugin] prepare for next development iteration
2020-07-16 04:40:04 -04:00
Debezium Builder
69cf375faa
[maven-release-plugin] prepare release v1.2.1.Final
2020-07-16 04:40:04 -04:00
Jos Huiting
da97be7842
DBZ-2307 - Improve logging when primary keys are missing
2020-07-15 11:18:48 -04:00
Omar Al-Safi
4d2467ca20
DBZ-2302: Allow OffsetCommitPolicy to be initiated with Properties object
2020-07-09 08:38:26 +02:00
Randall Hauch
5860812a63
DBZ-2275 Updated default database history DML filter default to handle additional RDS statements;
...
* Added two more regular expressions to the default database history DML filter. These are intended to catch and filter out `INSERT into mysql.rds_monitor…` statements that RDS may write in the binlog
* Added a new unit test that checks that the regular expressions do catch some example statements
2020-06-25 20:57:49 +02:00
Jiri Pechanec
63a2f6a0ae
DBZ-2255 Document JSON parsing in groovy-based SMTs
2020-06-25 11:08:59 +02:00
Debezium Builder
c0d30b87d9
[maven-release-plugin] prepare for next development iteration
2020-06-24 18:03:11 +02:00
Debezium Builder
0e2ac78301
[maven-release-plugin] prepare release v1.2.0.Final
2020-06-24 18:03:11 +02:00
Jiri Pechanec
87d93d44fd
DBZ-2221 Wait for keepalive thread initialization
2020-06-24 09:14:42 +02:00
Gunnar Morling
b80f064131
DBZ-2250 Allowing to use CloudEventsConverter with Avro via Apicurio
2020-06-22 14:41:10 +02:00
Debezium Builder
b03a4d0e50
[maven-release-plugin] prepare for next development iteration
2020-06-19 00:39:01 -04:00
Debezium Builder
93e60258d5
[maven-release-plugin] prepare release v1.2.0.CR2
2020-06-19 00:39:00 -04:00
Gunnar Morling
f2d13238aa
DBZ-2228 Using unique names for admin clients
2020-06-18 14:08:56 +02:00
Debezium Builder
65c39e9f11
[maven-release-plugin] prepare for next development iteration
2020-06-11 02:00:20 -04:00
Debezium Builder
bce332c900
[maven-release-plugin] prepare release v1.2.0.CR1
2020-06-11 02:00:19 -04:00
Chris Cranford
ac0d9ede8a
DBZ-2152 Fix test failure
2020-06-09 17:20:51 +02:00
Balázs Németh
0853940f3d
DBZ-2169 Fix for misuse of ConfigurationDefaults.RETURN_CONTROL_INTERVAL;
...
* Implemented Temporals.min()
* Fixed pollInterval calculation based on the documentation
2020-06-09 10:24:50 +02:00
Gunnar Morling
bfaadcf2dd
DBZ-2149 Starting/stopping Kafka just once in KafkaDatabaseHistoryTest;
...
By using distinct topic names in the tests, the same Kafka instance can
be used for all test methods. This brings down execution time of this
test from ~40sec to ~8sec.
2020-06-09 10:15:47 +02:00
Chris Cranford
c3b36c2417
DBZ-2149 Log begin/end of schema recovery
2020-06-09 10:15:47 +02:00
Gunnar Morling
1fdad23087
Revert "DBZ-2166 Handling error condition where schema field can not be found in"
...
This reverts commit 283740f37e
.
2020-06-08 20:12:50 +02:00
Dave Cumberland
4254e29667
DBZ-2152 Adding fix annotations to new tests
2020-06-08 13:57:35 -04:00
Dave Cumberland
a17e4cbad0
DBZ-2152 Adding another unit test with the key mapping definied
2020-06-08 13:57:35 -04:00
Dave Cumberland
223812def3
DBZ-2152 Updating the EventRouter to allow key types other than String
2020-06-08 13:57:35 -04:00
Fabian Aussems
283740f37e
DBZ-2166 Handling error condition where schema field can not be found in
...
TableSchemaBuilder
2020-06-08 12:55:16 +02:00
Gunnar Morling
1b62036cf4
DBZ-2150 Awaiting coordinator shutdown in embedded engine
2020-06-08 10:26:37 +02:00
Gunnar Morling
c54e377dd2
DBZ-1814 Exporting Hex and Base64 representations as string
2020-06-05 12:13:29 +02:00
Gunnar Morling
023064eedb
DBZ-1814 Renaming "raw" to "bytes"; also clarifying that this notion isn't Java-specific
2020-06-05 12:13:29 +02:00
Jeremy Finzel
5db614da21
DBZ-1814 added new binary handling mode option
2020-06-05 12:13:29 +02:00
Jiri Pechanec
86dac40dae
DBZ-2140 Internal configs must not be validated
2020-06-04 11:47:26 +02:00
Kewei Shang
a963e89704
DBZ-2136 Update ChangeEventQueue Javadoc by fixing typo
2020-06-02 09:38:12 +02:00
Jiri Pechanec
8e7f3d14ca
DBZ-1312 Omit constatnt db name in column and key mappers
2020-06-01 12:43:45 +02:00
Jiri Pechanec
20b45f84f3
DBZ-1657 Remove duplicate option definitions
2020-05-22 12:07:29 +02:00
Gunnar Morling
ed838fb2d4
DBZ-2089 Typo fix
2020-05-22 08:36:57 +02:00
Jiri Pechanec
84851297cb
DBZ-2089 JMX no longer mandatory
2020-05-22 08:36:57 +02:00
Debezium Builder
dc25c379e5
[maven-release-plugin] prepare for next development iteration
2020-05-19 03:40:25 -04:00
Debezium Builder
ad060181b9
[maven-release-plugin] prepare release v1.2.0.Beta2
2020-05-19 03:40:25 -04:00
Jiri Pechanec
e676d715c3
DBZ-2085 Apply custom converters to default values
2020-05-18 17:27:59 +02:00
Jiri Pechanec
c85546d611
DBZ-2074 Expose topic and headers in scripting SMTs
2020-05-15 11:13:07 +02:00