Commit Graph

1124 Commits

Author SHA1 Message Date
Hossein Torabi
ba048cf47d DBZ-3403 Semi Optimisation on MongoDB and MySQL connector for skipped.operations 2021-04-14 11:14:07 +02:00
Hossein Torabi
5d6705c3fe DBZ-2699 implement skipped.operation in core 2021-04-08 11:59:08 +02:00
Debezium Builder
8a88ba9b45 [maven-release-plugin] prepare for next development iteration 2021-04-08 04:04:00 -04:00
Debezium Builder
74152afe74 [maven-release-plugin] prepare release v1.5.0.Final 2021-04-08 04:04:00 -04:00
Chris Egerton
67f044ff87 DBZ-3371: Adjust scale of fallback BigDecimal values to match column schema 2021-04-06 10:35:23 +02:00
Jiri Pechanec
f50c3e8222 DBZ-3361 Prefer DDL over logical schema for MySQL 2021-03-30 11:35:12 +02:00
Chris Cranford
776180a0df DBZ-3347 Emit schema change events with correct source partition 2021-03-30 07:26:06 +02:00
Jiri Pechanec
a02e530c80 DBZ-2911 Use sequence only when available 2021-03-26 09:19:17 +01:00
JLDLaughlin
06b0475f17 DBZ-2911 Add additional LSN information to all sources
Adds a new "sequence" field to all sources via the AbstractSourceInfo
class. Specifically, this field is required to deduplicate records
from PostgreSQL sources in O(1) time. The sequence field is a stringified
list of metadata. For PostgreSQL sources, this will include the last
committed LSN and the current LSN. For other sources, this will be NULL
until implemented. A new integration test was added for PostgreSQL.
2021-03-26 09:19:17 +01:00
Debezium Builder
6b4d88bb51 [maven-release-plugin] prepare for next development iteration 2021-03-24 06:35:02 -04:00
Debezium Builder
c578715233 [maven-release-plugin] prepare release v1.5.0.CR1 2021-03-24 06:35:02 -04:00
Gunnar Morling
c7b7d5303a DBZ-3218 Avoiding duplicated validation (again ;) 2021-03-19 15:04:20 +01:00
Gunnar Morling
178140ae9c DBZ-3218 Removing superfluous method parameter 2021-03-19 15:04:20 +01:00
Anisha Mohanty
c761d92b2a DBZ-3218 Add separate method for logging warnings 2021-03-19 15:04:20 +01:00
Anisha Mohanty
6856dd9a80 DBZ-3218 Handle duplicate warnings for deprecated options 2021-03-19 15:04:20 +01:00
Jiri Pechanec
f7b6750ce3 DBZ-2511 Fix tests 2021-03-18 14:42:34 +01:00
Jiri Pechanec
495be2659b DBZ-2511 Dedicated logger for schema name adjuster 2021-03-18 14:42:34 +01:00
Jiri Pechanec
11e4880018 DBZ-3309 Revert to MySQL specific default snapshot fetch size 2021-03-18 11:08:26 +01:00
Gunnar Morling
647076ea69 DBZ-2957 Misc. fixes;
* Simplifying reg exp
* Adding leniency towards whitespace
* Reporting *all* problems including correct count
* Fixing incorrect actual/expected order
2021-03-17 14:04:16 +01:00
Anisha Mohanty
2e5a5ce001 DBZ-2957 Handle message.key.columns regex validation time complexity 2021-03-17 14:04:16 +01:00
Gunnar Morling
1f02d25738 DBZ-3271 Removing unused methods 2021-03-17 09:54:25 +01:00
Gunnar Morling
911b5e9a3b DBZ-3271 Unifying filter handling across relational connectors 2021-03-17 09:54:25 +01:00
Jiri Pechanec
f2eb868cde DBZ-2145 Provide Debezium BOM 2021-03-17 07:54:12 +01:00
Debezium Builder
f57057421b [maven-release-plugin] prepare for next development iteration 2021-03-12 04:38:17 -05:00
Debezium Builder
b57293013a [maven-release-plugin] prepare release v1.5.0.Beta2 2021-03-12 04:38:16 -05:00
Jiri Pechanec
123d9ef4eb DBZ-3213 Refactor Maven structure to deploy support modules 2021-03-11 16:14:15 +01:00
Gunnar Morling
4d61c57206 DBZ-3157 Adding test 2021-03-11 16:10:08 +01:00
Vladimir Osin
88f751ad3a DBZ-3157 Add event_serial_no to CloudEvent Id 2021-03-11 16:10:08 +01:00
Jiri Pechanec
c8eec4995b DBZ-3141 Changes after review 2021-03-10 19:12:34 +01:00
Jiri Pechanec
f23421920b DBZ-2919 Remove race condition 2021-03-10 19:12:34 +01:00
Jiri Pechanec
de0d540a9f DBZ-3141 Change schema signal for Oracle 2021-03-10 19:12:34 +01:00
Jiri Pechanec
15c7b11a1f DBZ-3141 Change schema signal tested for Postgres 2021-03-10 19:12:34 +01:00
Jiri Pechanec
ff7cb44801 DBZ-3141 Offsets are propagated to signal actions 2021-03-10 19:12:34 +01:00
Jiri Pechanec
7bcf9338a0 DBZ-3141 Added unit tests 2021-03-10 19:12:34 +01:00
Jiri Pechanec
7fdad2ed49 DBZ-3141 Support for signalling table 2021-03-10 19:12:34 +01:00
Anisha Mohanty
323b23498e DBZ-3084 Handle wrong warnings about deprecated options 2021-03-09 07:10:45 +01:00
Kewei Shang
158b616bfe DBZ-3199 Add support for SET column to Vitess Connector 2021-02-26 21:23:12 +01:00
Debezium Builder
b4c260da50 [maven-release-plugin] prepare for next development iteration 2021-02-23 14:04:01 -05:00
Debezium Builder
123a711a6d [maven-release-plugin] prepare release v1.5.0.Beta1 2021-02-23 14:04:01 -05:00
Chris Cranford
7e34c918e2 DBZ-3034 Add validate methods to CommonConnectorConfig 2021-02-23 10:28:17 +01:00
Chris Cranford
85f7688527 DBZ-3151 Add filter hook for table unique index resolution 2021-02-22 15:19:25 +01:00
Kewei Shang
92659d6a27 DBZ-3124 Overload builder() and schema() to take multiple values 2021-02-19 12:26:44 +01:00
Jiri Pechanec
1edad510ff DBZ-2614 Support for infinite timestamps 2021-02-19 10:44:06 +01:00
Gunnar Morling
440364e781 DBZ-2984 Removing unused field 2021-02-16 14:49:35 +01:00
Jiri Pechanec
8c0b87c667 DBZ-2984 Guess struct name only when unavailable 2021-02-16 14:49:35 +01:00
Gunnar Morling
184d1a2399 DBZ-3056 Misc. clean-up 2021-02-16 12:45:09 +01:00
Yilong Chang
a0b3704832 DBZ-3056 Validate all fields in subclasses. 2021-02-16 12:34:06 +01:00
Yilong Chang
76c7e0d426 DBZ-3056 Extract basic valdiation in parent class. 2021-02-16 12:34:06 +01:00
Gunnar Morling
cb2b2fc07a DBZ-3099 Doc fix 2021-02-16 10:11:23 +01:00
Jiri Pechanec
1c9f7a1c1c DBZ-3099 Document non-volatile fields accessed by a single thread 2021-02-16 10:02:17 +01:00
Jiri Pechanec
745bb6c11d DBZ-3099 Address review comments; extract history comparator for testability 2021-02-16 10:02:17 +01:00
Jiri Pechanec
cb1476ce18 DBZ-3099 Address review comments 2021-02-16 10:02:17 +01:00
Jiri Pechanec
21b15facb2 DBZ-3099 Snapshot buffering to handle config corner cases 2021-02-16 10:02:17 +01:00
Jiri Pechanec
6a9ca7d544 DBZ-3099 Fix schema recovery; Add table level locking 2021-02-16 10:02:17 +01:00
Jiri Pechanec
08e4988c72 DBZ-3093 Unify snapshot and streaming parsing 2021-02-16 10:02:17 +01:00
Jiri Pechanec
6607310f13 DBZ-3093 Separate parsing and change applying phase in MySQL schema 2021-02-16 10:02:17 +01:00
ani-sha
5f8a22a5df DBZ-2858 Make field descriptions consistent for time values 2021-02-16 09:55:35 +01:00
Gunnar Morling
b5b73b4409 DBZ-3057 Formatting fix 2021-02-16 09:40:13 +01:00
Chris Cranford
09de77cf7d DBZ-3057 Add some TableId helper methods 2021-02-16 09:40:13 +01:00
harveyyue
3a4bcb104b DBZ-3096 Getting RejectedExecutionException when checking topic settings from KafkaDatabaseHistory.checkTopicSettings 2021-02-12 08:37:16 +01:00
Kevin Pullin
2426de849a DBZ-3074 PgOutputMessageDecoder - Fix unordered keys
Fix issue where `PgOutputMessageDecoder` does not define primary key metadata in key-order.

`PgOutputMessageDecoder.getTablePrimaryKeyColumnNamesFromDatabase()` returns a `Set`, leading to unordered
primary keys in the metadata. Switch the logic to call `JdbcConnection.readPrimaryKeyNames()` which
_does_ correctly return an ordered primary key list.
2021-02-10 10:58:26 +01:00
Yilong Chang
0e63a5a651 DBZ-3075 Close Kafka Database History producer with a timeout of 30s. 2021-02-10 09:08:02 +01:00
Debezium Builder
e08df0a177 [maven-release-plugin] prepare for next development iteration 2021-02-06 01:25:23 -05:00
Debezium Builder
da0411bf70 [maven-release-plugin] prepare release v1.5.0.Alpha1 2021-02-06 01:25:23 -05:00
Jiri Pechanec
f57343dc3b DBZ-1865 Use OptionaLong; add comments 2021-02-04 09:07:55 +01:00
Jiri Pechanec
2374b6b045 DBZ-1865 First draft of MySQL rewrite 2021-02-04 09:07:55 +01:00
Gunnar Morling
6c5fb793af DBZ-2575 Marking option as private 2021-02-04 08:28:55 +01:00
Martin Sillence
164ab1330c DBZ-2575 Improve performance where tables we fetch are much less than total tables 2021-02-04 08:28:49 +01:00
Naveen Kumar KR
75882f35ac DBZ-2382 Support emitting TRUNCATE events in PG11+ PGOUTPUT plugin
default - TruncateHandlingMode.SKIP
supported configs - TruncateHandlingMode.SKIP / TruncateHandlingMode.INCLUDE
2021-02-03 13:53:01 +01:00
ani-sha
d4003c9172 DBZ-2420 Define database configs under RelationalDatabaseConnectorConfig 2021-02-03 08:05:42 +01:00
Debezium Builder
237b4ac0f1 [maven-release-plugin] prepare for next development iteration 2021-01-28 07:38:55 -05:00
Debezium Builder
06b1d0cbad [maven-release-plugin] prepare release v1.4.1.Final 2021-01-28 07:38:55 -05:00
Gunnar Morling
f76d8e0869 DBZ-2938 Calling snapshot listener only after complete() hook 2021-01-19 15:16:46 +01:00
Gunnar Morling
6d441bc538 DBZ-2938 Centralizing listener handling; avoiding some empty methods 2021-01-19 15:16:46 +01:00
rkerner
d7ac0f7e6b DBZ-2572 Fix docs for message.key.columns and skipped.operations 2021-01-12 14:08:21 +01:00
Debezium Builder
3010f99c32 [maven-release-plugin] prepare for next development iteration 2021-01-07 03:38:13 -05:00
Debezium Builder
54534538d9 [maven-release-plugin] prepare release v1.4.0.Final 2021-01-07 03:38:13 -05:00
Gunnar Morling
21c3003187 DBZ-2889 Don't access payload id field if not present 2021-01-06 14:46:58 +01:00
Chris Cranford
d32e4b3720 DBZ-2849 Added test 2021-01-05 12:42:08 +01:00
Chris Cranford
8409282739 DBZ-2849 Fix value converter field lookups with sanitized field names 2021-01-05 12:42:08 +01:00
Jiri Pechanec
d178365529 DBZ-2880 Guava bumped to 30.0 2021-01-05 12:35:32 +01:00
Jiri Pechanec
58713599ba DBZ-2167 Remove legacy parser completely; tests depending on parser move to MySQL 2020-12-21 16:30:15 +01:00
Jiri Pechanec
00f9e3b261 DBZ-2167 Use legacy parser for testing purposes only 2020-12-21 16:30:15 +01:00
Debezium Builder
b1ab996c3e [maven-release-plugin] prepare for next development iteration 2020-12-16 14:11:31 -05:00
Debezium Builder
8fc864b278 [maven-release-plugin] prepare release v1.4.0.CR1 2020-12-16 14:11:31 -05:00
Debezium Builder
cf921f56b4 [maven-release-plugin] prepare for next development iteration 2020-12-10 00:27:40 -05:00
Debezium Builder
c49a377487 [maven-release-plugin] prepare release v1.4.0.Beta1 2020-12-10 00:27:39 -05:00
Jiri Pechanec
8e333d9b1d DBZ-1818 DBZ-559 Changes after review
OpenTracing not used in names
Span for event time
Span SMT embedded in outbox
Handling of missing tracing libs in Connect runtime
Aligned tracing tag names with the standard ones
2020-12-08 16:09:09 +01:00
Jiri Pechanec
318340078b DBZ-1818 DBZ-559 Integration with OpenTracing 2020-12-08 16:09:09 +01:00
Jiri Pechanec
b7fede7d53 DBZ-735 Verify Avro serialization 2020-11-27 17:33:23 +01:00
Debezium Builder
4535549dcd [maven-release-plugin] prepare for next development iteration 2020-11-16 15:18:43 -05:00
Debezium Builder
3c70c5f7d5 [maven-release-plugin] prepare release v1.4.0.Alpha2 2020-11-16 15:18:43 -05:00
vanhoale
17a424d9ac DBZ-2662 Handle ChangeEventQueue by the size in bytes 2020-11-16 08:39:33 +01:00
rkerner
a611979268 DBZ-2580 apply PR feedback 2020-11-12 07:42:48 +01:00
rkerner
9c5db0cd2f DBZ-2580 Fix "The primary key cannot reference a non-existant column" error from MySQL DDL parser when CREATE TABLE statement starts with a primary key definition like "CREATE TABLE Products (PRIMARY KEY (id), ..." where the referenced primary key column is not yet defined 2020-11-12 07:42:48 +01:00
Chris Cranford
aea26a2949 DBZ-2742 Migrate initial.sync.max.threads to snapshot.max.threads 2020-11-11 09:08:05 +01:00
Ganesh Ramasubramanian
4716194ff9 DBZ-2606: Map new name for the fields and headers
Added ability to map new name for the fields and headers.
The "new name" mapping is completely optional and we will not
be affecting the existing functionality.

The optional "new name" is case-sensitive.
2020-11-04 08:57:16 +01:00
ani-sha
fd6c8f9502 DBZ-2690 get oracle specific changes into debezium-core 2020-11-04 07:21:10 +01:00
rkerner
c274c2b39b DBZ-2617 refactor to proper HeartbeatErrorHandler injection 2020-11-03 07:23:02 +01:00
rkerner
2a3f44ebe5 DBZ-2617 failing heartbeats should put Postgres connector into FAILED state to allow proper shutdown of Postgres when shutdown was triggered with shutdown mode "fast" 2020-11-03 07:23:02 +01:00