Commit Graph

3635 Commits

Author SHA1 Message Date
jcechace
0bd664e469 DBZ-2560 Extending KC image build script with ability to add extra libraries 2020-09-21 13:57:49 +02:00
cooksey
ba074abe4c DBZ-2550 Prevent duplicate events using catch up streaming
Normally when a connector gracefully shuts down, the connect framework attempts to commit offsets so the latest committed state gets acked on the replication stream.  While the connector is running, the framework periodically commits offsets. Debezium does not manage triggering an offset commit.  When the catch up streaming phase ends, there may be uncommitted state and the connector is unable to determine when the next commit will occur because the commit timing is externally managed. If a commit is not triggered between the end of the catch up streaming phase and the normal streaming phase after the snapshot, the connector may produce some duplicated messages.

Although the replication stream may be out of date, the internal OffsetContext is aware of the latest committed offset. When the snapshot phase recreates a new offset after catch up streaming, the previous offset has access to the latest state. Use the previous offset to forward known state to the new offset.
2020-09-21 09:28:15 +02:00
Jiri Pechanec
cc4dbdcc9c DBZ-2549 Reflect separate package in release pipeline 2020-09-21 07:43:58 +02:00
Tova Cohen
2e69b50ee1 DBZ-2554 CloudEvents doc needed the same format updates 2020-09-18 09:46:34 -04:00
Tova Cohen
c98476eb7a DBZ-2554 Distinguish titles and IDs with "Debezium" 2020-09-18 09:46:34 -04:00
Tova Cohen
e95d5467b0 DBZ-2554 Edit topic routing/event flattening SMTs format/style/annotations/conditions 2020-09-18 09:46:34 -04:00
Gunnar Morling
134c80b562 DBZ-78 Misc. fixes 2020-09-18 12:00:05 +02:00
rkerner
e9a4bf4126 DBZ-78 added fallback and default value behavior documentation 2020-09-18 11:56:01 +02:00
rkerner
18216fb9a5 DBZ-78 applied PR feedback: add CRD docs for downstream 2020-09-18 11:56:01 +02:00
rkerner
52dec438d7 DBZ-78 applied PR feedback 2020-09-18 11:56:01 +02:00
rkerner
5e62f307d0 DBZ-78 add Debezium auto-creation topics configuration documentation 2020-09-18 11:56:01 +02:00
Gunnar Morling
eddee904ba DBZ-2549 Documentation update 2020-09-18 11:17:17 +02:00
Gunnar Morling
a7f4dffc3b DBZ-2549 Documentation update 2020-09-18 11:17:17 +02:00
Jiri Pechanec
73c74189fa DBZ-2549 Move scripting code into a separate module 2020-09-18 11:17:17 +02:00
Debezium Builder
837669890a [maven-release-plugin] prepare for next development iteration 2020-09-16 12:18:08 -04:00
Debezium Builder
086dbd960b [maven-release-plugin] prepare release v1.3.0.Beta2 2020-09-16 12:18:08 -04:00
Chris Cranford
2a7caee77c [release] Fix changelog for 1.3.0.Beta2 2020-09-16 11:59:46 -04:00
Chris Cranford
9097e6b092 [release] Add 1.3 to release script 2020-09-16 11:59:46 -04:00
Chris Cranford
db2ccf7f2e [release] Changelog for 1.3.0.Beta2 2020-09-16 11:59:46 -04:00
Chris Cranford
a67f76a93b [release] Updated copyright 2020-09-16 11:59:46 -04:00
Jiri Pechanec
eed321d485
DBZ-2461 LSN must not be flushed after connection close 2020-09-16 11:40:52 +02:00
Ganesh Ramasubramanian
6ca025b1bb DBZ-2504: Add support for field and header prefix
Added support for field and header prefix.
Updated the documentation to include the options
 'add.fields.prefix' and 'add.headers.prefix'.
2020-09-16 11:31:05 +02:00
Chris Cranford
4ddb1d21c8 DBZ-137 Suggested documentation updates 2020-09-16 11:24:53 +02:00
Chris Cranford
70fbcc4757 DBZ-137 Add commit_scn and log miner metrics 2020-09-16 11:24:53 +02:00
Chris Cranford
6c9fed571f DBZ-137 Add doc section about database connection adapter 2020-09-16 11:24:53 +02:00
Jiri Pechanec
4ccbfefae1 DBZ-2522 Add contributor to copyright 2020-09-16 07:54:46 +02:00
James Gormley
8ef7c7d1f3 DBZ-2522 Updating sql server connector docs for column.include.list 2020-09-16 07:54:46 +02:00
James Gormley
6d0d60a326 DBZ-2522 Implementing column.include.list for sql server 2020-09-16 07:54:46 +02:00
James Gormley
45e44a2a1b DBZ-2522 Using updated terminology in test cases 2020-09-16 07:54:46 +02:00
James Gormley
b658375a3d DBZ-2522 Fixing EventProcessingFailureHandlingIT 2020-09-16 07:54:46 +02:00
James Gormley
5d1ef9a03d DBZ-2522 Fixing index out of bounds for columns excluded from cdc tables 2020-09-16 07:54:46 +02:00
Bob Roldan
c621777113 DBZ-2539 Conditionalize :toc: entry to render in upstream doc only. 2020-09-15 13:31:20 -04:00
Bob Roldan
240ec966d5 DBZ-2539 Insert file annotation to identify component for downstream doc 2020-09-15 13:31:20 -04:00
Bob Roldan
82e917b9c1 DBZ-2519 update example to introduce SMT classname as in msg filter doc 2020-09-15 09:44:09 -04:00
Bob Roldan
80d1422a69 DBZ-2519 Edit content-based routing doc for consistency 2020-09-15 09:44:09 -04:00
Bob Roldan
7521390de0 DBZ-2460 Incorporate Engineering comments on PR. 2020-09-15 09:35:30 -04:00
Bob Roldan
475628db74 DBZ-2460 Minor rewording 2020-09-15 09:35:30 -04:00
Bob Roldan
29fe29e113 DBZ-2460 Update info on evaluating Booleans. Convert var list to table 2020-09-15 09:35:30 -04:00
Bob Roldan
0904d9e2d1 DBZ-2460 Enhance JSR223 note; convert bulleted list to table, etc 2020-09-15 09:35:30 -04:00
Bob Roldan
9b2456b91e DBZ-2460 further editorial updates 2020-09-15 09:35:30 -04:00
Bob Roldan
3e277afbfb DBZ-2460 first edit pass 2020-09-15 09:35:30 -04:00
Aaron Brady
9468b1f074 DBZ-2466 Start connector when using Oracle MySQL, so afterEach can stop it 2020-09-15 11:43:49 +02:00
Aaron Brady
a5b30ffa78 DBZ-2466 Address Backup Lock review feedback 2020-09-15 11:43:49 +02:00
Aaron Brady
9057896c89 DBZ-2466 Percona Backup Locks option for snapshots 2020-09-15 11:43:49 +02:00
Aaron Brady
304ad4998c DBZ-2466 Add Percona Server profile to assembly 2020-09-15 11:43:49 +02:00
rkerner
011c3be948 DBZ-2527 Add missing ListOfRegex validator to all regex list fields and fix dependent fields to not include @Deprecated whitelist/blacklist fields 2020-09-15 10:05:04 +02:00
rkerner
385a873172 DBZ-2526 add DisplayName for MASK and TRUNCATE COLUMN fields 2020-09-15 10:05:04 +02:00
rkerner
5c5fb5dc2c DBZ-2516 add config fields for datatype.propagate.source.type and column.propagate.source.type properties 2020-09-15 10:05:04 +02:00
Gunnar Morling
73ba6d42a0 DBZ-137 Docs update 2020-09-14 12:33:43 +02:00
Chris Cranford
5694a27de1 DBZ-137 Documentation update 2020-09-14 12:33:43 +02:00