Commit Graph

1073 Commits

Author SHA1 Message Date
Debezium Builder
26883dad92 [maven-release-plugin] prepare for next development iteration 2020-10-22 03:45:59 -04:00
Debezium Builder
9adbc2d67e [maven-release-plugin] prepare release v1.4.0.Alpha1 2020-10-22 03:45:59 -04:00
Jiri Pechanec
2cded66d30 DBZ-2632 Minor improvements 2020-10-20 16:28:23 +02:00
Arik Cohen
2e8c43bd60 DBZ-2632: Forcibly abort the JDBC connection if failed to close it gracefully. 2020-10-20 16:28:23 +02:00
Jiri Pechanec
ee5353ef6f DBZ-2653 Process commits even with tx monitoring 2020-10-20 10:42:10 +02:00
Kaushik Iyer
7d9e8fdc46 DBZ-2456 fixes after rebase 2020-10-16 10:13:48 +02:00
Kaushik Iyer
f752b3ab39 DBZ-2456 updated common connector config with generalized naming 2020-10-16 10:13:48 +02:00
Kaushik Iyer
e2e6dc0932 DBZ-2456 selective snapshot implementation 2020-10-16 10:13:48 +02:00
Debezium Builder
95b8ed3877 [maven-release-plugin] prepare for next development iteration 2020-10-01 09:08:51 -04:00
Debezium Builder
19ad73beb2 [maven-release-plugin] prepare release v1.3.0.Final 2020-10-01 09:08:50 -04:00
Gunnar Morling
b792ac3274 DBZ-2495 Misc. fixes 2020-09-30 12:13:45 +02:00
Gunnar Morling
6b5a3100e9 DBZ-2495 Misc. clean-up 2020-09-30 12:13:45 +02:00
James Gormley
9e780ac481 DBZ-2495 Updating per PR comments 2020-09-30 12:13:45 +02:00
James Gormley
1b845535ab DBZ-2495 Updating tables to utilize Table.editor for creating new tables 2020-09-30 12:13:45 +02:00
James Gormley
c7b53e1992 DBZ-2495 Updating docs and comments 2020-09-30 12:13:45 +02:00
James Gormley
0d46ec47da DBZ-2495 Updating to handle captured_column_list without specify column.exclude.list 2020-09-30 12:13:45 +02:00
Gunnar Morling
c1b057f792 DBZ-2588 Introducing dedicated functional interface for key/value generators;
Also keys are always KC Struct instead of generic j.l.Object.
2020-09-25 06:01:59 +02:00
Debezium Builder
b146b69154 [maven-release-plugin] prepare for next development iteration 2020-09-24 04:56:42 -04:00
Debezium Builder
c51cfe8eb9 [maven-release-plugin] prepare release v1.3.0.CR1 2020-09-24 04:56:41 -04:00
Chris Cranford
df41cd8bfe DBZ-2396 Suggested changes 2020-09-24 07:41:36 +02:00
Chris Cranford
0db4997102 DBZ-2396 Support Converter delegation in ByteBufferConverter
In the even that the value supplied to the ByteBufferConverter
is not supported, e.g. not BYTES, the converter should then
support delegation to a configured converter.
2020-09-24 07:41:36 +02:00
Jiri Pechanec
73c74189fa DBZ-2549 Move scripting code into a separate module 2020-09-18 11:17:17 +02:00
Debezium Builder
837669890a [maven-release-plugin] prepare for next development iteration 2020-09-16 12:18:08 -04:00
Debezium Builder
086dbd960b [maven-release-plugin] prepare release v1.3.0.Beta2 2020-09-16 12:18:08 -04:00
Ganesh Ramasubramanian
6ca025b1bb DBZ-2504: Add support for field and header prefix
Added support for field and header prefix.
Updated the documentation to include the options
 'add.fields.prefix' and 'add.headers.prefix'.
2020-09-16 11:31:05 +02:00
rkerner
011c3be948 DBZ-2527 Add missing ListOfRegex validator to all regex list fields and fix dependent fields to not include @Deprecated whitelist/blacklist fields 2020-09-15 10:05:04 +02:00
rkerner
385a873172 DBZ-2526 add DisplayName for MASK and TRUNCATE COLUMN fields 2020-09-15 10:05:04 +02:00
rkerner
5c5fb5dc2c DBZ-2516 add config fields for datatype.propagate.source.type and column.propagate.source.type properties 2020-09-15 10:05:04 +02:00
Gunnar Morling
73ba6d42a0 DBZ-137 Docs update 2020-09-14 12:33:43 +02:00
Chris Cranford
a686c1c875 DBZ-137 Suggested changes 2020-09-14 12:33:43 +02:00
Chris Cranford
5ca36a84ad DBZ-137 Reimagine stream.fetch.size as query.fetch.size 2020-09-14 12:33:43 +02:00
Chris Cranford
8584831f51 DBZ-137 Use Duration for CONNECTION_TIMEOUT_MS 2020-09-14 12:33:43 +02:00
Chris Cranford
43440ca76f DBZ-137 Remove snapshot.skip.locks for SQL Server 2020-09-14 12:33:43 +02:00
Chris Cranford
72d59348ed DBZ-137 Fix rebase 2020-09-14 12:33:43 +02:00
AndreyIg
77662a0dde DBZ-137 contribution to the core, Oracle related 2020-09-14 12:33:43 +02:00
cooksey
8bcbdb639d DBZ-2094 Allow Postgres snapshotter to set streaming start on resume
When a connector resumes after previously streaming and takes a
snapshot, through a new method on the snapshotter interface,
shouldStreamEventsStartingFromSnapshot, can choose whether
to resume streaming from the last known streaming position or the
beginning of the snapshot. This is helpful for snapshotters that
may not want to resnapshot every table in the whitelist/blacklist
but not miss event on the tables that are skipped.
2020-09-09 10:54:11 +02:00
Grzegorz Kołakowski
c74579c1df DBZ-1068 Forward port the work by @grzegorz8 to master
Co-authored-by: GuyIEX <gpascarella@ironeaglex.com>
2020-09-04 12:06:57 +02:00
Nathan Mills
6e915b7cfd
DBZ-2473 Decrement remaining for skipped tables 2020-09-04 07:50:34 +02:00
Debezium Builder
9979db648f [maven-release-plugin] prepare for next development iteration 2020-08-28 05:04:26 -04:00
Debezium Builder
5c58b23756 [maven-release-plugin] prepare release v1.3.0.Beta1 2020-08-28 05:04:26 -04:00
Jiri Pechanec
d9f616caed DBZ-2303 Set source when new table detected 2020-08-24 13:08:09 +02:00
rkerner
2c5c093587 DBZ-2171 avoid divisive language 2020-08-21 09:29:04 +02:00
Gunnar Morling
4c3cf9f93e DBZ-2427 Logging inability to unregister metrics MBean as WARN 2020-08-13 14:05:11 -04:00
Gunnar Morling
dac7f3998c DBZ-2412 Housekeeping;
* Static logger
* Protecting against empty topic name
* Removing unused logger
2020-08-13 13:53:50 -04:00
Gunnar Morling
19ae707a18 DBZ-2412 Ignoring non change events in logical router 2020-08-13 13:53:50 -04:00
Gunnar Morling
44efd1bc6b DBZ-2423 Propagating port default values 2020-08-12 13:50:42 +02:00
Gunnar Morling
26b51e02cb DBZ-2418 Misc. clean-up 2020-08-12 09:28:35 +02:00
Jiri Pechanec
0b95df28d5 DBZ-2411 Use Scala 2.12 2020-08-11 09:20:53 +02:00
Jiri Pechanec
391d086529 DBZ-2411 Upgrade to Kafka 2.6.0 2020-08-11 09:20:53 +02:00
Debezium Builder
c4c50f5fc6 [maven-release-plugin] prepare for next development iteration 2020-08-06 00:46:18 -04:00
Debezium Builder
94cbeb95fa [maven-release-plugin] prepare release v1.3.0.Alpha1 2020-08-06 00:46:17 -04:00
rkerner
cad6cf62d3 DBZ-2315 fix BIT VARYING handling in Postgres connector
* removed little endian padding for BIT types in JdbcValueConverters (only used for Postgres yet)
* removed legacy format handling fot BIT related types in JdbcValueConverters#convertBits which were leftovers from PR #1408
* removed unnecessary zero-ing of a newly created byte array to improve performance for huge byte arrays
* updated Postgres connector docs
2020-08-03 14:58:31 +02:00
Cory Harper
724fc7ac98 DBZ-2362 add configurable restart wait time 2020-07-30 18:08:26 +02:00
Jiri Pechanec
5fcb221993 DBZ-1491 Database connection provides default value 2020-07-24 09:07:09 +02:00
Jiri Pechanec
45a6f9bef1 DBZ-1491 Refactor after review 2020-07-24 09:07:09 +02:00
Grzegorz Kołakowski
825ffe3848 DBZ-1491 Add SqlServerDefaultValueConverter class 2020-07-24 09:07:07 +02:00
Grzegorz Kołakowski
234341a471 DBZ-1491 Parse temporal values using queries on database
* Add tests for altering column default value
 * Add end-to-end test
2020-07-24 09:06:30 +02:00
Grzegorz Kołakowski
a8ea5e2256 DBZ-1491 Extract default values from schema
The goal is to enrich schema metadata with column default value.
2020-07-24 09:05:58 +02:00
Gunnar Morling
3ebfbcefdb DBZ-2329 Moving isNewTransaction() to SQL Server implementation 2020-07-24 07:57:50 +02:00
Lukasz Korzeniowski
1ada5c4b23 DBZ-2329 Do not skip events after restart when stream is not monotonic 2020-07-21 14:21:29 +02:00
Debezium Builder
e1ab38de84 [maven-release-plugin] prepare for next development iteration 2020-07-16 04:40:04 -04:00
Debezium Builder
69cf375faa [maven-release-plugin] prepare release v1.2.1.Final 2020-07-16 04:40:04 -04:00
Jos Huiting
da97be7842 DBZ-2307 - Improve logging when primary keys are missing 2020-07-15 11:18:48 -04:00
Omar Al-Safi
4d2467ca20 DBZ-2302: Allow OffsetCommitPolicy to be initiated with Properties object 2020-07-09 08:38:26 +02:00
Randall Hauch
5860812a63
DBZ-2275 Updated default database history DML filter default to handle additional RDS statements;
* Added two more regular expressions to the default database history DML filter. These are intended to catch and filter out `INSERT into mysql.rds_monitor…` statements that RDS may write in the binlog
* Added a new unit test that checks that the regular expressions do catch some example statements
2020-06-25 20:57:49 +02:00
Jiri Pechanec
63a2f6a0ae DBZ-2255 Document JSON parsing in groovy-based SMTs 2020-06-25 11:08:59 +02:00
Debezium Builder
c0d30b87d9 [maven-release-plugin] prepare for next development iteration 2020-06-24 18:03:11 +02:00
Debezium Builder
0e2ac78301 [maven-release-plugin] prepare release v1.2.0.Final 2020-06-24 18:03:11 +02:00
Jiri Pechanec
87d93d44fd DBZ-2221 Wait for keepalive thread initialization 2020-06-24 09:14:42 +02:00
Gunnar Morling
b80f064131 DBZ-2250 Allowing to use CloudEventsConverter with Avro via Apicurio 2020-06-22 14:41:10 +02:00
Debezium Builder
b03a4d0e50 [maven-release-plugin] prepare for next development iteration 2020-06-19 00:39:01 -04:00
Debezium Builder
93e60258d5 [maven-release-plugin] prepare release v1.2.0.CR2 2020-06-19 00:39:00 -04:00
Gunnar Morling
f2d13238aa DBZ-2228 Using unique names for admin clients 2020-06-18 14:08:56 +02:00
Debezium Builder
65c39e9f11 [maven-release-plugin] prepare for next development iteration 2020-06-11 02:00:20 -04:00
Debezium Builder
bce332c900 [maven-release-plugin] prepare release v1.2.0.CR1 2020-06-11 02:00:19 -04:00
Chris Cranford
ac0d9ede8a DBZ-2152 Fix test failure 2020-06-09 17:20:51 +02:00
Balázs Németh
0853940f3d
DBZ-2169 Fix for misuse of ConfigurationDefaults.RETURN_CONTROL_INTERVAL;
* Implemented Temporals.min()
* Fixed pollInterval calculation based on the documentation
2020-06-09 10:24:50 +02:00
Gunnar Morling
bfaadcf2dd DBZ-2149 Starting/stopping Kafka just once in KafkaDatabaseHistoryTest;
By using distinct topic names in the tests, the same Kafka instance can
be used for all test methods. This brings down execution time of this
test from ~40sec to ~8sec.
2020-06-09 10:15:47 +02:00
Chris Cranford
c3b36c2417 DBZ-2149 Log begin/end of schema recovery 2020-06-09 10:15:47 +02:00
Gunnar Morling
1fdad23087 Revert "DBZ-2166 Handling error condition where schema field can not be found in"
This reverts commit 283740f37e.
2020-06-08 20:12:50 +02:00
Dave Cumberland
4254e29667 DBZ-2152 Adding fix annotations to new tests 2020-06-08 13:57:35 -04:00
Dave Cumberland
a17e4cbad0 DBZ-2152 Adding another unit test with the key mapping definied 2020-06-08 13:57:35 -04:00
Dave Cumberland
223812def3 DBZ-2152 Updating the EventRouter to allow key types other than String 2020-06-08 13:57:35 -04:00
Fabian Aussems
283740f37e DBZ-2166 Handling error condition where schema field can not be found in
TableSchemaBuilder
2020-06-08 12:55:16 +02:00
Gunnar Morling
1b62036cf4 DBZ-2150 Awaiting coordinator shutdown in embedded engine 2020-06-08 10:26:37 +02:00
Gunnar Morling
c54e377dd2 DBZ-1814 Exporting Hex and Base64 representations as string 2020-06-05 12:13:29 +02:00
Gunnar Morling
023064eedb DBZ-1814 Renaming "raw" to "bytes"; also clarifying that this notion isn't Java-specific 2020-06-05 12:13:29 +02:00
Jeremy Finzel
5db614da21 DBZ-1814 added new binary handling mode option 2020-06-05 12:13:29 +02:00
Jiri Pechanec
86dac40dae DBZ-2140 Internal configs must not be validated 2020-06-04 11:47:26 +02:00
Kewei Shang
a963e89704 DBZ-2136 Update ChangeEventQueue Javadoc by fixing typo 2020-06-02 09:38:12 +02:00
Jiri Pechanec
8e7f3d14ca DBZ-1312 Omit constatnt db name in column and key mappers 2020-06-01 12:43:45 +02:00
Jiri Pechanec
20b45f84f3 DBZ-1657 Remove duplicate option definitions 2020-05-22 12:07:29 +02:00
Gunnar Morling
ed838fb2d4 DBZ-2089 Typo fix 2020-05-22 08:36:57 +02:00
Jiri Pechanec
84851297cb DBZ-2089 JMX no longer mandatory 2020-05-22 08:36:57 +02:00
Debezium Builder
dc25c379e5 [maven-release-plugin] prepare for next development iteration 2020-05-19 03:40:25 -04:00
Debezium Builder
ad060181b9 [maven-release-plugin] prepare release v1.2.0.Beta2 2020-05-19 03:40:25 -04:00
Jiri Pechanec
e676d715c3 DBZ-2085 Apply custom converters to default values 2020-05-18 17:27:59 +02:00
Jiri Pechanec
c85546d611 DBZ-2074 Expose topic and headers in scripting SMTs 2020-05-15 11:13:07 +02:00
Gunnar Morling
0b42c8d8c2 DBZ-2024 Adding topic.regex option to routing and filtering SMTs 2020-05-14 08:41:40 +02:00
Jiri Pechanec
c4f975bd0a DBZ-2061 Handle change of default value 2020-05-14 08:31:27 +02:00
Gunnar Morling
4f020ca6cf DBZ-1828 Removing deprecated SMT options 2020-05-13 21:36:12 +02:00
Chris Cranford
f449550b2d DBZ-2068 Skip tests incompatible with Kafka 1.x 2020-05-13 08:22:07 +02:00
Aaron Brady
8b42016f63 DBZ-2034 Make ByLogicalTableRouter SMT unique key enforcement optional 2020-05-08 13:56:18 -04:00
Debezium Builder
8d4203279e [maven-release-plugin] prepare for next development iteration 2020-05-07 05:03:40 -04:00
Debezium Builder
a4e2c8f36c [maven-release-plugin] prepare release v1.2.0.Beta1 2020-05-07 05:03:40 -04:00
Chris Cranford
c6ba581848 DBZ-2014 Suggested changes 2020-05-05 22:13:25 +02:00
Chris Cranford
cd43c4880f DBZ-2014 Do not include outbox eventType field by default 2020-05-05 22:13:25 +02:00
Chris Cranford
3c9ad5699b DBZ-2036 Change logging from warn to debug 2020-05-05 21:58:56 +02:00
Chris Cranford
81c3a808da DBZ-1963 Suggested changes 2020-05-05 08:45:58 +02:00
Chris Cranford
a1cb06299c DBZ-1963 Emit outbox events with value schema names 2020-05-05 08:45:58 +02:00
Jiri Pechanec
1b3fba4b8c DBZ-1904 Add drop table support 2020-05-04 12:31:05 +02:00
Jiri Pechanec
b9fa634968 DBZ-1904 Rename queueing methods 2020-05-04 12:31:05 +02:00
Jiri Pechanec
7f23d74d90 DBZ-1904 Add config param into a group 2020-05-04 12:31:05 +02:00
Jiri Pechanec
e350d74b17 DBZ-1904 Public facing schema history topic 2020-05-04 12:31:05 +02:00
Gunnar Morling
2310d6c792 DBZ-1969 Fixing incorrect assertion and utility which concealed this 2020-04-29 13:06:43 -04:00
Braden Groom
0b429ce880 DBZ-1969 Add support for Postgres enum arrays 2020-04-29 13:06:43 -04:00
Chris Cranford
230ddaebdc DBZ-1953 Generalized SkipWhenDatabaseVersion functionality 2020-04-29 12:27:38 +02:00
Gunnar Morling
5fb3b7f2a1 DBZ-2027 Removing some unused code 2020-04-29 09:56:59 +02:00
Gunnar Morling
e9632c2227 DBZ-1972 Formatting 2020-04-27 17:54:35 +02:00
Gunnar Morling
8b2448440b Revert "DBZ-2025 Reduce capture job poll interval"
This reverts commit 131063fb2be38f865019e5dcff74b6d78299f6d2; it prove
unstable on CI.
2020-04-27 16:38:01 +02:00
Jiri Pechanec
131063fb2b DBZ-2025 Reduce capture job poll interval 2020-04-27 13:56:29 +02:00
JanHendrikDolling
33d83731af DBZ-1972 move static Filed methods as static Filed constant to RelationalDatabaseConnectorConfig instead. 2020-04-27 12:31:19 +02:00
Gunnar Morling
f9bfe26754 DBZ-1753 Typo fix 2020-04-27 11:27:36 +02:00
Chris Cranford
f974f9f4ab DBZ-1753 Refactor common code between SQL Server and DB2 2020-04-24 13:45:06 -04:00
Jiri Pechanec
7e37237b52 DBZ-2000 Change expression param name; add docs 2020-04-24 15:44:52 +02:00
Gunnar Morling
90d2e24788 DBZ-2000 Misc. clean-up 2020-04-24 15:44:52 +02:00
Jiri Pechanec
3ceb6ced3e DBZ-2000 Content based router SMT 2020-04-24 15:44:52 +02:00
Gunnar Morling
03ce403972 DBZ-1750 Separating config definition editor and immutable config definitions 2020-04-24 15:35:28 +02:00
Jiri Pechanec
37f0dbcaf8 DBZ-1750 Limit access to parent defs; dont share the parent defs 2020-04-24 15:35:28 +02:00
Jiri Pechanec
0d8d6a41c5 DBZ-1750 Validate Kafka params only for Kafka history 2020-04-24 15:35:28 +02:00
Jiri Pechanec
af90507fa6 DBZ-1750 Centralize shared config definitions 2020-04-24 15:35:28 +02:00
Jiri Pechanec
412f288e88 DBZ-1923 Detect lost database history 2020-04-24 10:22:06 +02:00
Chris Cranford
24abc23f66 DBZ-2007 Upgrade Formatter and Impsort plugins 2020-04-23 11:21:06 +02:00
Gunnar Morling
2d0da37c03 DBZ-1993 Wrapping structs for simpler references in filter SMT;
Exposing as ProxyObject allows for simple property-style access. We also can prevent write access that way.
2020-04-23 08:33:30 +02:00
Gunnar Morling
0998fc533a DBZ-1895 Completing implementation;
* Restoring original derserializers; excluding skipped events in event handlers
* Adding test
2020-04-22 15:04:36 +02:00
Gunnar Morling
dae655c78b DBZ-2006 Misc. fixes to MongoDB connector docs 2020-04-22 11:42:21 +02:00
Gunnar Morling
1a5dd5db98 DBZ-1886 Misc. clean-up 2020-04-20 17:46:03 +02:00
Ivan Klass
75f19239d1 DBZ-1886 Introduce separate storageExists() method to support manual topic creation 2020-04-20 17:46:03 +02:00
Gunnar Morling
bc762f2989 DBZ-1962 Adding option descriptions 2020-04-17 15:24:42 +02:00
Jos Huiting
33fe631674 DBZ-1962 Fix PR feedback 2020-04-17 15:24:42 +02:00
Jos Huiting
8ba21a98ef DBZ-1962 Introduce column.whitelist for the Postgres connector 2020-04-17 15:24:42 +02:00
Debezium Builder
4ca9f92575 [maven-release-plugin] prepare for next development iteration 2020-04-16 02:22:00 -04:00
Debezium Builder
e704bccdaa [maven-release-plugin] prepare release v1.2.0.Alpha1 2020-04-16 02:22:00 -04:00
Gunnar Morling
786aab37d9 DBZ-1782 JavaDoc improvements;
As per Chris's suggestions.

Co-Authored-By: Chris Cranford <ccranfor@redhat.com>
2020-04-15 17:52:35 +02:00
Gunnar Morling
0673aba092 DBZ-1782 Inverting filtering logic, so to be consistent with Stream#filter() and other similar APIs 2020-04-15 17:52:35 +02:00
Gunnar Morling
0775bb1f4f DBZ-1782 Allowing any JSR 223 implementation for filtering SMT 2020-04-15 17:52:35 +02:00
Gunnar Morling
905629973e DBZ-1968 Removing SMTs deprecated in 0.10 2020-04-15 15:11:32 +02:00
Chris Cranford
e87bfefdf7 DBZ-1859 Renamed MetadataEvent to ConnectorEvent 2020-04-15 08:04:23 +02:00
Chris Cranford
57af80afd5 DBZ-1859 Enhance MongoDB connector metrics 2020-04-15 08:04:23 +02:00
Gunnar Morling
680f1f73af DBZ-1782 Misc. fixes 2020-04-09 18:09:56 +02:00