Commit Graph

1401 Commits

Author SHA1 Message Date
Chris Cranford
09e58c5a5c DBZ-5006 Allow "/" characters in Oracle table names 2022-04-27 08:52:29 -04:00
Sergei Morozov
fd46a670ad DBZ-5033: Ignore null offsets when recovering database history 2022-04-26 11:40:21 +02:00
Sergei Morozov
e79305cf93 DBZ-5033: Update Javadocs in DatabaseHistory 2022-04-26 11:40:21 +02:00
Vojtech Juranek
da5378d095 DBZ-4949 Adjust one test to use features > Java 11
Adjust one test to use multiline strings, which is a feature added in
Java 15 and thus verify Java source level > 11 works for tests.
2022-04-22 05:53:51 +02:00
Brad Morgan
8e25c52585 DBZ-4989 Passthrough payload schema in EventRouterDelegate 2022-04-14 12:17:53 +02:00
Jiri Pechanec
053bf602af DBZ-4979 Guard against NPE for non-table related DDLs 2022-04-12 09:36:14 -04:00
Debezium Builder
4a6de64c25 [maven-release-plugin] prepare for next development iteration 2022-04-05 14:49:52 +00:00
Debezium Builder
683851fd90 [maven-release-plugin] prepare release v1.9.0.Final 2022-04-05 14:49:51 +00:00
Debezium Builder
ab7fe3b96a [maven-release-plugin] prepare for next development iteration 2022-03-25 16:45:06 +00:00
Debezium Builder
d19fc234ee [maven-release-plugin] prepare release v1.9.0.CR1 2022-03-25 16:45:06 +00:00
Sergei Morozov
010f8c9d50 DBZ-3535: Rework CloudEventsConverter 2022-03-24 09:37:05 +01:00
Sergei Morozov
854186e846 DBZ-3535: Rework ByLogicalTableRouter 2022-03-24 09:37:05 +01:00
Sergei Morozov
139b129cf5 DBZ-3535: Rework TransactionMonitor 2022-03-24 09:37:05 +01:00
Sergei Morozov
fccc31c35b DBZ-3535: Rework ConnectTableChangeSerializer 2022-03-24 09:37:05 +01:00
Sergei Morozov
b258674782 DBZ-3535: Introduce schema.name.adjustment.mode 2022-03-24 09:37:05 +01:00
Sergei Morozov
3a03ff0186 DBZ-3535: Remove unused EventDispatcher constructor parameter 2022-03-24 09:37:05 +01:00
Vojtech Juranek
0163c07800 DBZ-4786 Avoid NPE when removing column which doesn't exists 2022-03-23 07:53:06 +01:00
Vojtech Juranek
fdfc7a6647 DBZ-3762 Filter out MySQL inline comments by default 2022-03-21 16:57:43 +01:00
kate
4b2d5c29be DBZ-4878 Re-read incremental snapshot chunk on DDL event 2022-03-18 07:55:43 +01:00
harveyyue
044e454e0a DBZ-4832 Don't set truststore/keystore parameters to system variables 2022-03-15 11:25:28 +01:00
Josh Ribera
020e845f44 DBZ-4809 Add task id and partition to the logging context for
multi-partition connectors.

Co-authored-by: Sergei Morozov <morozov@tut.by>
2022-03-09 20:46:38 +01:00
Gunnar Morling
d20a6f00e7 DBZ-4801 Using JdbcConfiguration in JdbcConnection 2022-03-07 17:53:45 +01:00
Gunnar Morling
c1c322ecf0 DBZ-4801 Simplifying JdbcConnection constructors 2022-03-07 17:53:45 +01:00
harveyyue
856ab85cef DBZ-4824 Expect the null value with snapshot CapturedTables metric when skipping snapshotting 2022-03-07 12:19:13 +01:00
Debezium Builder
fece16039c [maven-release-plugin] prepare for next development iteration 2022-03-03 17:00:08 +00:00
Debezium Builder
e0d8a0d809 [maven-release-plugin] prepare release v1.9.0.Beta1 2022-03-03 17:00:08 +00:00
yingyingtang-brex
eb3f02b9c0 DBZ-4813 indicating message id and key in outbox log messages 2022-03-03 10:47:20 +01:00
Josh Ribera
fc6d55e935 DBZ-4783: Add task id label to metrics 2022-03-01 14:21:39 +01:00
Mike Kamornikov
37a67d5dcf DBZ-4783: Manage change tables per partition 2022-03-01 14:21:39 +01:00
Chris Cranford
e472a06da4 DBZ-4773 Fix formatting 2022-02-23 08:49:31 +01:00
Gunnar Morling
a144d84257 DBZ-4773 Using right constructor signature based on Kafka client version 2022-02-23 08:49:31 +01:00
Chris Cranford
9103a220b7 DBZ-4744 Correctly parse signal table name when using domain name 2022-02-22 15:27:21 +01:00
Jiri Pechanec
d13a1635d7 DBZ-3952 Store schema for tables not in schema collection list 2022-02-17 07:41:00 +01:00
Sergei Morozov
2d14b5b09f DBZ-4478: Multi-partition metrics for SQL Server 2022-02-16 11:31:38 +01:00
Sergei Morozov
6d0aa9408c DBZ-4478: Add Partition parameter to io.debezium.pipeline.source.spi.*Listener interfaces 2022-02-16 11:31:38 +01:00
Sergei Morozov
af8ea4d73d DBZ-4478: Move logging from metrics to coordinator 2022-02-16 11:31:38 +01:00
Sergei Morozov
24ff3c9eba DBZ-4478: Remove throws InterruptedException from DataChangeEventListener::onEvent 2022-02-16 11:31:38 +01:00
Jiri Pechanec
54ba2cd598 DBZ-4736 Handle string NULL default values correctly 2022-02-11 12:06:55 +01:00
harveyyue
8431a614a2 DBZ-4730 Expect plain value instead of scientific exponential notation when using decimal string mode 2022-02-10 17:40:12 +01:00
Debezium Builder
7a47f478a4 [maven-release-plugin] prepare for next development iteration 2022-02-09 07:23:10 +00:00
Debezium Builder
b1dd35025e [maven-release-plugin] prepare release v1.9.0.Alpha2 2022-02-09 07:23:10 +00:00
Farid Uyar
a559e51b08 DBZ-4547 Create History Topic with optional RF 2022-02-08 16:58:13 +01:00
Paweł Malon
7f7e71b36a DBZ-4704 Replaced hasNotField with hasField and negated the result of it 2022-02-07 20:39:52 +01:00
Paweł Malon
abd8c4202f DBZ-4704 Fixed the problem with expanding JSON payload that contains arrays of objects 2022-02-07 20:39:52 +01:00
Chris Cranford
055d5ba474 DBZ-4701 executeWithoutCommitting throw exception when auto-commit is enabled 2022-02-07 06:21:11 +01:00
Snigdhajyoti Ghosh
0ab45ecab0 DBZ-4518 Make kafka query timeout configurable 2022-02-04 14:08:15 +01:00
Chris Cranford
3980210c0c DBZ-4648 Relax case requirement for SnapshotType 2022-02-03 18:15:23 +01:00
nathan
d5dcde8f7a DBZ-4646 Changing exists method for FileDataBaseHistory to test if file is empty. 2022-02-03 16:47:22 +01:00
Mark Drilling
8a5279d304
DBZ-4314 Define Oracle category grouping and ordering (#3151) 2022-02-02 15:33:01 -06:00
Jiri Pechanec
74ea147152 DBZ-3679 Clarify log messages 2022-02-02 14:33:41 -05:00
Chris Cranford
088df2db8e DBZ-4584 Fix test failures 2022-01-31 09:14:46 +01:00
Chris Cranford
dcd8607108 DBZ-4584 Support case-sensitive Oracle incremental snapshots 2022-01-31 09:14:46 +01:00
Debezium Builder
33e2f5cd0f [maven-release-plugin] prepare for next development iteration 2022-01-26 19:00:20 +00:00
Debezium Builder
a77b6f8b8e [maven-release-plugin] prepare release v1.9.0.Alpha1 2022-01-26 19:00:20 +00:00
Jiri Pechanec
55894e57c2 DBZ-4610 Upgrade to Kafka 3.1.0 2022-01-25 15:26:42 +01:00
Chris Cranford
bd604a23cd DBZ-4419 Add "none" option to skipped.operations 2022-01-22 00:33:21 +01:00
Chris Cranford
773debc340 DBZ-4419 Deprecate TruncateHandlingMode, favor skipped.operations 2022-01-22 00:33:21 +01:00
Jiri Pechanec
664d989e42 DBZ-4577 Provide custom error message handling 2022-01-21 10:24:19 +01:00
Jiri Pechanec
8f5bb31eab DBZ-3700 Strings a bytes only when converters present
When database was used with non-UTF8 charset the snapshot was sending
byte array with UTF-8 bytes but MySQL converter was using database based
charset to convert it to String leading to malformed data.
It is necessary to convert strings using the database charset but it
might be a problem in case of non-charset matching.
The current solution
1) Uses strings by default
2) Switches to byte array only when converters are present to cover
original use case
3) If the conversion fails it switches back to string
2022-01-19 16:20:32 +01:00
Jiri Pechanec
8fd3243a86 DBZ-2224 Test logging based on logback 2022-01-18 11:24:21 +01:00
Sergei Morozov
17fba7258b DBZ-4541: Promote failure to register metrics to exception 2022-01-14 19:50:57 +01:00
Sergei Morozov
6d97433d18 DBZ-4541: Remove logger from Metrics register/unregister signatures 2022-01-14 19:50:57 +01:00
Chris Cranford
610e8eba73 DBZ-4548 Fix DatabaseVersionResolver comparison logic 2022-01-13 13:29:34 -05:00
Sergei Morozov
0329859cda DBZ-4459: Introduce interfaces and default implementations for change event source metrics 2022-01-12 14:32:19 +01:00
Gunnar Morling
f0b4a5ba9f DBZ-4351 Misc. clean-up 2022-01-06 19:03:46 +01:00
harveyyue
291e8ac221 DBZ-4351 Add create/update/delete event seen metrics for monitor upstream dml operation 2022-01-06 19:03:46 +01:00
harveyyue
831217053c DBZ-4503 Drop the primary key column getting exception 2022-01-03 10:57:07 +01:00
harveyyue
3670b0c6ef DBZ-4497 Parse multiple signed/unsigned keyword from ddl statement failed 2022-01-03 10:47:03 +01:00
Gunnar Morling
92665f1799 DBZ-4472 Improving log output 2021-12-20 15:58:14 +01:00
Jiri Pechanec
d13e6f546c DBZ-4467 Skip unsupported test for Db2 2021-12-20 11:29:30 +01:00
Debezium Builder
67c9663c98 [maven-release-plugin] prepare for next development iteration 2021-12-16 10:11:29 +00:00
Debezium Builder
8d532d895f [maven-release-plugin] prepare release v1.8.0.Final 2021-12-16 10:11:28 +00:00
Chris Cranford
f99ad7512f DBZ-3940 Add connector-driven EventRouter configuration 2021-12-13 13:52:31 +01:00
Jiri Pechanec
740b6b21f6 DBZ-4298 Support non-JDBC logical values in keys for incremental snapshots
Incremental snapshot source gets the PK values after they are processed
by value converter. This is usally correct solution. The values are
passed to chunk queries. There are few datatypes like unconstrained
NUMERIC in PostgreSQL that must be converted to a Debezium specific
class to keep and enrich the value. Such class is not known to JDBC
driver. This commit intorduce a marker interface that enables the code
to identify the value not recognized by JDBC and provides a method to
obtain the original value.
2021-12-13 12:29:21 +01:00
Gunnar Morling
0023cb10a5
DBZ-4393 Create a Debezium Schema Generator for Debezium connectors
* added an API generator for Debezium connectors and static API definitions for connectors in a separate module
* added Maven plug-in
* added GH workflow for debezium-schema-generator

Co-authored-by: rkerner <rkerner.mobil@gmail.com>
Co-authored-by: Anisha Mohanty <anishamohanty23@gmail.com>
2021-12-10 12:35:09 +01:00
Gunnar Morling
73cfe71342 DBZ-4423 Micro-optimizing Strings::duration() 2021-12-10 11:30:18 +01:00
Debezium Builder
f53e6c00df [maven-release-plugin] prepare for next development iteration 2021-12-09 20:16:04 +00:00
Debezium Builder
826837701a [maven-release-plugin] prepare release v1.8.0.CR1 2021-12-09 20:16:04 +00:00
Jiri Pechanec
b1dcfcb0b8 DBZ-3342 Incremental snapshot support for MongoDB 2021-12-09 10:03:41 +01:00
Tom Billiet
ec9f122e0a DBZ-4385 Truncate support for Oracle 2021-12-08 15:00:53 -05:00
Gunnar Morling
1f8fd22685 DBZ-4341 Improving logging;
Also fixing incorrect JavaDoc.
2021-12-08 13:12:54 +01:00
Chris Cranford
8f007e08a2 DBZ-4388 Don't set default value if resolution fails 2021-12-07 16:38:22 +01:00
Debezium Builder
b5439b54bf [maven-release-plugin] prepare for next development iteration 2021-11-30 09:38:47 +00:00
Debezium Builder
125f6e5aa2 [maven-release-plugin] prepare release v1.8.0.Beta1 2021-11-30 09:38:47 +00:00
Jiri Pechanec
2789dd610a DBZ-4272 Throw exception for non matching schema 2021-11-30 09:28:44 +01:00
Jiri Pechanec
5e40e1cdf5 DBZ-4272 Skip connector test using annotations 2021-11-30 09:28:44 +01:00
Jiri Pechanec
358eb0379b DBZ-4272 Disable tests for SQL Server 2021-11-30 09:28:44 +01:00
Chris Cranford
0a10d5b484 DBZ-4272 Trigger schema refresh before snapshot; only for PostgreSQL 2021-11-30 09:28:44 +01:00
Chris Cranford
3a8d1ff838 DBZ-4272 Incrementally snapshot after schema change without NPE 2021-11-30 09:28:44 +01:00
Gunnar Morling
2157256398 DBZ-4364 Awaiting table metadata to become queryable 2021-11-29 09:23:18 +01:00
Jiri Pechanec
1c5f3b2330 DBZ-4364 Make byte[] and ByteBuffer comparable 2021-11-29 09:23:18 +01:00
Jiri Pechanec
4a2a60db74 DBZ-4364 Fix missing method call 2021-11-29 09:23:18 +01:00
Gunnar Morling
560ef2babd DBZ-2363 Misc. adjustments;
* Adding Strings::isBlank()
* Simplifying test
* Using long in cases where we know it's present
* Removing some unused code
* Testing binary handling mode
* Avoiding TX id value overflow
* Assigning names to schemas to avoid issues as in DBZ-650
2021-11-26 14:29:15 +01:00
lairen
a0898ed440 DBZ-2363 Adding support for pg_logical_emit messages in Postgres connector for the pgoutput plugin 2021-11-26 14:29:15 +01:00
Jacob Gminder
f7f8748062 DBZ-2975: Move the poll interval logic to the ChangeEventSourceCoordinator. 2021-11-22 12:30:31 +01:00
Gunnar Morling
a7c6d18489 DBZ-4309 Removing legacy logger 2021-11-19 08:04:44 +01:00
Andrey Yegorov
b6d2c41fb0 DBZ-4309 Avoid calling 'LoggerFactory.getLogger(getClass())' for each new instance of the RelationalChangeRecordEmitter 2021-11-18 22:37:51 +01:00
Chris Cranford
2730187a4b DBZ-4012 Deprecate table.field.event.payload.id 2021-11-18 21:45:41 +01:00
Chris Cranford
ab86cd18a9 DBZ-4012 Suggested changes 2021-11-18 21:45:41 +01:00
Chris Cranford
349b9690f4 DBZ-4012 Suggested changes 2021-11-18 21:45:41 +01:00