Commit Graph

1663 Commits

Author SHA1 Message Date
mfvitale
0ca87d8031 DBZ-4027 During event dispatch process only signal from source channel 2023-05-05 13:35:40 +02:00
mfvitale
952340286e DBZ-4027 Make signalling channel configurable
DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Code style

DBZ-4027 Make SignalPayload more generic and extensible

DBZ-4027 Rename DatabaseSignalChannel to SourceSignalChannel

DBZ-4027 Improve logging

DBZ-4027 Moved SPI file definition to debezium-core

DBZ-4027 Move SignalProcessor synchronization point to be processed only when a signal cdc event arrives.

DBZ-4027 Add EventDispatcher constructor without signalProcessor for spanner connector

DBZ-4027 Fix NPE

DBZ-4027 Fix NPE

DBZ-4027 Formatting

DBZ-4027 Correctly manage signal on not supported connector

DBZ-4027 Use the correct MongoDbOffset

DBZ-4027 Correctly initialize offset for Oracle and SqlServer connectors

DBZ-4027 Register SPI implementations

DBZ-4027 Improve SignalProcessor instantiation

DBZ-4027 Pass source info in case of SchemaChanges action

DBZ-4027 Manage close event in a synchronous way

DBZ-4027 Correctly init offset context also in case of snapshot mode 'never'

DBZ-4027 Fix MySqlMetricsIT test

DBZ-4027 Move KafkaSignalChannel to core

DBZ-4027 Move KafkaSignalChannel to core

DBZ-4027 Set pass offset context after initial snapshot to SignalProcessor

DBZ-4027 Pass OffsetContext to signal processor

DBZ-4027 Pass CommonConnectorConfig to SignalChannelReader init method

DBZ-4027 Move Incremental snapshot window actions to dedicated package

DBZ-4027 Align SignalsIT test with new code

DBZ-4027 Fix SignalsIT test

DBZ-4027 Fix SignalProcessor scheduling

DBZ-4027 Moved DatabaseSignalChannel and SignalChannelReader to dedicated package

DBZ-4027 Start SignalProcessor from ChangeEventSourceCoordinator

DBZ-4027 Create SignalProcessor and renamed Signal to DatabaseSignalChannel

DBZ-4027 Initial refactoring of signal feature
2023-05-05 13:35:40 +02:00
Jiri Pechanec
58ef4f0b98 DBZ-6157 Disable potentially dangerous MySQL JDBC props 2023-05-04 11:54:04 +02:00
Jiri Pechanec
6b1455666a DBZ-6411 Message content logged only at TRACE 2023-05-03 06:51:28 +02:00
mfvitale
e5d4332835 DBZ-6411 Fix HeaderToValue schema update cache.
When the connector starts for the first time with initial snapshot record with read operation will be created. In this phase the ExtractChangedRecordState will produce no headers and since HeaderToValue was not skipping record without headers the cache of the new schema will be created only with the old fields.
2023-05-03 06:51:28 +02:00
harveyyue
3860a4faca DBZ-6388 __source_ts_ms r (read) operation date is set to future for SQL Server 2023-05-02 08:11:02 -04:00
Chris Cranford
4664dd6df6 DBZ-6355 Add log.mining.transaction.retention.ms; deprecate log.mining.transaction.retention.hours 2023-04-27 09:28:29 -04:00
eizners
fa42a1b0ce DBZ-6355 fix unit tests errors 2023-04-27 09:28:29 -04:00
Sergey Eizner
cac5485176 DBZ-6355 Change to the last offset 2023-04-27 09:28:29 -04:00
Jiri Pechanec
22e19c0ff3 DBZ-6209 Extract additional common code 2023-04-27 08:13:58 +02:00
harveyyue
795f0dd977 DBZ-6209 Refactor storage implementations 2023-04-27 08:13:58 +02:00
Jiri Pechanec
8c52ccfd0c DBZ-6391 Use incdluded term for consistency 2023-04-26 12:31:15 +02:00
zalmane
2b8bf33876 DBZ-6391 change logging level 2023-04-26 12:31:15 +02:00
Debezium Builder
bb1b79f3bd [maven-release-plugin] prepare for next development iteration 2023-04-20 09:09:43 +02:00
Debezium Builder
5483b4de99 [maven-release-plugin] prepare release v2.2.0.Final 2023-04-20 09:09:43 +02:00
Ronak Jain
762122ecf5 DBZ-2979 If column.include.list/column.exclude.list are used and the target table receives an update for the excluded (or not included) column - such events should be ignored
* Add Mysql and SqlServer Connector Changes
* Change Docs
* Clean Test Case
* Formatting and Checkstyle

Addresses/Closes DBZ-2979
2023-04-19 09:04:29 +02:00
Jiri Pechanec
5eb5e3555d DBZ-6324 Rename flaky test config option 2023-04-18 08:22:43 +02:00
Jiri Pechanec
ab66f038e4 DBZ-6324 Add retry for flaking tests 2023-04-18 08:22:43 +02:00
Jiri Pechanec
f64c35de29 DBZ-6324 Provide @Flaky annotation for CI unstable tests 2023-04-18 08:22:43 +02:00
Debezium Builder
976a47a3a3 [maven-release-plugin] prepare for next development iteration 2023-04-14 08:46:06 +00:00
Debezium Builder
1ca8399e8b [maven-release-plugin] prepare release v2.2.0.CR1 2023-04-14 08:46:06 +00:00
Jiri Pechanec
8a648fbb4d DBZ-4488 Refactor code to keep error handler immutable 2023-04-13 08:22:45 +02:00
Jacob Gminder
5e615ff796 DBZ-4488: Reuse the ErrorHandler between retries 2023-04-13 08:22:45 +02:00
Jiri Pechanec
84d0d580bc DBZ-4488 Refactor the code 2023-04-13 08:22:45 +02:00
Jacob Gminder
9d5b6df0f7 DBZ-4488: Allow SQL Server connectors to set a maximum number of retriable exception restarts. 2023-04-13 08:22:45 +02:00
Jiri Pechanec
943b9485d7 DBZ-6216 Remove registry name for tag query 2023-04-12 14:07:35 +02:00
harveyyue
c525f34cd0 DBZ-6070 DDL events not stored in schema history topic for excluded tables 2023-04-06 12:10:30 +02:00
Jiri Pechanec
39ee498e64 DBZ-5902 Fix wording 2023-04-06 10:55:19 +02:00
mfvitale
bad6ad6bae DBZ-5902 Change fields configuration from 'partition.payload.field' to 'partition.payload.fields' 2023-04-06 10:55:19 +02:00
mfvitale
41416293ca DBZ-5902 Refactor and better field validation 2023-04-06 10:55:19 +02:00
mfvitale
fdf79b0379 DBZ-5902 Add warn log for deprecation 2023-04-06 10:55:19 +02:00
mfvitale
34f021988e DBZ-5902 Add new PartitionRouting SMT that will replace ComputePartition 2023-04-06 10:55:19 +02:00
mfvitale
6e9528a47d DBZ-5902 Deprecate ComputePartition SMT 2023-04-06 10:55:19 +02:00
mfvitale
80ac1b362f DBZ-5902 Moved list validation to CommonConnectorConfig 2023-04-06 10:55:19 +02:00
Debezium Builder
5c650fd054 [maven-release-plugin] prepare for next development iteration 2023-03-31 10:00:16 +00:00
Debezium Builder
68ffc4833b [maven-release-plugin] prepare release v2.2.0.Beta1 2023-03-31 10:00:16 +00:00
Jiri Pechanec
b6c984e4de DBZ-6072 Sort imports 2023-03-28 17:06:58 +02:00
Jiri Pechanec
0e42935b35 DBZ-6072 Use more descriptive name 2023-03-28 16:59:05 +02:00
harveyyue
5dd4b53ec3 DBZ-6072 NPE when setting schema.history.internal.store.only.captured.tables.ddl=true 2023-03-28 16:59:05 +02:00
harveyyue
2f46c83b2a DBZ-6104 java.lang.NullPointerException in MySQL connector with max.queue.size.in.bytes 2023-03-28 14:51:52 +02:00
harveyyue
c9c1ce8a2f DBZ-6185 Zerofill property failed for different int types 2023-03-28 09:34:58 +02:00
Jiri Pechanec
e3fdd873e1 DBZ-6201 Skip expensive logging 2023-03-22 14:02:44 +01:00
mfvitale
2d70f47c40 DBZ-6201: Add more fail test scenarios 2023-03-22 14:02:44 +01:00
mfvitale
16191a5a35 DBZ-6201: Add logs 2023-03-22 14:02:44 +01:00
mfvitale
039a3f04e8 DBZ-6201: Add support for move and nested fields for HeaderToValue SMT 2023-03-22 14:02:44 +01:00
mfvitale
c45bf39eb0 DBZ-6201: Add new HeaderToValue SMT to copy headers to value 2023-03-22 14:02:44 +01:00
mfvitale
4e544debdc DBZ-6201: Fix validation message that reports null as value 2023-03-22 14:02:44 +01:00
Chris Cranford
1c5f14f7fd DBZ-6225 DBZ-6226 DBZ-6231 Introduce CustomConverter for JDBC sink consistency
In DBZ-6225, it was identified that the schema type associated with TINYINT(1)
and BOOLEAN data types wasn't consistent. This converter implementation will
always emit BOOLEAN data types as INT16, similar to TINYINT(1).

In DBZ-6226, it was identified that the schema type associated with REAL data
types wasn't consistent, emitted as FLOAT64 during snapshot and FLOAT32 in
the streaming phase. This converter implementation will always emit such types
as FLOAT64 for consistency.

In DBZ-6231, during streaming the DDL parser will get the column type as the
literal type supplied by the user's DDL, i.e. NCHAR or NVARCHAR but when the
DDL is sourced from SHOW CREATE TABLE, MySQL returns these columns as types
of CHAR/VARCHAR but with a character set.  By passing the character_set in
the schema consistency, we can deduce nationalized vs non-nationalized.
2023-03-21 14:53:56 +01:00
harveyyue
1877906dd8 DBZ-6197 Remove duplicated createDdlFilter method from historized connector config 2023-03-13 10:56:43 +01:00
Nir Levy
16f460d2d7 DBZ-6190: Fix support for ddl filter 2023-03-09 08:48:43 +01:00