mfvitale
524987227f
DBZ-6566 Shutdown blockingSnapshotExecutor when ChangeEventSourceCoordinator stops
2023-08-03 09:22:24 +02:00
mfvitale
61651a94b5
DBZ-6566 Force blocking snapshot even if a snapshot has already executed
2023-08-03 09:22:24 +02:00
mfvitale
def7379a0d
DBZ-6566 Support Blocking snapshot for MySQL
2023-08-03 09:22:24 +02:00
mfvitale
6e0b323e83
DBZ-6566 Set correct log context during blocking snapshot
2023-08-03 09:22:24 +02:00
mfvitale
70f16af7c4
DBZ-6566 Support Blocking snapshot for PostgreSQL
2023-08-03 09:22:24 +02:00
jcechace
699a1612ae
DBZ-6670 Replace error handler instead of recreate in order to keep retry counter
2023-07-25 13:46:56 +02:00
mfvitale
8ea2b3c5b7
DBZ-6669 Snapshot will get data when only signal data collection table is specified
2023-07-24 10:02:13 -04:00
Debezium Builder
267dfda8a6
[maven-release-plugin] prepare for next development iteration
2023-07-17 10:40:52 +00:00
Debezium Builder
0d3cef6a29
[maven-release-plugin] prepare release v2.4.0.Alpha1
2023-07-17 10:40:52 +00:00
Chris Cranford
de8cf36b2c
DBZ-6660 Introduce internal.log.mining.max.scn.deviation.ms
...
We hypothesize that there could be a situation where we may be mining precisely
around the CURRENT_SCN and this may lead to situations where LGWR may not have
flushed all records for the same SCN before being mined by the connector.
2023-07-17 10:22:46 +02:00
Chris Cranford
56ddbe21d5
DBZ-6650 Avoid ConcurrentModificationException during dispatch
2023-07-14 11:21:39 +02:00
harveyyue
e3f684170e
DBZ-6472 MySqlSnapshotChangeEventSource parallel execute createSchemaEventsForTables
2023-07-13 13:21:59 +02:00
Adam Strickland
0b15fe4556
[docs] Doc of type of return value
2023-07-13 13:08:41 +02:00
Jiri Pechanec
764427081b
DBZ-6517 DBZ-6654 Make CE converter compatible with Kafka 3.5
2023-07-13 11:26:21 +02:00
harveyyue
aa139203aa
DBZ-6603 Update property "custom.metric.tags" to connector docs
2023-07-13 09:52:33 +02:00
harveyyue
5bc7fd1f7c
DBZ-6603 Support for custom tags in the connector metrics
2023-07-13 09:52:33 +02:00
jcechace
5cb579bf12
DBZ-6525 Removing runtime dependency on core for testcontainers module in order to prevent logback.xml downstream conflicts
2023-07-11 12:07:04 +02:00
Chris Cranford
6a41761f85
DBZ-6573 Restore spacing
2023-06-30 09:39:32 +02:00
tjwornjs
2767f469aa
DBZ-6573 add max retries config to common config
2023-06-30 09:39:32 +02:00
bdbene
6f29ce08ef
DBZ-6605 Fix DataCollections for snapshot completion notification
...
DBZ-6605 Use new 'scanned_collection' field for snapshot completion signal
Updated the docs for fixed snapshot completion notification
2023-06-27 12:01:26 +02:00
mfvitale
d5febcc054
DBZ-6417 Provide extensible signal action mechanism through SPI
2023-06-23 15:03:27 +02:00
mfvitale
9ed928a1c9
DBZ-6416 Improve tests stability
2023-06-22 13:22:34 +02:00
mfvitale
34e28ac52d
DBZ-6416 Add initial snapshot notifications
2023-06-22 13:22:34 +02:00
mfvitale
7d28833ba8
DBZ-6588 Skip tombstone record on HeaderToValue SMT
2023-06-21 14:11:44 +02:00
jcechace
8f70de42c1
DBZ-6578 Cleaning up connection string propagation between discovery and tasks
2023-06-21 12:22:23 +02:00
Debezium Builder
60f730d383
[maven-release-plugin] prepare for next development iteration
2023-06-20 13:21:31 +00:00
Debezium Builder
150396e4c8
[maven-release-plugin] prepare release v2.3.0.Final
2023-06-20 13:21:31 +00:00
mfvitale
91152c4c5d
DBZ-6559 Add test to cover topic name strategy
2023-06-16 11:48:34 +02:00
Jiri Pechanec
7f81f4bc6b
DBZ-6559 Handle keeping the first char as exception
2023-06-16 11:48:34 +02:00
mfvitale
87217a1474
DBZ-6559 Restore backward compatibility for field that stars with a digit
2023-06-16 11:48:34 +02:00
Jiri Pechanec
067683e56a
DBZ-6076 Consistent config for all connectors
2023-06-15 11:55:28 +02:00
Anil Dasari
0431a8b9fc
DBZ-6076: Add support for custom SourceInfoStructMaker for adding new fields to source field
2023-06-15 11:55:28 +02:00
Chris Cranford
ef8059f3ff
DBZ-6528 Fix Oracle NPE when storing only captured table schemas with signals
2023-06-14 10:48:19 +02:00
mfvitale
b7eb4b5c62
DBZ-6543 Fix NPE due to optional fields in PartitionRouting SMT
2023-06-14 07:22:27 +02:00
Andrei Isac
573a91be07
DBZ-6533: fix table order on partial match when snapshotting
2023-06-12 15:40:03 +02:00
kate
24d3425fdf
DBZ-6552 Notify when incremental snapshot fails
2023-06-12 15:14:40 +02:00
mfvitale
3aaca16024
DBZ-6543 Fix NPE due to optional fields in PartitionRouting SMT
2023-06-09 13:47:18 +02:00
Debezium Builder
5d1b00588a
[maven-release-plugin] prepare for next development iteration
2023-06-09 09:23:38 +00:00
Debezium Builder
d1e6107517
[maven-release-plugin] prepare release v2.3.0.CR1
2023-06-09 09:23:37 +00:00
Vojtech Juranek
b795519514
DBZ-6529 Use Murmur hash for PartitionRouting SMT
...
Add option to use better hash function than default Java Object::hash
function to get better hashes which would be more equally spred over
the hash space and thus more equally over the Kafka partitions.
To preserve backward compatibility, previous Java `hashCode` function
is used a default.
Add Murmur3 hash function for computing hashes of the fields.
Murmur3 implementation is taken from from Infinispan project code base.
To allow users eventually use their own make `computePartition()`
protected so it can be overriden in the subclasses.
2023-06-07 11:48:30 +02:00
mfvitale
61e75cc00d
DBZ-6424 Move JMX register/unregister code from Metrics to JmxUtils
2023-06-07 07:36:58 +02:00
mfvitale
b0b968fcce
DBZ-6424 Move JMX register/unregister code from Metrics to JmxUtils
2023-06-07 07:36:58 +02:00
mfvitale
394a9e32d8
DBZ-6424 Move JMX register/unregister code from Metrics to JmxUtils
2023-06-07 07:36:58 +02:00
mfvitale
7338fde84e
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
...
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
2023-06-07 07:36:58 +02:00
mfvitale
769d9361e7
DBZ-6424 Add JMX Notification channel
...
This channel permits to send Debezium notification to JMX bean
DBZ-6424 Add JMX Signal channel
This channel permits to send signal to Debezium through the JMX operation
2023-06-07 07:36:58 +02:00
mfvitale
f93affc3ca
DBZ-6511 Add java docs for SignalProcessor getSignalChannel
method
2023-06-05 17:33:07 +02:00
mfvitale
ba8a3e5062
DBZ-6511 Expose reset method on SignalChannelReader to reset processed signals
2023-06-05 17:33:07 +02:00
mfvitale
4d56bbd15a
DBZ-6511 Restore Kafka signal offset only when KafkaSignalChannel is enabled
2023-06-05 17:33:07 +02:00
Ronak Jain
3f3967c2bb
DBZ-6366: Code Improvements for skip.messages.without.change
...
Addresses/Closes DBZ-6366
2023-06-05 14:50:25 +02:00
Jiri Pechanec
8a1cd95086
DBZ-6515 Add units to the description
2023-06-05 11:27:57 +02:00
mfvitale
155207dc9b
DBZ-6509 Add FileSignalChannel to META-INF/services
2023-05-31 14:39:35 +02:00
mfvitale
e1c1e65336
DBZ-6408 Add documentation for notifications
2023-05-30 17:17:52 +02:00
Debezium Builder
aa5b2131fc
[maven-release-plugin] prepare for next development iteration
2023-05-26 10:35:20 +00:00
Debezium Builder
9cc9b3c29a
[maven-release-plugin] prepare release v2.3.0.Beta1
2023-05-26 10:35:20 +00:00
mfvitale
ed2157bfc9
DBZ-6486 ExtractNewRecordState skip fields that not exists on payload
2023-05-26 11:30:37 +02:00
Jiri Pechanec
b72f084eac
DBZ-6486 Check schema fields
2023-05-26 11:30:37 +02:00
mfvitale
08122371eb
DBZ-6486 ExtractNewRecordState can now take arbitrary fields from payload
2023-05-26 11:30:37 +02:00
mfvitale
7ba9f57035
DBZ-6496 Fix signal.poll.interval.ms
default value
2023-05-24 18:51:49 +02:00
mfvitale
dc00f64bbf
DBZ-6447 Incremental snapshot notification ids are now equal to the id sent in the execute-snapshot
signal.
2023-05-23 09:57:05 +02:00
harveyyue
016daf6fd8
DBZ-6490 Convert the field BigDecimal value to bytes failure of ApproximateStructSizeCalculator
2023-05-23 08:38:06 +02:00
Chris Cranford
d40da4f810
DBZ-6457 Use PDB for parallel snapshot connections
2023-05-19 14:04:14 +02:00
Ismail Simsek
2683b31394
DBZ-4027 implement FileSignalChannel
2023-05-15 15:34:43 +02:00
Vojtech Juranek
ee4ebaa957
DBZ-6439 Don't scan all the tables when loading schema
...
Currently we scan all the tables, which may result into a substantial
delay in initial snapshot when the database is very large. We need to
filter out tables which we are not interested in.
Add back table filter when loading schema of tables. As per comment of
this block of code, passing all tables and table filter should be faster
than passing only list of tables we are interested in.
2023-05-15 14:11:47 +02:00
Vojtech Juranek
f4596fa2e0
DBZ-6460 Enable setting log level in tests via log interceptor
2023-05-15 14:11:47 +02:00
mfvitale
85278e76af
DBZ-6462 Stop SignalProcessor after test execution in SignalProcessorTest
2023-05-15 08:51:02 +02:00
mfvitale
e41bbe7191
DBZ-6462 Schedule SignalProcessor task with a more relaxed scheduling. Removes parallelStreams while processing signalChannelReaders
2023-05-15 08:51:02 +02:00
Debezium Builder
a5b44767bc
[maven-release-plugin] prepare for next development iteration
2023-05-11 11:05:24 +00:00
Debezium Builder
1b1a7531f2
[maven-release-plugin] prepare release v2.3.0.Alpha1
2023-05-11 11:05:24 +00:00
mfvitale
7429b4e6f1
DBZ-1973 Add incremental snapshot notifications
2023-05-10 12:22:02 +02:00
mfvitale
811b800f55
DBZ-1973 Add support for Notifications
...
DBZ-1973 Add more tests for other connectors
DBZ-1973 Add send method with offset parameter
DBZ-1973 Instantiate NotificationService in the task class
DBZ-1973 Instantiate NotificationService in the task class
DBZ-4027 Move specific sink channel configuration to SinkNotificationChannel
DBZ-4027 Remove not used SPI file
DBZ-1973 Moved SPI file definition to debezium-core
DBZ-1973 Rename KafkaNotificationChannel to more generic SinkNotificationChannel
DBZ-1973 Code refactor
DBZ-1973 Improve configuration property description
DBZ-1973 Improve test
DBZ-1973 Add close method to NotificationChannel
DBZ-1973 Implement KafkaNotificationChannel
DBZ-1973 Add NotificationService and LogNotificationChannel
2023-05-10 12:22:02 +02:00
mfvitale
0ca87d8031
DBZ-4027 During event dispatch process only signal from source channel
2023-05-05 13:35:40 +02:00
mfvitale
952340286e
DBZ-4027 Make signalling channel configurable
...
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Code style
DBZ-4027 Make SignalPayload more generic and extensible
DBZ-4027 Rename DatabaseSignalChannel to SourceSignalChannel
DBZ-4027 Improve logging
DBZ-4027 Moved SPI file definition to debezium-core
DBZ-4027 Move SignalProcessor synchronization point to be processed only when a signal cdc event arrives.
DBZ-4027 Add EventDispatcher constructor without signalProcessor for spanner connector
DBZ-4027 Fix NPE
DBZ-4027 Fix NPE
DBZ-4027 Formatting
DBZ-4027 Correctly manage signal on not supported connector
DBZ-4027 Use the correct MongoDbOffset
DBZ-4027 Correctly initialize offset for Oracle and SqlServer connectors
DBZ-4027 Register SPI implementations
DBZ-4027 Improve SignalProcessor instantiation
DBZ-4027 Pass source info in case of SchemaChanges action
DBZ-4027 Manage close event in a synchronous way
DBZ-4027 Correctly init offset context also in case of snapshot mode 'never'
DBZ-4027 Fix MySqlMetricsIT test
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Set pass offset context after initial snapshot to SignalProcessor
DBZ-4027 Pass OffsetContext to signal processor
DBZ-4027 Pass CommonConnectorConfig to SignalChannelReader init method
DBZ-4027 Move Incremental snapshot window actions to dedicated package
DBZ-4027 Align SignalsIT test with new code
DBZ-4027 Fix SignalsIT test
DBZ-4027 Fix SignalProcessor scheduling
DBZ-4027 Moved DatabaseSignalChannel and SignalChannelReader to dedicated package
DBZ-4027 Start SignalProcessor from ChangeEventSourceCoordinator
DBZ-4027 Create SignalProcessor and renamed Signal to DatabaseSignalChannel
DBZ-4027 Initial refactoring of signal feature
2023-05-05 13:35:40 +02:00
Jiri Pechanec
58ef4f0b98
DBZ-6157 Disable potentially dangerous MySQL JDBC props
2023-05-04 11:54:04 +02:00
Jiri Pechanec
6b1455666a
DBZ-6411 Message content logged only at TRACE
2023-05-03 06:51:28 +02:00
mfvitale
e5d4332835
DBZ-6411 Fix HeaderToValue schema update cache.
...
When the connector starts for the first time with initial snapshot record with read operation will be created. In this phase the ExtractChangedRecordState will produce no headers and since HeaderToValue was not skipping record without headers the cache of the new schema will be created only with the old fields.
2023-05-03 06:51:28 +02:00
harveyyue
3860a4faca
DBZ-6388 __source_ts_ms r (read) operation date is set to future for SQL Server
2023-05-02 08:11:02 -04:00
Chris Cranford
4664dd6df6
DBZ-6355 Add log.mining.transaction.retention.ms; deprecate log.mining.transaction.retention.hours
2023-04-27 09:28:29 -04:00
eizners
fa42a1b0ce
DBZ-6355 fix unit tests errors
2023-04-27 09:28:29 -04:00
Sergey Eizner
cac5485176
DBZ-6355 Change to the last offset
2023-04-27 09:28:29 -04:00
Jiri Pechanec
22e19c0ff3
DBZ-6209 Extract additional common code
2023-04-27 08:13:58 +02:00
harveyyue
795f0dd977
DBZ-6209 Refactor storage implementations
2023-04-27 08:13:58 +02:00
Jiri Pechanec
8c52ccfd0c
DBZ-6391 Use incdluded term for consistency
2023-04-26 12:31:15 +02:00
zalmane
2b8bf33876
DBZ-6391 change logging level
2023-04-26 12:31:15 +02:00
Debezium Builder
bb1b79f3bd
[maven-release-plugin] prepare for next development iteration
2023-04-20 09:09:43 +02:00
Debezium Builder
5483b4de99
[maven-release-plugin] prepare release v2.2.0.Final
2023-04-20 09:09:43 +02:00
Ronak Jain
762122ecf5
DBZ-2979 If column.include.list/column.exclude.list are used and the target table receives an update for the excluded (or not included) column - such events should be ignored
...
* Add Mysql and SqlServer Connector Changes
* Change Docs
* Clean Test Case
* Formatting and Checkstyle
Addresses/Closes DBZ-2979
2023-04-19 09:04:29 +02:00
Jiri Pechanec
5eb5e3555d
DBZ-6324 Rename flaky test config option
2023-04-18 08:22:43 +02:00
Jiri Pechanec
ab66f038e4
DBZ-6324 Add retry for flaking tests
2023-04-18 08:22:43 +02:00
Jiri Pechanec
f64c35de29
DBZ-6324 Provide @Flaky annotation for CI unstable tests
2023-04-18 08:22:43 +02:00
Debezium Builder
976a47a3a3
[maven-release-plugin] prepare for next development iteration
2023-04-14 08:46:06 +00:00
Debezium Builder
1ca8399e8b
[maven-release-plugin] prepare release v2.2.0.CR1
2023-04-14 08:46:06 +00:00
Jiri Pechanec
8a648fbb4d
DBZ-4488 Refactor code to keep error handler immutable
2023-04-13 08:22:45 +02:00
Jacob Gminder
5e615ff796
DBZ-4488: Reuse the ErrorHandler between retries
2023-04-13 08:22:45 +02:00
Jiri Pechanec
84d0d580bc
DBZ-4488 Refactor the code
2023-04-13 08:22:45 +02:00
Jacob Gminder
9d5b6df0f7
DBZ-4488: Allow SQL Server connectors to set a maximum number of retriable exception restarts.
2023-04-13 08:22:45 +02:00
Jiri Pechanec
943b9485d7
DBZ-6216 Remove registry name for tag query
2023-04-12 14:07:35 +02:00
harveyyue
c525f34cd0
DBZ-6070 DDL events not stored in schema history topic for excluded tables
2023-04-06 12:10:30 +02:00
Jiri Pechanec
39ee498e64
DBZ-5902 Fix wording
2023-04-06 10:55:19 +02:00
mfvitale
bad6ad6bae
DBZ-5902 Change fields configuration from 'partition.payload.field' to 'partition.payload.fields'
2023-04-06 10:55:19 +02:00
mfvitale
41416293ca
DBZ-5902 Refactor and better field validation
2023-04-06 10:55:19 +02:00
mfvitale
fdf79b0379
DBZ-5902 Add warn log for deprecation
2023-04-06 10:55:19 +02:00
mfvitale
34f021988e
DBZ-5902 Add new PartitionRouting SMT that will replace ComputePartition
2023-04-06 10:55:19 +02:00
mfvitale
6e9528a47d
DBZ-5902 Deprecate ComputePartition SMT
2023-04-06 10:55:19 +02:00
mfvitale
80ac1b362f
DBZ-5902 Moved list validation to CommonConnectorConfig
2023-04-06 10:55:19 +02:00
Debezium Builder
5c650fd054
[maven-release-plugin] prepare for next development iteration
2023-03-31 10:00:16 +00:00
Debezium Builder
68ffc4833b
[maven-release-plugin] prepare release v2.2.0.Beta1
2023-03-31 10:00:16 +00:00
Jiri Pechanec
b6c984e4de
DBZ-6072 Sort imports
2023-03-28 17:06:58 +02:00
Jiri Pechanec
0e42935b35
DBZ-6072 Use more descriptive name
2023-03-28 16:59:05 +02:00
harveyyue
5dd4b53ec3
DBZ-6072 NPE when setting schema.history.internal.store.only.captured.tables.ddl=true
2023-03-28 16:59:05 +02:00
harveyyue
2f46c83b2a
DBZ-6104 java.lang.NullPointerException in MySQL connector with max.queue.size.in.bytes
2023-03-28 14:51:52 +02:00
harveyyue
c9c1ce8a2f
DBZ-6185 Zerofill property failed for different int types
2023-03-28 09:34:58 +02:00
Jiri Pechanec
e3fdd873e1
DBZ-6201 Skip expensive logging
2023-03-22 14:02:44 +01:00
mfvitale
2d70f47c40
DBZ-6201: Add more fail test scenarios
2023-03-22 14:02:44 +01:00
mfvitale
16191a5a35
DBZ-6201: Add logs
2023-03-22 14:02:44 +01:00
mfvitale
039a3f04e8
DBZ-6201: Add support for move and nested fields for HeaderToValue SMT
2023-03-22 14:02:44 +01:00
mfvitale
c45bf39eb0
DBZ-6201: Add new HeaderToValue SMT to copy headers to value
2023-03-22 14:02:44 +01:00
mfvitale
4e544debdc
DBZ-6201: Fix validation message that reports null as value
2023-03-22 14:02:44 +01:00
Chris Cranford
1c5f14f7fd
DBZ-6225 DBZ-6226 DBZ-6231 Introduce CustomConverter for JDBC sink consistency
...
In DBZ-6225, it was identified that the schema type associated with TINYINT(1)
and BOOLEAN data types wasn't consistent. This converter implementation will
always emit BOOLEAN data types as INT16, similar to TINYINT(1).
In DBZ-6226, it was identified that the schema type associated with REAL data
types wasn't consistent, emitted as FLOAT64 during snapshot and FLOAT32 in
the streaming phase. This converter implementation will always emit such types
as FLOAT64 for consistency.
In DBZ-6231, during streaming the DDL parser will get the column type as the
literal type supplied by the user's DDL, i.e. NCHAR or NVARCHAR but when the
DDL is sourced from SHOW CREATE TABLE, MySQL returns these columns as types
of CHAR/VARCHAR but with a character set. By passing the character_set in
the schema consistency, we can deduce nationalized vs non-nationalized.
2023-03-21 14:53:56 +01:00
harveyyue
1877906dd8
DBZ-6197 Remove duplicated createDdlFilter method from historized connector config
2023-03-13 10:56:43 +01:00
Nir Levy
16f460d2d7
DBZ-6190: Fix support for ddl filter
2023-03-09 08:48:43 +01:00
Debezium Builder
55d65e9618
[maven-release-plugin] prepare for next development iteration
2023-03-08 11:38:17 +00:00
Debezium Builder
6d183c3dc1
[maven-release-plugin] prepare release v2.2.0.Alpha3
2023-03-08 11:38:16 +00:00
subodh
b781960019
DBZ-6179 Fix swallowing of exception cause of finally block in RelationalSnapshotChangeEventSource
2023-03-06 10:28:57 +01:00
Jiri Pechanec
98dcdb8f55
DBZ-6023 Remove nnecessary code
2023-03-06 10:25:59 +01:00
PlugaruT
7d0492fee5
DBZ-6023 Unit tests for buildChunkQuery and buildMaxPrimaryKeyQuery methods
2023-03-06 10:25:59 +01:00
PlugaruT
472162e46e
DBZ-6023 Add support for providing a surrogate key when triggering incremental snapshots
...
For tables that have a composite Primary Key, SQL queries for incremental snapshot perform bad. This change allows to provide a surrogate key that will be used for the queries. The key should be unique in order for the snapshot to guarantee consistency.
2023-03-06 10:25:59 +01:00
Jiri Pechanec
0585c6beb8
DBZ-823 Add NULL instance for the class
2023-03-03 12:47:15 +01:00
Jiri Pechanec
d492ec892c
DBZ-823 Use enum for enumeration
2023-03-03 12:47:15 +01:00
Jiri Pechanec
3da2434c6c
DBZ-823 Restrict the type with generics
2023-03-03 12:47:15 +01:00
Jiri Pechanec
5e666f00ee
DBZ-823 Add JavaDoc to new interfaces
2023-03-03 12:47:15 +01:00
Jiri Pechanec
7d92dc0211
DBZ-823 Don't use getter based name
2023-03-03 12:47:15 +01:00
Jiri Pechanec
0f4adb9fe6
DBZ-823 Use better name for CF with main connection
2023-03-03 12:47:15 +01:00
ggaborg
e875e784a0
DBZ-823 CR changes
2023-03-03 12:47:15 +01:00
ggaborg
065dc997b9
DBZ-823 Optionally parallelize initial snapshots
2023-03-03 12:47:15 +01:00
Jiri Pechanec
0656de70c9
DBZ-6156 Add an extra test case
2023-03-03 11:20:21 +01:00
jcechace
7d3cca89bd
DBZ-6032 MongoConnection allways rethrows InterruptedException
2023-03-03 05:46:33 +01:00
jcechace
922191cdd7
DBZ-6032 Connection management refactoring
2023-03-03 05:46:33 +01:00
harveyyue
440192851d
DBZ-6156 Add hostname validator to connector configuration
2023-02-28 13:22:14 +01:00
Vojtech Juranek
0868bfd55e
DBZ-3594 Remove unused deprecated method
2023-02-23 09:26:07 -05:00
Debezium Builder
2245ceda44
[maven-release-plugin] prepare for next development iteration
2023-02-16 11:28:25 +00:00
Debezium Builder
20f3b16e8a
[maven-release-plugin] prepare release v2.2.0.Alpha2
2023-02-16 11:28:25 +00:00
Chris Cranford
a7d1bb0f86
DBZ-5283 Decoupled changed SMT vs new state SMT, updated docs
2023-02-10 13:53:05 +01:00
Chris Cranford
297225d141
DBZ-5283 Add unchanged field support to ExtractNewRecordState
2023-02-10 13:53:05 +01:00
Chris Cranford
f8eb32cf6f
DBZ-5283 Move ExtractChangedRecordState tests to own class
2023-02-10 13:53:05 +01:00
harveyyue
4cb6e7c5b3
DBZ-5283 Introduce a new SMT ExtractChangedRecordState to capture the names of changed/unchanged field within headers
2023-02-10 13:53:05 +01:00
Jeremy Ford
517b43b870
DBZ-5827 ActivateTracingSpan wrong timestamps reported
...
Change txLogSpan to use processingTimestamp for span finish.
2023-02-07 13:25:44 +01:00
Ismail Simsek
3548daf8b9
DBZ-6075 Use Instantiator to load ClassLoader
2023-02-07 07:24:03 +01:00
Jiri Pechanec
3a646519de
DBZ-5743 Modify Javadoc
2023-02-02 13:21:45 +01:00
harveyyue
57b902ff8c
DBZ-5743 Introduce alternative AVRO_UNICODE mode for field name selector
2023-02-02 13:21:45 +01:00
harveyyue
f093d7a6c9
DBZ-5743 Introduce alternative AVRO_UNICODE mode for schema adjuster
2023-02-02 13:21:45 +01:00
harveyyue
003bef0736
DBZ-5743 Introduce a new unicode topic naming strategy
2023-02-02 13:21:45 +01:00
Jiri Pechanec
795014e040
DBZ-6037 Data no longer logged on error
2023-01-26 13:30:20 +01:00
Jiri Pechanec
ad3c6981bd
DBZ-5879 Remove duplicate dependency
2023-01-24 14:29:00 +01:00
Jiri Pechanec
8f3ca36e23
DBZ-6026 Don't update last offsets in statistics
2023-01-24 14:26:05 +01:00
Vojtech Juranek
f00f2a8735
DBZ-5917 Fix querying column metadata for tables with back slash in the name
...
If the table contains back slash, which is ANSI SQL escape chracter,
in its name, querying column metadata would fail in some cases (*)
which would result into NPE. Fix table name before the query and
escape escape character if the table name contains it.
Default backslash works for all currently supported databases as it's
ANSI SQL standard,
(*) everything works when we collect column metadata for all tables,
i.e. when there are no excluded tables
2023-01-20 00:01:11 +01:00
Debezium Builder
04f5291a2d
[maven-release-plugin] prepare for next development iteration
2023-01-19 11:32:10 +00:00
Debezium Builder
027527963e
[maven-release-plugin] prepare release v2.2.0.Alpha1
2023-01-19 11:32:10 +00:00
Anil Dasari
e1f9d64fbb
DBZ-5852:Snapshotter#snapshotCompleted is invoked regardless of snapshot result
2023-01-19 09:24:02 +01:00
Jiri Pechanec
1ec41fa120
DBZ-5879 Added logging; removed commented out code
2023-01-18 14:46:14 +01:00
Jeremy Ford
553cb9b1dd
DBZ-5879 Add wait to tests in order to maintain expectations of existing integration tests
2023-01-18 14:46:14 +01:00
Jeremy Ford
f2a6f0b8db
DBZ-5879 Support retrying database connection failures during connector start
...
Reworked when task requests start from subclasses
Add support for restarting
2023-01-18 14:46:14 +01:00
Jiri Pechanec
e8729bb0eb
DBZ-5996 Properly handle zoned timestamp width in Oracle
2023-01-18 10:42:36 +01:00
Jiri Pechanec
bcd6787cdc
DBZ-6008 Revert "DBZ-3653 Introduce incremental.snapshot.isolation.mode config"
...
This reverts commit 6d8bcd42fd
.
2023-01-17 10:40:20 +01:00
Jiri Pechanec
8738e62459
DBZ-6008 Revert "DBZ-3653 Remove extra modifiers"
...
This reverts commit bdfc1e6861
.
2023-01-17 10:40:20 +01:00
Jiri Pechanec
0c5e4f0051
DBZ-6002 Fix tests now requiring fractions
2023-01-13 12:00:00 +01:00
harveyyue
bea65acc0e
DBZ-6002 Run PostgresConnectorIT.shouldReceiveChangesForChangeColumnDefault() failed
2023-01-13 12:00:00 +01:00
akshansh
444dab3ffa
DBZ-5996 Handled case where fractionalWidth was -1
2023-01-11 12:09:26 +01:00
akshansh
b87d80f462
DBZ-5996 Implemented custom DateTimeFormatter to parse ZonedDateTime while preserving length of column.
2023-01-11 12:09:26 +01:00
harveyyue
5e56ad75f8
DBZ-5985 Disable to print invalid estimated row count when config "snapshot.select.statement.overrides"
2023-01-11 09:20:15 +01:00
Jiri Pechanec
bdfc1e6861
DBZ-3653 Remove extra modifiers
2023-01-11 08:55:33 +01:00
govi20
6d8bcd42fd
DBZ-3653 Introduce incremental.snapshot.isolation.mode config
2023-01-11 08:55:33 +01:00
Jiri Pechanec
8ca5e0a9b5
DBZ-4669 Reformat after plugin version consolidation
2023-01-11 06:44:53 +01:00
Mark Lambert
f0a1fa8a67
DBZ-4669 Use the newer version of plugins to consolidate remaining duplicates plugin versions
2023-01-11 06:44:53 +01:00
Mark Lambert
e5d16b14a6
DBZ-2439 Remove redundant modifiers
2023-01-11 06:44:53 +01:00
harveyyue
d941bc928b
DBZ-5966 Truncate records incompatible with ExtractNewRecordState
2023-01-05 13:51:35 +01:00
Jiri Pechanec
26bcb67cdc
DBZ-5967 Provide additional checks
2023-01-05 08:12:21 +01:00
mfvitale
c2d7d82d30
DBZ-5967: Fix negative partition number in ComputePartition SMT
...
Also improved check on configuration consistency to avoid 0 as partition number.
2023-01-05 08:12:21 +01:00
Debezium Builder
cf1720e4cb
[maven-release-plugin] prepare for next development iteration
2022-12-22 10:11:54 +00:00
Debezium Builder
b4ea9b46cd
[maven-release-plugin] prepare release v2.1.0.Final
2022-12-22 10:11:53 +00:00
harveyyue
6d70bf7b9d
DBZ-5344 Add complex json array test case to EventRouterTest
2022-12-19 12:24:24 +01:00
harveyyue
348e77d334
DBZ-5946 Print the readable data class name in JdbcValueConverters.handleUnknownData
2022-12-19 11:02:38 +01:00
Debezium Builder
177a60a9d3
[maven-release-plugin] prepare for next development iteration
2022-12-16 09:33:38 +00:00
Debezium Builder
f805db8c3c
[maven-release-plugin] prepare release v2.1.0.Beta1
2022-12-16 09:33:38 +00:00
Vojtech Juranek
1c0300cf2a
DBZ-5838 Disable schema caching
...
While documentation says that schema caching is disabled by default,
which would everyone sane assume anyway, it's actually turned of and
clean result in very strange test failures.
See also https://github.com/Apicurio/apicurio-registry/issues/2997
2022-12-12 14:44:05 +01:00
mfvitale
4c941b63ed
DBZ-5847: Add support for all connectors
...
Support for qualified table names
2022-12-12 14:40:56 +01:00
Marcelo Avancini
7754163c5d
DBZ-2432: Fixing misuse of assertj
...
- Some usages of assertj was not callig the actual assert of the DSL, fixing it
- Changing the Assertions to be staticlly imported
2022-12-08 13:10:26 +01:00
Jiri Pechanec
97ea30f25d
DBZ-5893 Mask connection string in connector option logging
2022-12-07 08:41:34 +01:00
jcechace
2468d73783
DBZ-5893 Masking credentials in connection string when logging
2022-12-07 05:45:58 +01:00
Debezium Builder
81896ed334
[maven-release-plugin] prepare for next development iteration
2022-11-30 07:52:24 +00:00
Debezium Builder
1aac97b87c
[maven-release-plugin] prepare release v2.1.0.Alpha2
2022-11-30 07:52:23 +00:00
Jiri Pechanec
f50ed10377
DBZ-5796 Fix typo in test description
2022-11-16 14:55:01 +01:00
harveyyue
b124d91a93
DBZ-5796 For outbox transformation, when 'table.expand.json.payload' is set to true null values are not correctly deserialized
2022-11-16 14:55:01 +01:00
Debezium Builder
e311dd1a67
[maven-release-plugin] prepare for next development iteration
2022-11-10 13:29:06 +00:00
Debezium Builder
e3bf53440a
[maven-release-plugin] prepare release v2.1.0.Alpha1
2022-11-10 13:29:05 +00:00
Anil Dasari
c781832f41
DBZ-5822 Change visibility of BaseSourceTask#logStatistics method to protected
2022-11-10 08:02:48 +01:00
Chris Cranford
1d9a21bb85
DBZ-5759 Log per topic record counts
2022-11-10 07:31:28 +01:00
Vojtech Juranek
96e4326d35
DBZ-2131 Use system property to determine if Apicurio should be used
2022-11-09 08:44:45 +01:00
Vojtech Juranek
41d3d7d155
DBZ-2131 Run tests with Apicurio registry
...
When Apicurio profile is used and container with Apicurio started, use
Apicurio registry in the tests. Fall back to in-memory Confluent
registry otherwise.
2022-11-09 08:44:45 +01:00
harveyyue
e6f0551451
DBZ-5801 The merge method of configuration is not work
2022-11-08 14:19:51 +01:00
jcechace
f46c127d11
DBZ-5779 Migrated debezium-core to AssertJ
2022-11-02 08:50:26 +01:00
Chris Cranford
dc5de5ae87
DBZ-5760 Only log full row data at TRACE level
2022-10-26 08:13:34 +02:00
Chris Cranford
c9084dd3f7
DBZ-5615 Mask apicurio registry credentials
2022-10-25 05:48:54 +02:00
Chris Cranford
a0299ec594
DBZ-5615 Mask schema registry credentials
2022-10-25 05:48:54 +02:00
Jiri Pechanec
ad2733eb1a
DBZ-5727 Simplify checking of column filter presence
2022-10-20 12:47:47 +02:00
harveyyue
0e9bd53629
DBZ-5727 Columns are not excluded when doing incremental snapshots
2022-10-20 12:47:47 +02:00
Jiri Pechanec
8a02907576
DBZ-5610 Truncate events are not emitted by every connector
2022-10-20 08:57:28 +02:00
harveyyue
a37d3332df
DBZ-5610 MySQL Connector capture TRUNCATE command as message in table topic
2022-10-20 08:57:28 +02:00
Debezium Builder
4d35fb8011
[maven-release-plugin] prepare for next development iteration
2022-10-14 11:26:32 +00:00
Debezium Builder
7bd1551c99
[maven-release-plugin] prepare release v2.0.0.Final
2022-10-14 11:26:32 +00:00
harveyyue
299f93afd7
DBZ-5709 The DDL_FILTER of SchemaHistory doesn't work for including break lines ddl statement
2022-10-14 09:21:40 +02:00
Vojtech Juranek
820b9d10d5
DBZ-5329 Add LSN to Postgres transation ID
...
Add `EventDispatcher` constructor which accepts `TransactionMonior`
instance as a parameter and in case of Postgres pass into
`EventDispatcher` custom `PostgresTransactionMonitor` which adjusts
transactions IDs by adding LSN, i.e. Postgres transaction is now of
form `txId:LSN`.
2022-10-11 15:38:33 +02:00
Vojtech Juranek
4554a63afe
DBZ-5329 Allow override transaction message key/value
...
Allow subclasses override transaction metadata keys and values.
2022-10-11 15:38:33 +02:00
Debezium Builder
884254aba1
[maven-release-plugin] prepare for next development iteration
2022-10-07 09:27:52 +00:00
Debezium Builder
a0f3263fa6
[maven-release-plugin] prepare release v2.0.0.CR1
2022-10-07 09:27:51 +00:00
Jiri Pechanec
99dff302f1
DBZ-5612 Support for formatted messages
2022-10-04 13:45:43 +02:00
rajdangwal
58ab632ea3
DBZ-5612 log records only at trace level
2022-10-04 13:45:43 +02:00
rajdangwal
c03bcf8606
DBZ-5612 Remove records from being logged at all levels
2022-10-04 13:45:43 +02:00
harveyyue
f6b72f060c
DBZ-5671 Remove logic name parameter from sub connector config
2022-10-03 11:40:17 -04:00
Jiri Pechanec
7d2cf85279
DBZ-5541 Change default in the description
2022-09-29 09:47:25 +02:00
Sergei Morozov
e4d46c81b1
DBZ-5541: Default Debezium schema.name.adjustment.mode to "none"
2022-09-29 09:47:25 +02:00
harveyyue
14b08c1748
DBZ-5654 Outbox pattern nested payload leads to connector crash
2022-09-28 10:20:22 -04:00
harveyyue
d1153e8ad4
DBZ-5594 Clean up "logical name" config
2022-09-22 14:29:37 +02:00
Jiri Pechanec
70d5b2db50
DBZ-5639 Inform about snapshot reexcution
2022-09-22 13:14:28 +02:00
harveyyue
3b4d19c174
DBZ-5639 Logging enhancement for non-incremental snapshot in postgres connector
2022-09-22 13:14:28 +02:00
Ismail Simsek
b966f4b65b
DBZ-5591 Snapshot process setting source.ts_ms
2022-09-20 17:12:54 -04:00
Jiri Pechanec
29fcb0e047
DBZ-5585 Add last resort classloader
2022-09-19 17:15:29 +02:00
harveyyue
288c93434f
DBZ-5585 Remove classloader parameter from JdbcConnection
2022-09-19 17:15:29 +02:00
harveyyue
20d56734d4
DBZ-5585 Modify the Instantiator to not require classloader
2022-09-19 17:15:29 +02:00
Vojtech Juranek
2a0e2a7e81
DBZ-5603 Move test containers to quay.io
2022-09-19 09:12:46 +02:00
Debezium Builder
218d414552
[maven-release-plugin] prepare for next development iteration
2022-09-16 06:36:40 +00:00
Debezium Builder
567f57c47b
[maven-release-plugin] prepare release v2.0.0.Beta2
2022-09-16 06:36:39 +00:00
harveyyue
09b99840d8
DBZ-4617 Deprecate internal key/value converter options
2022-09-14 12:49:30 +02:00
Vojtech Juranek
8b8d4828f3
DBZ-5043 Make topic prefix field manadatory and with proper group
2022-09-14 12:44:38 +02:00
harveyyue
665edab844
DBZ-5589 Mysql connector can't handle the case sensitive of rename/change column statement
2022-09-13 09:36:25 +02:00
Vojtech Juranek
acc6be6df3
DBZ-5043 Move TOPIC_PREFIX into CommonConnectorConfig
...
`TOPIC_PREFIX` is now mandatory to all connectors therefore it make
sense to have it in common config. Beside that, it also makes it more
easy to use it in Debezium UI without any workarounds - if the field is
not member of the given connector config, the field has to be
explicitely added into known fields otherwise is invisible for UI.
2022-09-13 09:25:54 +02:00
Anisha Mohanty
f41b6dd458
DBZ-5044 Get rid of schema factory instances
2022-09-08 10:35:20 +02:00
Anisha Mohanty
b0dc7e557a
DBZ-5044 Update schemaHistoryColumnSchema to add new fields
2022-09-08 10:35:20 +02:00
Anisha Mohanty
ec792527e9
DBZ-5044 Use SchemaFactory for centralizing Postgres schemas
2022-09-08 10:35:20 +02:00
Anisha Mohanty
cfd0242e28
DBZ-5044 Sort imports & fix test failures
2022-09-08 10:35:20 +02:00
Anisha Mohanty
91ccb7578f
DBZ-5044 Centralize datatype schemas
2022-09-08 10:35:20 +02:00
Jiri Pechanec
77efd818da
DBZ-5044 More descriptive names for schema history schemas
2022-09-08 10:35:20 +02:00
Jiri Pechanec
b8bcac1aef
DBZ-5044 Ignore versions for schema comparison
2022-09-08 10:35:20 +02:00
Jiri Pechanec
fa2a7ba60c
DBZ-5044 Fix transaction metadata schema
2022-09-08 10:35:20 +02:00
Anisha Mohanty
aea7c1ffa6
DBZ-5044 Use SchemaFactory for ConnectTable & EvenyDispatcher schemas
2022-09-08 10:35:20 +02:00
Anisha Mohanty
8fb2ffd70a
DBZ-5044 Add review suggestions
2022-09-08 10:35:20 +02:00
Jiri Pechanec
cd07b8a06d
DBZ-5044 Rename and reorder schema attributes
2022-09-08 10:35:20 +02:00
Anisha Mohanty
1a55869f5b
DBZ-5044 Reword schema names
2022-09-08 10:35:20 +02:00
Anisha Mohanty
867ff0a117
DBZ-5044 Use seperate version for each schema
2022-09-08 10:35:20 +02:00
Anisha Mohanty
ec0ac3fea5
DBZ-5044 Centralize creation of Message Schemas
2022-09-08 10:35:20 +02:00
Vojtech Juranek
402df78381
DBZ-5043 Rename schema.history to schema.history.internal
2022-09-07 12:57:31 +02:00
Vojtech Juranek
b6c31f3da5
DBZ-5043 Replace 'database history' with 'database schema history'
...
Reflect renaming database history to schema history in comments,
configuration descriptions and log and error messages.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
5bffd7839a
DBZ-5043 Change default schema history suffix
...
Change default schema history name from `$LOGICAL_NAME-dbhistory` to
`$LOGICAL_NAME-schemahistory`.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0494a16a79
DBZ-5043 Rename DatabaseHistory to SchemaHistory
...
Rename also all related classes and varibales.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0823cedf25
DBZ-5043 Replace database.server.name with topic.prefix
2022-09-07 12:57:31 +02:00
Vojtech Juranek
a7a293c5c8
DBZ-5043 Move driver pass-through parameters into dedicated namespace
...
Currently we mix database parameters like DB hostname, username or
password with other JDBC driver parameters which are passed to JDBC
driver as it is. Move these pass-through parameters into dedicated
namespace `driver.*` to separate them from common database parameters.
JDBC parameters which are defined on the connector level still stay
in `database` namespace.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
5d1b72e95f
DBZ-5043 Simplify code after renaming database.history namespace
...
Filtering `database.history` namespace from pass-through parameters is now
not needed as it doesn't interfere with other `database.*` parameters.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
25aa6c4acc
DBZ-5043 Change namespace to database.history
to schema.history
2022-09-07 12:57:31 +02:00
harveyyue
48d37857b6
DBZ-5588 Field validation errors are misleading for positive, non-zero expectations
2022-09-07 09:18:38 +02:00
harveyyue
58f4d3d912
DBZ-5480 Augment a uniqueness key filed/value in regex topic naming strategy
2022-09-06 14:58:57 +02:00
Claus Ibsen
1f48bfc515
[docs] Typo in description (remove space before dot)
2022-09-06 09:48:57 +02:00
Alexey Loubyansky
c4b95f4ea2
DBZ-5561 Use TCCL to load configuration classes by default
2022-09-06 08:42:45 +02:00
Vadzim Ramanenka
917de37a01
DBZ-5557: Supply partition when comiting offsets with source database
...
Keep track of partitions offsets being committed belong to in
`BaseSourceTask`. Supply the partition along with the offset in calls to
`commitOffset`.
2022-09-05 14:04:23 +02:00
Jiri Pechanec
6e53798407
DBZ-5538 Remove duplicated formatting message
2022-09-05 10:35:55 +02:00
harveyyue
f4c7b52451
DBZ-5538 Unclear validation error when required field is missing
2022-09-05 10:35:55 +02:00
ggaborg
664fd9e1b0
DBZ-3293 unit tests for filters
2022-09-01 20:16:47 +02:00
ggaborg
244de2bd96
DBZ-3293 Add signal table automatically to include list
2022-09-01 20:16:47 +02:00
thangdc94
57210095d2
DBZ-5439 Fix testcase
2022-09-01 09:32:17 +02:00
Jiri Pechanec
de53591dba
DBZ-5244 Retry all communication exceptions by default
2022-08-31 09:45:09 -04:00
harveyyue
7ccd40129c
DBZ-5569 Oracle connector's max.queue.size.in.bytes is invalid
2022-08-31 09:39:29 -04:00
Jiri Pechanec
1c291d75fd
DBZ-5568 Use only Java 11 for tests
2022-08-31 09:19:23 +02:00
pkgonan
fd309193c6
DBZ-5544 Add document
2022-08-24 09:50:23 +02:00
pkgonan
b031770300
DBZ-5544 Support BASE64_URL_SAFE in BinaryHandlingMode
2022-08-24 09:50:23 +02:00
tjwornjs
ca009d8492
DBZ-5489 add column comment to schema
2022-08-24 09:46:01 +02:00
Anisha Mohanty
669024ed46
DBZ-5484 Better error reporting in SKIP/WARN cases
2022-08-23 10:02:29 -04:00
harveyyue
0308a1cbb6
DBZ-5535 Disable preferring DDL before logical schema in history recovery
2022-08-23 14:16:35 +02:00
Jiri Pechanec
34b78e6bce
DBZ-2766 Make queue more generic
2022-08-23 13:43:41 +02:00
Jiri Pechanec
f770e0afb1
DBZ-2766 Remove Guava dependency
2022-08-23 13:43:41 +02:00
ggaborg
bf20a9c70f
DBZ-5533 Unit test fails on Windows
2022-08-19 09:44:26 -04:00
Sergei Morozov
1c99437890
DBZ-5470: Remove the dependency of JdbcConnection on DatabaseSchema
2022-08-19 11:06:59 +02:00
Chris Cranford
37b97970a6
DBZ-5497 Skip truncate operations by default
...
* Promote truncate skip default from PostgreSQL to core library
* Adjust behavior for Oracle with truncates skipped by default
2022-08-19 11:05:29 +02:00
Vivek Wassan
d89e71ebeb
DBZ-5327 Smart Backfills
2022-08-17 10:05:55 +02:00
harveyyue
a08420b21c
DBZ-5511 Keep consistence with method serialize of Connect/Json Table Change Serializer
2022-08-16 16:58:07 +02:00
Vojtech Juranek
8e0ad245e5
DBZ-5475 Ensure Outbox JSON NUMBER arrays are of the same type
2022-08-05 11:00:46 -04:00
Vojtech Juranek
7256f62f6b
DBZ-5475 Remove unneeded checks from Outbox payload conversion
...
* remove float check as we return `float64` by default anyway
* simplify/narrow the checks when detemining array type, also add a test
for array having the first element `null`
2022-08-05 11:00:46 -04:00
Chris Cranford
d4261bd2fd
DBZ-5476 Fix record accumulator during statistics log entry
2022-08-04 15:28:46 -04:00
harveyyue
2481172309
DBZ-5471 Missing the regex properties validation before start connector of DefaultRegexTopicNamingStrategy
2022-08-04 10:49:29 -04:00
Vojtech Juranek
c7d8353d7b
DBZ-4727 Add snapshot paused JXM metrics
...
Add two JXM objects:
* `snapshotPaused` - detemines if the incremental snapshot is paused
or not
* `snapshotPausedDurationInSeconds` - overall time when the incremental
snapshot was paused. The time adds up - if the snapshot was paused
e.g. two times, the `snapshotPausedDurationInSeconds` is the sum of
these two paused times.
2022-07-29 12:41:58 -04:00
Vojtech Juranek
ed504173e1
DBZ-4727 Implement incremental snaphost pause/resume
...
Intorduce two new signals:
* `pause-snapshot` - pauses running incremental snapshot
* `resume-snapshot` - resumes paused incremental snapshot
If the incremental snapshot is running and pause signal is sent,
currently processed chunk is finished and then no other chunk is read.
Snapshot state is kept in running state. Once snapshot is resume,
chunk is reverted to position which was last sent to the broker and
process of reading chunks and emitting `read` events continues until
all tables are read and sent.
This patch handles only table-based signals. MySQL Kafka-based signals
will be addressed later.
2022-07-29 12:41:58 -04:00
harveyyue
5f430f1a8e
DBZ-5457 Deprecate legacy topic selector for all connectors
2022-07-28 12:58:59 -04:00
ggaborg
2c8d71a36f
DBZ-5452 Fix unit test failing on Windows platform
2022-07-26 15:50:28 -04:00
Debezium Builder
1ab51ae8d0
[maven-release-plugin] prepare for next development iteration
2022-07-26 18:13:10 +00:00
Debezium Builder
d247a48b79
[maven-release-plugin] prepare release v2.0.0.Beta1
2022-07-26 18:13:09 +00:00
harveyyue
7730610320
DBZ-5425 Remove the duplicated SimpleDdlParserListener from mysql connector
2022-07-20 14:35:34 +02:00
Chris Cranford
75ad893d67
DBZ-5396 Scope attributes inside table document
2022-07-19 15:41:02 -04:00
Chris Cranford
76dafedea7
DBZ-5396 Add multi-type accessors for attribute values
2022-07-19 15:41:02 -04:00
Chris Cranford
f412c2e6a4
DBZ-5396 Add support for connector-specific relational model attributes
2022-07-19 15:41:02 -04:00
harveyyue
daf329bc06
DBZ-5413 Update the topic naming strategy doc to all connectors
2022-07-18 10:18:12 -04:00
harveyyue
77c7db163d
DBZ-5414 Validate topic naming strategy relative topic name properties
2022-07-18 09:58:51 -04:00
Chris Cranford
3acf317668
DBZ-5404 Restore build compatibility with Kafka 2.x and 3.0.x
2022-07-14 08:40:42 +02:00
harveyyue
6ea7abc8fc
DBZ-4180 Pluggable topic selector
2022-07-13 14:45:25 +02:00
Jiri Pechanec
d8baac7c46
DBZ-5229 Reorganize code for inheritance use
2022-07-12 17:31:19 +02:00
Hossein Torabi
1ea02be9c2
DBZ-5229 define centralized and modular aproach for debezium storage
2022-07-12 17:31:19 +02:00
Chris Cranford
d2a76a7ffc
DBZ-5295 Reselect LOB columns on primary key update
2022-07-12 11:59:34 +02:00
Chris Cranford
7b08fb8d3d
DBZ-5356 Rewrite loop
2022-07-12 11:01:10 +02:00
Chris Cranford
deaa34d71e
DBZ-5356 Avoid NPE during index evaluation
2022-07-12 11:01:10 +02:00
harveyyue
0842e5cd0e
DBZ-5366 MySQL Connector column hash v2 does not work
2022-07-06 14:23:57 -04:00
Paweł Malon
707a43d3bc
DBZ-5367 Fixed the problem where JSON expanding failed for Outbox event payloads that contained nested arrays where first array was empty and further array contained a Struct
...
With proposed changes empty arrays are skipped, as it allows us to delay defining the schema for them until an array with elements is found, as it seems that there is no way to modify a schema that was already defined for a given field
2022-07-06 13:47:54 -04:00
Debezium Builder
6fd1441af3
[maven-release-plugin] prepare for next development iteration
2022-07-01 09:21:49 +00:00
Debezium Builder
99402af634
[maven-release-plugin] prepare release v2.0.0.Alpha3
2022-07-01 09:21:48 +00:00
Vojtech Juranek
ac06623459
DBZ-5240 Move table types into separate method
...
This will allow connectors to override the default values and return
types the connector supports.
2022-06-30 10:46:54 +02:00
Gunnar Morling
cb53332561
DBZ-5335 Formatting
2022-06-30 09:54:13 +02:00
Mickael Maison
5ff71aa992
DBZ-5335 Fix formatting issues in the connectors configdefs
2022-06-30 09:35:54 +02:00
harveyyue
20dc353262
DBZ-5170 Mysql Commit Timestamp
2022-06-27 15:50:52 +02:00
Naveen Kumar KR
b75c9e635b
DBZ-5277 ChangeEventQueue - Support Multiple Writer (doEnqueue) and Reader (poll) Threads
...
* ReentrantLock based write and read coordination
* Condition based Inter Thread communication for isFull and isNotFull conditions
2022-06-27 15:37:00 +02:00
Naveen Kumar KR
823a49fff4
DBZ-5277 ChangeEventQueue Test with Multiple Writer and Reader Threads
...
Assertion would fail if one or both of the scenario occurs
1) Deadlock between Writer and Reader Threads
2) Thread Starvation between Writer and Reader Threads
2022-06-27 15:37:00 +02:00
Jiri Pechanec
9ada2f28ce
DBZ-4902 Change for exponential strategy
2022-06-27 14:29:51 +02:00
kanha gupta
1b17372d2d
DBZ-4902 Replace long durations with Duration
2022-06-27 14:29:51 +02:00
Chris Cranford
ef99595d91
DBZ-5270 Reduce table schema capture logging verbosity
...
When enabling the `database.history.store.only.captured.tables.ddl`,
the logging in RelationalSnapshotChangeEventSource does not take into
account this setting when logging, leading to a false impression of
what tables will actually be captured in the schema history.
2022-06-27 08:47:34 +02:00
Jiri Pechanec
5e1ae24e92
DBZ-5272 Notify all threads in case of mutliple writers
2022-06-21 07:08:39 +02:00
Chris Cranford
3fb5bf1339
DBZ-4451 Introduce SchemaChangeEvent factory methods
2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081
DBZ-4451 Store previous table id in alter schema change events
...
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5
DBZ-4834 Fix incremental snapshot of table added to include list
2022-06-16 08:39:16 +02:00
Vojtech Juranek
25eba9749d
DBZ-5198 Add support for SQL server specific table delimiters
...
Add suppoprt for `TableId` delimiters and provide implementation for
SQL server. SQL server allows to use reserved words in table names or
names with spaces, but they have to wrapped by `[]` characters, e.g.
`[dbname].[table name]`.
Debezium can handle spaces e.g. in table include list, but fails when
parsing snapshot select, therefore the parsing with predicates is used
only for parsing spanshot select for now. If useful, it can be used
later on on other places as well.
2022-06-13 10:57:15 +02:00
Vojtech Juranek
796e17b2c6
DBZ-5198 Move predicates for parsing TableId into dedicate class
...
Moving the predicates into parsing context decouples the predicates from
the private `TableIdTokenizer` class. Moving the predicates into
separate class would allow us to provide database specific
implementations.
2022-06-13 10:57:15 +02:00
Debezium Builder
6fff3a2876
[maven-release-plugin] prepare for next development iteration
2022-06-09 11:49:26 +00:00
Debezium Builder
b2e2e351df
[maven-release-plugin] prepare release v2.0.0.Alpha2
2022-06-09 11:49:26 +00:00
Vojtech Juranek
4a48eb33df
DBZ-5045 Remove legacy V1 source info
2022-06-08 09:11:00 +02:00
Vojtech Juranek
250e160b81
DBZ-5045 Removed unused methods
2022-06-08 09:11:00 +02:00
Vojtech Juranek
ef8d7f7af7
DBZ-5045 Remove deprecated OBSOLETE_NAME_FOR_SKIP_FAILURE_HANDLING option value
2022-06-08 09:11:00 +02:00
Vojtech Juranek
ff5ddd7959
DBZ-5045 Remove deprecated FIELD_PAYLOAD_ID
...
`table.field.event.payload.id` is replaced by `table.field.event.key`.
Also fix default value of `table.field.event.key` which was missing.
2022-06-08 09:11:00 +02:00
Vojtech Juranek
0530dc2de9
DBZ-5045 Remove deprecated STORE_ONLY_MONITORED_TABLES_DDL
2022-06-08 09:11:00 +02:00
Vojtech Juranek
4e7b30b718
DBZ-5045 Remove deprecated DEBEZIUM_OPERATION_INVALID_BEHAVIOR
2022-06-08 09:11:00 +02:00
Vojtech Juranek
b0b3066ed5
DBZ-5045 Remove deprecated COLUMN_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
5031594b75
DBZ-5045 Remove deprecated COLUMN_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
b3a6131cee
DBZ-5045 Remove deprecated DATABASE_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
0cbb57f4bf
DBZ-5045 Remove deprecated DATABASE_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
f84893a8b4
DBZ-5045 Remove deprecated SCHEMA_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
261188482f
DBZ-5045 Remove deprecated SCHEMA_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
6abbcee6a3
DBZ-5045 Remove deprecated TABLE_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
db08e894fc
DBZ-5045 Remove deprecated TABLE_WHITELIST
2022-06-08 09:11:00 +02:00
harveyyue
acb743bd34
DBZ-5206 Remove the deprecated monitored tables metrics
2022-06-07 16:13:29 -04:00
Chris Cranford
3e820c4047
DBZ-4251 Adjust semantics with collection removal
2022-06-02 07:30:07 +02:00
Chris Cranford
187e3250a3
DBZ-4251 Add regex support for stop signal
2022-06-02 07:30:07 +02:00
Chris Cranford
9d71296b5c
DBZ-4251 Support stopping incremental snapshots
2022-06-02 07:30:07 +02:00
Jiri Pechanec
88b7ae2050
DBZ-5136 Move sourceInfo to common class
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
c64b8af31a
DBZ-5136 reintroduce the CommonOffsetContext class and refactor common functionality into it
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
2d72dae649
DBZ-5136 further refining of the algorithm for determining position in snapshot and IT test fixes
2022-06-01 16:56:29 +02:00
Jiri Pechanec
08e8c3d795
DBZ-5136 Move common class to common package
2022-06-01 16:56:29 +02:00
Andrew Walker
d0759461a4
DBZ-5136: Convert RelationalOffsetContext to CommonOffsetContext
2022-06-01 16:56:29 +02:00
Andrew Walker
01a81f8bc1
DBZ-5136: Refactored SnapshotContext setters
2022-06-01 16:56:29 +02:00