Commit Graph

8506 Commits

Author SHA1 Message Date
mfvitale
7ba9f57035 DBZ-6496 Fix signal.poll.interval.ms default value 2023-05-24 18:51:49 +02:00
dependabot[bot]
f6fa87ce1f [ci] Bump sqlite-jdbc in /debezium-storage/debezium-storage-jdbc
Bumps [sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.36.0.3 to 3.41.2.2.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.36.0.3...3.41.2.2)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-24 07:35:04 +02:00
mfvitale
547ecd02ca DBZ-6447 Fix incremental snapshot notification on MongoDb connector 2023-05-23 09:57:05 +02:00
mfvitale
dc00f64bbf DBZ-6447 Incremental snapshot notification ids are now equal to the id sent in the execute-snapshot signal. 2023-05-23 09:57:05 +02:00
harveyyue
016daf6fd8 DBZ-6490 Convert the field BigDecimal value to bytes failure of ApproximateStructSizeCalculator 2023-05-23 08:38:06 +02:00
ani-sha
f20fe32b4f DBZ-6492 Support logical_replication_clause in Oracle 2023-05-23 08:30:36 +02:00
ani-sha
6d404d242e DBZ-6479 Add missing privileges as per MySQL 8.0.x 2023-05-23 08:25:49 +02:00
Martin Medek
97a6171536 DBZ-3202 Added postgresql replica DB instance and test case for debezium streaming from replica 2023-05-22 09:30:10 +02:00
Kanthi Subramanian
e1fd00b580 DBZ-6476 Change encoding of JDBC offset data from UTF-16 to UTF-8 2023-05-22 08:25:11 +02:00
Jiri Pechanec
d5a3095717 DBZ-6457 Fix formatting 2023-05-19 14:04:14 +02:00
Chris Cranford
d40da4f810 DBZ-6457 Use PDB for parallel snapshot connections 2023-05-19 14:04:14 +02:00
ani-sha
e91f483062 DBZ-5339 Enable Jolokia using DebeziumContainer class 2023-05-18 11:08:03 -04:00
Vojtech Juranek
a58f4b4d4f DBZ-6439 Filter tables only when user want to capture schema for included tables only 2023-05-16 14:48:48 -04:00
angsdey2
80c910c652 DBZ-6470 Refer correctly to the source database
While creating signaling table we need to have privileges to create table in source database not target
2023-05-16 13:16:09 +02:00
Jiri Pechanec
d992ae5f4e DBZ-6461 Rename to delivery timeout; Increase default value 2023-05-16 13:03:49 +02:00
mfvitale
d08e72fda7 DBZ-6461 Document new debezium.sink.pubsub.wait.message.computation.timeout.ms and debezium.sink.pubsublite.wait.message.computation.timeout.ms properties to configure the wait time on message processing by Google PubSub 2023-05-16 13:03:49 +02:00
Ismail Simsek
2683b31394 DBZ-4027 implement FileSignalChannel 2023-05-15 15:34:43 +02:00
Jochen Schalanda
bdb43a2235 DBZ-6459 [MariaDB] Add support for userstat plugin keywords
The MariaDB User Statistics (userstat) plugin adds 4 new keywords:

`USER_STATISTICS`, `CLIENT_STATISTICS`, `INDEX_STATISTICS`, and `TABLE_STATISTICS`

Additionally, it adds unusual variants of the `SHOW` and `FLUSH` statements which aren't covered by the MariaDB SQL grammar yet, for example

```
SHOW TABLE_STATISTICS;
FLUSH USER_STATISTICS;
```

See also:

- https://mariadb.com/kb/en/user-statistics/
- https://mariadb.com/kb/en/show-client-statistics/
- https://mariadb.com/kb/en/show-index-statistics/
- https://mariadb.com/kb/en/show-table-statistics/
- https://mariadb.com/kb/en/show-user-statistics/

Refs https://github.com/antlr/grammars-v4/pull/3400
2023-05-15 14:56:36 +02:00
Vojtech Juranek
ee4ebaa957 DBZ-6439 Don't scan all the tables when loading schema
Currently we scan all the tables, which may result into a substantial
delay in initial snapshot when the database is very large. We need to
filter out tables which we are not interested in.

Add back table filter when loading schema of tables. As per comment of
this block of code, passing all tables and table filter should be faster
than passing only list of tables we are interested in.
2023-05-15 14:11:47 +02:00
Vojtech Juranek
f4596fa2e0 DBZ-6460 Enable setting log level in tests via log interceptor 2023-05-15 14:11:47 +02:00
jcechace
2de3f0d9a4 DBZ-6421 Ability to recover from cursor which doesn't return resume token when there is no change document (e.g. in case of DocumentDB connection) 2023-05-15 14:08:57 +02:00
jcechace
5dfe52dbc4 DBZ-6421 Streaming pipeline is now compatible with DocumentDB 2023-05-15 14:08:57 +02:00
jcechace
dad7d0ce14 DBZ-6421 Ability to run tests against external deployment of MongoDB and AWS DocumentDB specifically 2023-05-15 14:08:57 +02:00
mfvitale
85278e76af DBZ-6462 Stop SignalProcessor after test execution in SignalProcessorTest 2023-05-15 08:51:02 +02:00
mfvitale
e41bbe7191 DBZ-6462 Schedule SignalProcessor task with a more relaxed scheduling. Removes parallelStreams while processing signalChannelReaders 2023-05-15 08:51:02 +02:00
Jiri Pechanec
bd0c760d9f [jenkins-jobs] Remove obsolete Jira check 2023-05-11 14:46:03 +02:00
Debezium Builder
34d4a73829 [release] Development version for testing module deps 2023-05-11 11:13:35 +00:00
Debezium Builder
a5b44767bc [maven-release-plugin] prepare for next development iteration 2023-05-11 11:05:24 +00:00
Debezium Builder
1b1a7531f2 [maven-release-plugin] prepare release v2.3.0.Alpha1 2023-05-11 11:05:24 +00:00
Debezium Builder
7ba7d07692 [release] Stable 2.3.0.Alpha1 for testing module deps 2023-05-11 11:03:11 +00:00
Jiri Pechanec
ff85a7839a [release] Changelog for 2.3.0.Alpha1 2023-05-11 12:52:03 +02:00
Jiri Pechanec
b57ac1ca5a DBZ-1973 Guard against uninited fields in Xstream 2023-05-11 11:13:38 +02:00
Jiri Pechanec
ac24e54332 DBZ-6437 Modify parent for server BOM during release 2023-05-11 06:50:50 +02:00
mfvitale
016785b141 DBZ-6443 Add step to build Debezium Connector MySQL in store build job 2023-05-10 14:08:01 +02:00
mfvitale
7429b4e6f1 DBZ-1973 Add incremental snapshot notifications 2023-05-10 12:22:02 +02:00
mfvitale
811b800f55 DBZ-1973 Add support for Notifications
DBZ-1973 Add more tests for other connectors

DBZ-1973 Add send method with offset parameter

DBZ-1973 Instantiate NotificationService in the task class

DBZ-1973 Instantiate NotificationService in the task class

DBZ-4027 Move specific sink channel configuration to SinkNotificationChannel

DBZ-4027 Remove not used SPI file

DBZ-1973 Moved SPI file definition to debezium-core

DBZ-1973 Rename KafkaNotificationChannel to more generic SinkNotificationChannel

DBZ-1973 Code refactor

DBZ-1973 Improve configuration property description

DBZ-1973 Improve test

DBZ-1973 Add close method to NotificationChannel

DBZ-1973 Implement KafkaNotificationChannel

DBZ-1973 Add NotificationService and LogNotificationChannel
2023-05-10 12:22:02 +02:00
Jiri Pechanec
f7fd293b42 [ci] Execute release from candidate branch 2023-05-10 12:14:12 +02:00
Chris Cranford
1a44431114 DBZ-6442 Support FORCE-clause in DROP TABLE 2023-05-10 12:10:15 +02:00
gongchanghua
38cdcb2152 DBZ-6446 Support shrink clause for Oracle index partition 2023-05-09 12:15:15 +02:00
Chris Cranford
fc4a31e3d7 DBZ-6436 Improve Oracle LogMiner failure logging
This change explicitly captures the state of two critical tables when the connector
fails to start a LogMiner session, namely V$LOGMNR_LOGS and V$LOGMNR_PARAMETERS.
While these were previously captured by the LogMinerDatabaseStateWriter, the old
code required DEBUG log level and that isn't always feasible, so this change will
capture these in ERROR log level regardless moving forward.
2023-05-09 12:09:36 +02:00
Chris Cranford
c16df4f2c4 DBZ-6254 Document new configuration options 2023-05-09 11:31:08 +02:00
Chris Cranford
30eb53c764 DBZ-6254 Add LogMiner Query and Username Include Filtering
* Add `log.mining.query.filter.mode` configuration option
* Add `log.mining.username.include.list` configuration option
2023-05-09 11:31:08 +02:00
kate
6c338754b3 DBZ-6440 Init Kafka signal channel 2023-05-09 06:48:39 +02:00
Kanthi Subramanian
f70b215b51 DBZ-6433 Include column names in default insert for JDBC schema history. 2023-05-05 13:41:44 +02:00
Kanthi Subramanian
e97e54352f DBZ-6433 Make insert configurable for JDBC Schema history. 2023-05-05 13:41:44 +02:00
Kanthi Subramanian
7b146b094d DBZ-6433 Make insert and delete queries configurable for JDBC offset storage. 2023-05-05 13:41:44 +02:00
mfvitale
0ca87d8031 DBZ-4027 During event dispatch process only signal from source channel 2023-05-05 13:35:40 +02:00
mfvitale
952340286e DBZ-4027 Make signalling channel configurable
DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Add an Incremental snapshot test with kafka signaling

DBZ-4027 Code style

DBZ-4027 Make SignalPayload more generic and extensible

DBZ-4027 Rename DatabaseSignalChannel to SourceSignalChannel

DBZ-4027 Improve logging

DBZ-4027 Moved SPI file definition to debezium-core

DBZ-4027 Move SignalProcessor synchronization point to be processed only when a signal cdc event arrives.

DBZ-4027 Add EventDispatcher constructor without signalProcessor for spanner connector

DBZ-4027 Fix NPE

DBZ-4027 Fix NPE

DBZ-4027 Formatting

DBZ-4027 Correctly manage signal on not supported connector

DBZ-4027 Use the correct MongoDbOffset

DBZ-4027 Correctly initialize offset for Oracle and SqlServer connectors

DBZ-4027 Register SPI implementations

DBZ-4027 Improve SignalProcessor instantiation

DBZ-4027 Pass source info in case of SchemaChanges action

DBZ-4027 Manage close event in a synchronous way

DBZ-4027 Correctly init offset context also in case of snapshot mode 'never'

DBZ-4027 Fix MySqlMetricsIT test

DBZ-4027 Move KafkaSignalChannel to core

DBZ-4027 Move KafkaSignalChannel to core

DBZ-4027 Set pass offset context after initial snapshot to SignalProcessor

DBZ-4027 Pass OffsetContext to signal processor

DBZ-4027 Pass CommonConnectorConfig to SignalChannelReader init method

DBZ-4027 Move Incremental snapshot window actions to dedicated package

DBZ-4027 Align SignalsIT test with new code

DBZ-4027 Fix SignalsIT test

DBZ-4027 Fix SignalProcessor scheduling

DBZ-4027 Moved DatabaseSignalChannel and SignalChannelReader to dedicated package

DBZ-4027 Start SignalProcessor from ChangeEventSourceCoordinator

DBZ-4027 Create SignalProcessor and renamed Signal to DatabaseSignalChannel

DBZ-4027 Initial refactoring of signal feature
2023-05-05 13:35:40 +02:00
Jiri Pechanec
58ef4f0b98 DBZ-6157 Disable potentially dangerous MySQL JDBC props 2023-05-04 11:54:04 +02:00
Stephen Clarkson
4be4f5c874 [docs] resolve grammatical error with note
Property `skip.messages.without.change` in Postgresql Connector documentation has a note with a grammatical error.
2023-05-04 06:11:05 +02:00