Commit Graph

9142 Commits

Author SHA1 Message Date
Chris Cranford
35a536aefc DBZ-7085 Apply MariaDB unique fetchSize requirements
Per https://jira.mariadb.org/browse/CONJ-977, the MariaDB team does not
believe that the Integer.MIN_VALUE solution for triggering streaming is
spec compliant and as of MariaDB 3.x+ drivers, they explicitly will not
allow this.

This change explicitly sets the fetchSize to 1 to mimic the same style
of behavior that the default fetchSize uses when using MySQL. If a user
provides a custom fetchSize, that will take precedences.
2023-11-09 10:00:33 +01:00
Chris Cranford
47df821c6d DBZ-7085 Don't use MySQL test artifacts in JDBC storage tests 2023-11-09 10:00:33 +01:00
Chris Cranford
b849effaf9 DBZ-7085 Add MariaDB and use it with MariaDB profile 2023-11-09 10:00:33 +01:00
Chris Cranford
0f4d2f2b1a DBZ-7085 Fix test suite to protocol-based factory URL 2023-11-09 10:00:33 +01:00
ani-sha
700942b258 DBZ-6874 Drop MySQL 5.7, test MySQL 8.0 & adjust documentation 2023-11-09 09:26:26 +01:00
Chris Cranford
7532e52228 DBZ-7120 Add a connection max retry limit to Redis schema history 2023-11-09 07:59:58 +01:00
Jiri Pechanec
9d84d01f64 DBZ-6726 Switch to ArrayList for repeated iteration 2023-11-09 07:57:12 +01:00
Jakub Cechacek
89b07a013e DBZ-6726 support for $changeStreamSplitLargeEvent aggregation in MongoDB connector 2023-11-09 07:57:12 +01:00
dependabot[bot]
7b23b31408 [ci] Bump tj-actions/changed-files from 39.2.4 to 40.1.0
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39.2.4 to 40.1.0.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v39.2.4...v40.1.0)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-08 17:39:57 -05:00
Jiri Pechanec
be2ec95aa8 [docs] Fix param names 2023-11-08 13:27:34 +01:00
Jakub Cechacek
2f70c30962 DBZ-6875 Ensuring root user is created when using authenticated mongo rs 2023-11-08 13:13:43 +01:00
Rafael Câmara
83e6f28e41 [docs] Add commas to SQL Server connector configuration example. 2023-11-08 12:54:35 +01:00
ani-sha
16ab761bed DBZ-6873 Deprecate expire_logs_days, use binlog_expire_logs_seconds 2023-11-08 07:06:27 +01:00
ani-sha
e7a9430de3 DBZ-6873 Support MySQL 8.2 2023-11-08 07:06:27 +01:00
ani-sha
54be3bb663 DBZ-6873 Use percona server version 8.0 2023-11-08 07:06:27 +01:00
ani-sha
519c382153 DBZ-6873 Support MySQL 8.1 2023-11-08 07:06:27 +01:00
Bob Roldan
f9c321024b DBZ-7108 Minor edit to description of mongodb.connection.mode property 2023-11-08 05:59:39 +01:00
Jakub Cechacek
bddf8802ec DBZ-7108 Switched default connection mode for sharded clusters from replica_set to sharded 2023-11-07 10:36:53 +01:00
Anatolii Popov
3ad0779158 DBZ-7109 Upgrade SQLServer JDBC driver to support sensitivity classification 2023-11-07 10:23:27 +01:00
Vojtech Juranek
cf7f0f3801 DBZ-7110 Use better name for DebeziumEngine.Builder implementation 2023-11-07 10:13:50 +01:00
Vojtech Juranek
c34cf6920c DBZ-7110 Remove deprecated EmbeddedEngine interface 2023-11-07 10:13:50 +01:00
rkerner
68481038b3 DBZ-7105 Fix exceptions that are triggered by RelationalBaseSourceConnector#validate when calling #validateConnection() with an invalid configuration. Other, not-checked fields can cause an exception when the ConnectorConfig class is instantiated to setup the connection. Now we are only allowing connection validation when all config options are free of errors.
closes to https://issues.redhat.com/browse/DBZ-7105
2023-11-06 15:02:51 +01:00
Vojtech Juranek
11d2ff0b9b DBZ-7007 Use better name for auxiliary variable detecing if engine is running 2023-11-06 10:50:21 +01:00
Vojtech Juranek
89054bf8a8 DBZ-7007 Introduce TestingDebeziumEngine and switch to it in AbstractConnectorTest
Postgres `RecordsStreamProducerIT` reliaes on
EmebeddedEngine.runWithTask(). As this method effectively expose
engine's internal task and tests do the asserts against the state
of the task, it's hard to replace it. If we want to keep the tests,
the most simple approach seems to expose engine task in similar way
how EmbeddedEngine does that.

Add interface for testing Debezium engine, which would define minimal
set of methods which needs to be exposed by the implementing classes to
be able to run the testsuite against the Debezium engine. The number of
such methods should be as low as possible. Implementing classes would
typically act as proxies to actual `DebeziumEngine` implementations.

Add `TestingDebeziumEngine` implementation for `EmbeddedEngine` and
switch to `TestingDebeziumEngine` in `AbstractConnectorTest`.
2023-11-06 10:50:21 +01:00
Vojtech Juranek
fdab2cd61b DBZ-7007 Remove engine.isRunning() from testsuite 2023-11-06 10:50:21 +01:00
Vojtech Juranek
89277591dd DBZ-7007 Replace engine.await() with Awaitility condition
In the second case replace infinite loop by one minute wait. If this is
not sufficient for any case, we can increase the timeout in the future.
2023-11-06 10:50:21 +01:00
Vojtech Juranek
9b1d5172ff DBZ-6213 Remove unneeded code from the testsuite
After fixing start method to really start the connector, this wait loop
is not needed any more.
2023-11-06 10:50:21 +01:00
Vojtech Juranek
1a58845507 DBZ-7007 Use close() instead of EmbeddedEngine#stop() method 2023-11-06 10:50:21 +01:00
Vojtech Juranek
953f1c77aa DBZ-7007 Switch to DebeziuEngine in ConnectorOutputTest 2023-11-06 10:50:21 +01:00
Vojtech Juranek
d0c2658c43 DBZ-7007 Switch to DebeziuEngine in EmbeddedEngineTest 2023-11-06 10:50:21 +01:00
Vojtech Juranek
3295844cc3 DBZ-7007 Use property based configuration in the emebdded tests 2023-11-06 10:50:21 +01:00
Vojtech Juranek
9d26dd4cf4 DBZ-7007 Move embedded engine config into separate interface
This config will be re-used by possible other implementations of
DebeiumEngine API in the embedded package. As DebeziumEngine API
can have completely different implementations and thus also config,
the class is called `EmbeddedEngineConfig` as it's assumed to be used
only by embedded engine "family" of implementations.

To keep backward compatibility, the config options are extracted into
an interface and `EmbeddedEngine` implements this interface, thus
allowing to use these options in custom classes without any need for the
code changes.
2023-11-06 10:50:21 +01:00
Vojtech Juranek
e073dbd1be DBZ-7007 Use DebeziumEngine API instead of EmbeddedEngine in postgres tests 2023-11-06 10:50:21 +01:00
Ondrej Babec
b0ee60df1d DBZ-6996
In order to stabilize testsuite on Docker and Kuberetes following changes were made:
- Remove port forwarding from testsuite
- Increase poll timeout
- Separate Mongo test classes based on deployment and streaming mode
- Allow both Docker and Kubernetes Mongo deployments use connection string
- Create container for testsuite
- Create kubernetes deployment for the testsuite
2023-11-04 11:00:46 +01:00
Jakub Cechacek
e48af62c0f DBZ-6996 Updating system tests to properly use MongoDB in a container 2023-11-04 11:00:46 +01:00
caicancai
4b40bb935e DBZ-7095 The MySqlParser.g4 parser has a priority problem 2023-11-03 16:39:23 +01:00
Chris Cranford
e67620d30e DBZ-7047 Add comment about abstract method usage 2023-11-02 08:58:28 -04:00
Chris Cranford
d67f145971 DBZ-7047 Fixed logic to match previous behavior 2023-11-02 08:58:28 -04:00
Chris Cranford
3803a277d6 DBZ-7047 Clean-up more cache iterator/stream call sites. 2023-11-02 08:58:28 -04:00
Chris Cranford
de0969899f DBZ-7047 Only iterate active transactions when DEBUG enabled 2023-11-02 08:58:28 -04:00
Chris Cranford
16d70a1642 DBZ-7047 Utilize closeable iterators
It is recommended by Infinispan that specific calls that return a collection
of elements should be treated as a closable object so that any and all the
potential resources associated with the operation are closed.
2023-11-02 08:58:28 -04:00
Chris Cranford
642b720982 DBZ-7047 Don't use parallel streaming
Using parallel streams, this causes a severe loss of performance.
2023-11-02 08:58:28 -04:00
Jakub Cechacek
a3affaf6a8 DBZ-7058 Fixed index handling in FieldSelector 2023-11-02 13:29:39 +01:00
Chris Cranford
b0e0674ef2 DBZ-6763 Don't hard fail the secrets check 2023-11-02 09:00:45 +01:00
rkerner
de6c1976f9 DBZ-6763 Add integration tests for the Debezium Connect REST extension for the Oracle connector / Oracle Connect REST extension
closes to https://issues.redhat.com/browse/DBZ-6763
2023-11-02 09:00:45 +01:00
Roman Kudryashov
3d56341375 DBZ-7060 Rename metadata.location -> metadata.source 2023-10-27 13:55:18 +02:00
harveyyue
767c342506 DBZ-7062 Update transformation property "delete.tombstone.handling.mode" to debezium doc 2023-10-27 11:17:02 +02:00
Jiri Pechanec
f2511a5b05 [release] Add breaking change 2023-10-26 19:02:23 +02:00
Debezium Builder
33b7ad2cf4 [release] Development version for testing module deps 2023-10-26 15:49:32 +00:00
Debezium Builder
78c5204444 [maven-release-plugin] prepare for next development iteration 2023-10-26 15:39:35 +00:00