Commit Graph

8166 Commits

Author SHA1 Message Date
Chris Cranford
7384a1695e DBZ-6150 Upgrade to Quarkus 2.16.3.Final 2023-02-27 12:39:45 +01:00
Bobby Tiernay
37a6fea350 DBZ-6152: Remove hardcoded list of system database exclusions that are not required for change streaming. 2023-02-24 09:43:39 +01:00
Bobby Tiernay
5d80712781 DBZ-5201: Add MongoDB connector support for server-side filtering of replica set change streams.
This change migrates from doing client-side filtering of streaming
change events to performing it server-side via MongoDB's support
for aggregation pipelines. This can drastically cut down on the amount
of data sent between the client and server. In addition, a new approach
to heartbeating has been added which relies on the `postBatchResumeToken`
to advance the the cursor even when no changes occur on the pipeline.
2023-02-23 18:14:08 +01:00
Vojtech Juranek
fcb0e77bbb DBZ-3594 Warn when obtaning table schema which should be alread present
We load all the schemas of the captured tables when the connector
starts. If we process a record from a table which schema is not
available, this means we have some bug in the intial schema loading.
Don't fail in such case, but print a warning about that.
2023-02-23 09:26:07 -05:00
Vojtech Juranek
0868bfd55e DBZ-3594 Remove unused deprecated method 2023-02-23 09:26:07 -05:00
Chris Cranford
e320abc834 DBZ-6143 Unify Oracle TIMESTAMP WITH TIME ZONE format
When taking a snapshot, the Oracle connector was converting the TIMESTAMP
WITH TIME ZONE value to GMT and per the documentation, the value should
be emitted in the time zone of the data.

The snapshot emitted value in GMT is temporally accurate, so there is no
data inconsistency, but the emitted format itself was inconsistent when
looking at how the column data was emitted during a snapshot versus in a
streaming event.
2023-02-23 10:43:38 +01:00
Xinbin Huang
76a8f8dbe1 DBZ-6118: Update ExtractNewDocumentState to parse change stream output 2023-02-23 09:09:04 +01:00
Xinbin Huang
2c57b93384 DBZ-6118: Cleanup Mongo oplog code logic 2023-02-23 09:09:04 +01:00
Vojtech Juranek
ffc9c2f80d [docs] Fix poll.interval.ms default value 2023-02-22 13:49:40 -05:00
Chris Cranford
24fe8fe572 DBZ-6136 Add toJson to TestContainers ConnectorConfiguration impl 2023-02-21 18:31:57 +01:00
Eugene Abramchuk
fcc44edf0b DBZ-6113 Correct max reconnect attempts
Originally there are max+1 attempts
2023-02-21 08:10:26 +01:00
dependabot[bot]
a587b3b7ef [ci] Bump tj-actions/changed-files from 35.5.2 to 35.5.6
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 35.5.2 to 35.5.6.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v35.5.2...v35.5.6)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-21 07:52:16 +01:00
Xinbin Huang
a3d75e798a DBZ-6116: Support String type for key in Mongo incremental snapshot 2023-02-20 13:55:54 +01:00
Chris Cranford
dd9ca95bec DBZ-6125 Fix potential stale offsets when using PDBs
DBZ-5648 introduced a regression where transaction start, commit, and rollback
events were only being read from within the scope of the configured PDB that
the connector was capturing changes instead of the entire Oracle database.
This can lead to situations where the offsets may not be advanced as quickly
in a low traffic PDB environment, potentially causing stale offsets.
2023-02-20 13:29:11 +01:00
Chris Cranford
a89212efef DBZ-6120 Correctly handle quoting table/schema names 2023-02-20 13:26:49 +01:00
Mehmet Firat KOMURCU
4c9ab87704 DBZ-6123 fix typo "evemts" to "events" in docs.
fix typo in sql server docs
2023-02-20 08:29:54 +01:00
Debezium Builder
bf347da71a [release] Development version for testing module deps 2023-02-16 11:35:58 +00:00
Debezium Builder
2245ceda44 [maven-release-plugin] prepare for next development iteration 2023-02-16 11:28:25 +00:00
Debezium Builder
20f3b16e8a [maven-release-plugin] prepare release v2.2.0.Alpha2 2023-02-16 11:28:25 +00:00
Debezium Builder
0fce13b4e9 [release] Stable 2.2.0.Alpha2 for testing module deps 2023-02-16 11:26:07 +00:00
Jiri Pechanec
68a6a32323 [release] Changelog for 2.2.0.Alpha2 2023-02-16 12:09:05 +01:00
Bob Roldan
bb0f972af4 DBZ-6006 Update MongoDB flattening docs for downstream 2023-02-16 08:27:31 +01:00
Chris Cranford
58f3770ad1 DBZ-6079 Change query.fetch.size default from 0 to 2000 for Oracle 2023-02-16 08:15:22 +01:00
Jiri Pechanec
8f286dd3ca DBZ-6049 Fix release process for separated server repo 2023-02-16 08:13:11 +01:00
Jiri Pechanec
49e37e70ea DBZ-6049 Changes for separate server repo 2023-02-16 08:13:11 +01:00
Jiri Pechanec
8d08bcd57a DBZ-5825 Add Cassandra repo 2023-02-16 08:09:59 +01:00
Jiri Pechanec
3341292ba8 DBZ-5825 Switch to Java 20 2023-02-16 08:09:59 +01:00
Jiri Pechanec
b20e1e6eee DBZ-5825 Add Spanner and Server repos 2023-02-16 08:09:59 +01:00
Chris Cranford
fbd495d63f DBZ-5825 Use Java 19 in Java Outreach action 2023-02-16 08:09:59 +01:00
Chris Cranford
6b40103eaa DBZ-6107 Fix bulk updates when LOB is enabled 2023-02-16 07:29:37 +01:00
Jacob Barrieault
0a868f28c6 DBZ-6111 fix docs link to debezium-server Kinesis sink 2023-02-15 14:43:02 -05:00
Yang Wu
e98089e9a3 DBZ-6038 Expose walltime from mongo record parser 2023-02-14 09:48:17 +01:00
Chris Cranford
7cd7cad094 DBZ-6091 Fix SSN NumericOverflow exception 2023-02-14 09:44:21 +01:00
Jiri Pechanec
907d694139 DBZ-6102 Upgrade Kafka to 3.4.0 2023-02-14 09:41:20 +01:00
Chris Cranford
0771923416 DBZ-6094 Reduce frequency of "Transaction already processed" debug message 2023-02-14 09:09:44 +01:00
dependabot[bot]
728382f453 [ci] Bump tj-actions/changed-files from 35.5.0 to 35.5.2
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 35.5.0 to 35.5.2.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v35.5.0...v35.5.2)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-14 08:42:49 +01:00
Gunnar Morling
5ff4df1417 [docs] Adding missing comma in listing 2023-02-13 10:31:01 -05:00
Chris Cranford
c5d96a2aec DBZ-5758 Repackaged reactive extension
In order to avoid potential conflicts or problems with reusing the
same class names, this commit repackages all reactive code into a
separate io.debezium.outbox.reactive package rather than using the
same non-reactive package io.debezium.outbox.
2023-02-10 14:13:23 +01:00
Chris Cranford
885856a0f3 DBZ-5758 Backport DBZ-5821 to the Outbox Reactive Tracer-based dispatcher. 2023-02-10 14:13:23 +01:00
Chris Cranford
93220a1641 DBZ-5758 Formatting 2023-02-10 14:13:23 +01:00
Chris Cranford
b15ee3dfec DBZ-5758 Cleanup build warnings, removed deprecations 2023-02-10 14:13:23 +01:00
Chris Cranford
05fb9f24c5 DBZ-5758 Relocated common artifacts in debezium-quarkus-outbox-common module 2023-02-10 14:13:23 +01:00
Chris Cranford
1f41336b38 DBZ-5758 Use exposed mapped port for Outbox Reactive tests 2023-02-10 14:13:23 +01:00
Chris Cranford
31e49efc35 DBZ-5758 Introduce common modules for common code 2023-02-10 14:13:23 +01:00
ingmarfjolla
63fa947982 DBZ-5758 Reactive Outbox Extension 2023-02-10 14:13:23 +01:00
ingmarfjolla
a08c7e23c8 DBZ-5758 reactive outbox 2023-02-10 14:13:23 +01:00
Chris Cranford
375be6d61d DBZ-5283 Updated documentation 2023-02-10 13:53:05 +01:00
Chris Cranford
a7d1bb0f86 DBZ-5283 Decoupled changed SMT vs new state SMT, updated docs 2023-02-10 13:53:05 +01:00
Chris Cranford
bf46b298a2 DBZ-5283 Event Changes documentation updates 2023-02-10 13:53:05 +01:00
Chris Cranford
3d5c4328db DBZ-5283 Document unchanged field support to ExtractNewRecordState 2023-02-10 13:53:05 +01:00