Commit Graph

1247 Commits

Author SHA1 Message Date
harveyyue
225277c5c0 DBZ-7251 Return generic DataCollectionId instead of REST response DataCollection in getMatchingCollections method 2023-12-19 12:53:57 +01:00
harveyyue
4719396ffc DBZ-7145 Introduce a common parent connect for all Debezium source connectors 2023-12-19 12:53:57 +01:00
harveyyue
5553059fb8 DBZ-7167 Provide a public API from the connector implementations to retrieve the list of matching collections or tables based on the different include-/exclude lists 2023-12-19 12:53:57 +01:00
Vojtech Juranek
37ebb8ecae DBZ-7275 Close connection registerd with bean registry
New connection created for bean registry is never closed. Close it
during when stopping the task.
2023-12-19 10:22:02 +01:00
Vojtech Juranek
e48786c194 DBZ-7276 Grant DELETE on signalling table in the incremental snapshot tests
Delete is needed for `insert_delete` watermarking strategy.
2023-12-19 06:23:25 +01:00
Debezium Builder
2c1def7241 [maven-release-plugin] prepare for next development iteration 2023-12-14 09:43:13 +00:00
Debezium Builder
ef8260f802 [maven-release-plugin] prepare release v2.5.0.CR1 2023-12-14 09:43:12 +00:00
Chris Cranford
25ef5fa7d0 DBZ-4321 Fix test failures with Oracle XStream 2023-12-13 11:27:40 -05:00
Chris Cranford
2b02b3982e DBZ-4321 Rework configuration options 2023-12-13 11:27:40 -05:00
Chris Cranford
ada157d117 DBZ-4321 Introduce BeanRegistry and ServiceRegistry
As a part of this work to handle injection in a cleaner way, this commit
adds two new broad concepts called `BeanRegistry` and `ServiceRegistry`.

A BeanRegistry is a glorified registry of different objects that are not
necessarily services but may be desired by a service. This contract will
allow Debezium to integrate in the future with other CDI providers.

A ServiceRegistry is more of an internal concept, where various systems
can be started based on their dependency order and provides a universal
way to split larger parts of the code into smaller, focused modules that
can be accessed using the Service Locator pattern.
2023-12-13 11:27:40 -05:00
Chris Cranford
26b3881941 DBZ-4321 Add MongoDB hook & polish injection handler 2023-12-13 11:27:40 -05:00
Chris Cranford
59027ed5ed DBZ-4321 New PostProcessor contract and Column Reselection 2023-12-13 11:27:40 -05:00
Chris Cranford
9a5d04578e DBZ-7237 Align LOB behavior in snapshot and streaming
The snapshot phase was not setting the unavailable value placeholder when the
user had configured LOB as off, this aligns that behavior to be consistent
with the behavior from streaming.
2023-12-12 10:36:23 +01:00
Chris Cranford
d27154dc60 DBZ-7218 Log warnings if log destination misconfigured 2023-12-11 07:51:24 -05:00
Jiri Kulhanek
28188692d7 DBZ-7236 simple fix of inversed if logic 2023-12-07 05:00:39 -05:00
Chris Cranford
aea6cc003d
[ci] Use a relaxed property check for tests 2023-12-06 18:04:59 -05:00
mfvitale
4fedfbba03 DBZ-6834 Provide INSERT/DELETE semantics for incremental snapshot watermarking 2023-12-06 14:10:26 +01:00
Debezium Builder
0fd1c0dc9a [maven-release-plugin] prepare for next development iteration 2023-12-04 13:55:35 +00:00
Debezium Builder
3e2d75f0da [maven-release-plugin] prepare release v2.5.0.Beta1 2023-12-04 13:55:35 +00:00
Chris Cranford
9976f9848f DBZ-7207 Allow XStream attach errors to be retried 2023-12-01 11:44:28 +01:00
Jiri Kulhanek
8192620d71 DBZ-7208 - implemented check for NPE 2023-11-30 20:28:43 -05:00
Chris Cranford
7ddbeceb82 DBZ-7192 Checkstyle fixes 2023-11-30 00:06:05 -05:00
Jiri Kulhanek
18c672ab6f DBZ-7192: move abandonedTransactionsCache to common abstract layer
also makes sure that events are correctly removed in ISPN event processor after transaction is abandoned.
Also fixes scenario with event number based threshold abandonment in ISPN - events comming afterwards would be still processed.
2023-11-30 00:06:05 -05:00
Jiri Kulhanek
c0f141f51f DBZ-7192: move abandoned transactions impl to Abstract layer
So it is available for ISPN. ISPN does not store list of abandoned transactions

fix rebase

cosmetic changes
2023-11-30 00:06:05 -05:00
mfvitale
5ddf75cbea DBZ-6878 Fix NotificationsIT for Oracle 2023-11-29 12:08:11 -05:00
“vsantonastaso”
8c1c369449 DBZ-6878 add table specific notification in initial snapshot 2023-11-29 08:32:21 +01:00
Chris Cranford
425543d1e0 DBZ-7194 Add GGSHAREDCAP to excluded schemas 2023-11-28 19:56:57 -05:00
Chris Cranford
937677cc89 DBZ-7194 Use a more relaxed version pattern 2023-11-28 19:56:57 -05:00
baabgai
02ad10550f DBZ-7194 Update OracleDatabaseVersion.java
match only to the first occurrence of 'Release ' Substring in Oracle Banner
2023-11-28 19:56:57 -05:00
Jiri Kulhanek
71d80af0b7 DBZ-7169 - populating processed transaction cache only if lobs enabled - in sync with memory event processor 2023-11-27 17:55:56 -05:00
Chris Cranford
aa229424a4 DBZ-7158 Treat Oracle RAC redo thread log sequences independently 2023-11-23 14:14:46 +01:00
rkerner
aeaba3ae70 DBZ-6762 Add "validate filters" endpoint/s for connector-specific Connect REST Extensions
closes to https://issues.redhat.com/browse/DBZ-6762
2023-11-22 06:10:40 +01:00
Chris Cranford
4e1d03c5aa DBZ-7153 Fix test failures for undo operations 2023-11-18 09:11:31 -05:00
Jiri Kulhanek
371237d167 DBZ-7153 - memory cache recreated during startup (from ISPN cache) 2023-11-18 09:11:31 -05:00
Kosta Kostelnik
3508913a3d DBZ-7153 - using an in-mem cache to process faster 2023-11-18 09:11:31 -05:00
Debezium Builder
1521445908 [maven-release-plugin] prepare for next development iteration 2023-11-10 10:26:05 +00:00
Debezium Builder
6c6f6e9138 [maven-release-plugin] prepare release v2.5.0.Alpha2 2023-11-10 10:26:05 +00:00
Chris Cranford
34df6ade4d DBZ-6942 Verify no log sequence gaps on log switch
There is a corner case where it's possible the Oracle connector may query
the Oracle metadata tables quicker than the ARC process can generate an
archive log history record in V$ARCHIVED_LOG, and this can lead to a race
condition where we may incorrectly advance the connector forward to start
mining a group of logs when a log sequence gap exists in the log ranges.

For users who use the online_catalog strategy, there are some checks that
LogMiner does automatically which it skips, and one is with log sequence
gaps. This fix enforces that check by Debezium even for users who may use
the faster online_catalog mode so that no logs are omitted and events
could be missed.
2023-11-10 09:49:10 +01:00
Chris Cranford
b834a3a66a DBZ-6963 Test should only apply to LogMiner 2023-11-09 10:58:01 +01:00
Chris Cranford
81d30a1224 DBZ-6963 Fix test failure 2023-11-09 10:58:01 +01:00
Chris Cranford
a4346222a7 DBZ-6963 Checkstyle fixes 2023-11-09 10:58:01 +01:00
Chris Cranford
b69dbd53e6 DBZ-6963 Do not merge events if table has no LOB columns
There was a possible situation where if a long transaction consisted of
updating and inserting into the same table with identical keys with a
given sequence that the commit handler would merge several events for a
table without LOB columns, resulting in a difference in expected events
in the Kafka topic vs what was seen in LogMiner.
2023-11-09 10:58:01 +01:00
Vojtech Juranek
9d26dd4cf4 DBZ-7007 Move embedded engine config into separate interface
This config will be re-used by possible other implementations of
DebeiumEngine API in the embedded package. As DebeziumEngine API
can have completely different implementations and thus also config,
the class is called `EmbeddedEngineConfig` as it's assumed to be used
only by embedded engine "family" of implementations.

To keep backward compatibility, the config options are extracted into
an interface and `EmbeddedEngine` implements this interface, thus
allowing to use these options in custom classes without any need for the
code changes.
2023-11-06 10:50:21 +01:00
Chris Cranford
e67620d30e DBZ-7047 Add comment about abstract method usage 2023-11-02 08:58:28 -04:00
Chris Cranford
d67f145971 DBZ-7047 Fixed logic to match previous behavior 2023-11-02 08:58:28 -04:00
Chris Cranford
3803a277d6 DBZ-7047 Clean-up more cache iterator/stream call sites. 2023-11-02 08:58:28 -04:00
Chris Cranford
de0969899f DBZ-7047 Only iterate active transactions when DEBUG enabled 2023-11-02 08:58:28 -04:00
Chris Cranford
16d70a1642 DBZ-7047 Utilize closeable iterators
It is recommended by Infinispan that specific calls that return a collection
of elements should be treated as a closable object so that any and all the
potential resources associated with the operation are closed.
2023-11-02 08:58:28 -04:00
Chris Cranford
642b720982 DBZ-7047 Don't use parallel streaming
Using parallel streams, this causes a severe loss of performance.
2023-11-02 08:58:28 -04:00
rkerner
de6c1976f9 DBZ-6763 Add integration tests for the Debezium Connect REST extension for the Oracle connector / Oracle Connect REST extension
closes to https://issues.redhat.com/browse/DBZ-6763
2023-11-02 09:00:45 +01:00