Chris Cranford
25ef5fa7d0
DBZ-4321 Fix test failures with Oracle XStream
2023-12-13 11:27:40 -05:00
Chris Cranford
2b02b3982e
DBZ-4321 Rework configuration options
2023-12-13 11:27:40 -05:00
Chris Cranford
ada157d117
DBZ-4321 Introduce BeanRegistry
and ServiceRegistry
...
As a part of this work to handle injection in a cleaner way, this commit
adds two new broad concepts called `BeanRegistry` and `ServiceRegistry`.
A BeanRegistry is a glorified registry of different objects that are not
necessarily services but may be desired by a service. This contract will
allow Debezium to integrate in the future with other CDI providers.
A ServiceRegistry is more of an internal concept, where various systems
can be started based on their dependency order and provides a universal
way to split larger parts of the code into smaller, focused modules that
can be accessed using the Service Locator pattern.
2023-12-13 11:27:40 -05:00
Chris Cranford
26b3881941
DBZ-4321 Add MongoDB hook & polish injection handler
2023-12-13 11:27:40 -05:00
Chris Cranford
59027ed5ed
DBZ-4321 New PostProcessor
contract and Column Reselection
2023-12-13 11:27:40 -05:00
Chris Cranford
9a5d04578e
DBZ-7237 Align LOB behavior in snapshot and streaming
...
The snapshot phase was not setting the unavailable value placeholder when the
user had configured LOB as off, this aligns that behavior to be consistent
with the behavior from streaming.
2023-12-12 10:36:23 +01:00
Chris Cranford
d27154dc60
DBZ-7218 Log warnings if log destination misconfigured
2023-12-11 07:51:24 -05:00
Jiri Kulhanek
28188692d7
DBZ-7236 simple fix of inversed if logic
2023-12-07 05:00:39 -05:00
Chris Cranford
aea6cc003d
[ci] Use a relaxed property check for tests
2023-12-06 18:04:59 -05:00
mfvitale
4fedfbba03
DBZ-6834 Provide INSERT/DELETE semantics for incremental snapshot watermarking
2023-12-06 14:10:26 +01:00
Debezium Builder
0fd1c0dc9a
[maven-release-plugin] prepare for next development iteration
2023-12-04 13:55:35 +00:00
Debezium Builder
3e2d75f0da
[maven-release-plugin] prepare release v2.5.0.Beta1
2023-12-04 13:55:35 +00:00
Chris Cranford
9976f9848f
DBZ-7207 Allow XStream attach errors to be retried
2023-12-01 11:44:28 +01:00
Jiri Kulhanek
8192620d71
DBZ-7208 - implemented check for NPE
2023-11-30 20:28:43 -05:00
Chris Cranford
7ddbeceb82
DBZ-7192 Checkstyle fixes
2023-11-30 00:06:05 -05:00
Jiri Kulhanek
18c672ab6f
DBZ-7192: move abandonedTransactionsCache to common abstract layer
...
also makes sure that events are correctly removed in ISPN event processor after transaction is abandoned.
Also fixes scenario with event number based threshold abandonment in ISPN - events comming afterwards would be still processed.
2023-11-30 00:06:05 -05:00
Jiri Kulhanek
c0f141f51f
DBZ-7192: move abandoned transactions impl to Abstract layer
...
So it is available for ISPN. ISPN does not store list of abandoned transactions
fix rebase
cosmetic changes
2023-11-30 00:06:05 -05:00
mfvitale
5ddf75cbea
DBZ-6878 Fix NotificationsIT for Oracle
2023-11-29 12:08:11 -05:00
“vsantonastaso”
8c1c369449
DBZ-6878 add table specific notification in initial snapshot
2023-11-29 08:32:21 +01:00
Chris Cranford
425543d1e0
DBZ-7194 Add GGSHAREDCAP to excluded schemas
2023-11-28 19:56:57 -05:00
Chris Cranford
937677cc89
DBZ-7194 Use a more relaxed version pattern
2023-11-28 19:56:57 -05:00
baabgai
02ad10550f
DBZ-7194 Update OracleDatabaseVersion.java
...
match only to the first occurrence of 'Release ' Substring in Oracle Banner
2023-11-28 19:56:57 -05:00
Jiri Kulhanek
71d80af0b7
DBZ-7169 - populating processed transaction cache only if lobs enabled - in sync with memory event processor
2023-11-27 17:55:56 -05:00
Chris Cranford
aa229424a4
DBZ-7158 Treat Oracle RAC redo thread log sequences independently
2023-11-23 14:14:46 +01:00
rkerner
aeaba3ae70
DBZ-6762 Add "validate filters" endpoint/s for connector-specific Connect REST Extensions
...
closes to https://issues.redhat.com/browse/DBZ-6762
2023-11-22 06:10:40 +01:00
Chris Cranford
4e1d03c5aa
DBZ-7153 Fix test failures for undo operations
2023-11-18 09:11:31 -05:00
Jiri Kulhanek
371237d167
DBZ-7153 - memory cache recreated during startup (from ISPN cache)
2023-11-18 09:11:31 -05:00
Kosta Kostelnik
3508913a3d
DBZ-7153 - using an in-mem cache to process faster
2023-11-18 09:11:31 -05:00
Debezium Builder
1521445908
[maven-release-plugin] prepare for next development iteration
2023-11-10 10:26:05 +00:00
Debezium Builder
6c6f6e9138
[maven-release-plugin] prepare release v2.5.0.Alpha2
2023-11-10 10:26:05 +00:00
Chris Cranford
34df6ade4d
DBZ-6942 Verify no log sequence gaps on log switch
...
There is a corner case where it's possible the Oracle connector may query
the Oracle metadata tables quicker than the ARC process can generate an
archive log history record in V$ARCHIVED_LOG, and this can lead to a race
condition where we may incorrectly advance the connector forward to start
mining a group of logs when a log sequence gap exists in the log ranges.
For users who use the online_catalog strategy, there are some checks that
LogMiner does automatically which it skips, and one is with log sequence
gaps. This fix enforces that check by Debezium even for users who may use
the faster online_catalog mode so that no logs are omitted and events
could be missed.
2023-11-10 09:49:10 +01:00
Chris Cranford
b834a3a66a
DBZ-6963 Test should only apply to LogMiner
2023-11-09 10:58:01 +01:00
Chris Cranford
81d30a1224
DBZ-6963 Fix test failure
2023-11-09 10:58:01 +01:00
Chris Cranford
a4346222a7
DBZ-6963 Checkstyle fixes
2023-11-09 10:58:01 +01:00
Chris Cranford
b69dbd53e6
DBZ-6963 Do not merge events if table has no LOB columns
...
There was a possible situation where if a long transaction consisted of
updating and inserting into the same table with identical keys with a
given sequence that the commit handler would merge several events for a
table without LOB columns, resulting in a difference in expected events
in the Kafka topic vs what was seen in LogMiner.
2023-11-09 10:58:01 +01:00
Vojtech Juranek
9d26dd4cf4
DBZ-7007 Move embedded engine config into separate interface
...
This config will be re-used by possible other implementations of
DebeiumEngine API in the embedded package. As DebeziumEngine API
can have completely different implementations and thus also config,
the class is called `EmbeddedEngineConfig` as it's assumed to be used
only by embedded engine "family" of implementations.
To keep backward compatibility, the config options are extracted into
an interface and `EmbeddedEngine` implements this interface, thus
allowing to use these options in custom classes without any need for the
code changes.
2023-11-06 10:50:21 +01:00
Chris Cranford
e67620d30e
DBZ-7047 Add comment about abstract method usage
2023-11-02 08:58:28 -04:00
Chris Cranford
d67f145971
DBZ-7047 Fixed logic to match previous behavior
2023-11-02 08:58:28 -04:00
Chris Cranford
3803a277d6
DBZ-7047 Clean-up more cache iterator/stream call sites.
2023-11-02 08:58:28 -04:00
Chris Cranford
de0969899f
DBZ-7047 Only iterate active transactions when DEBUG enabled
2023-11-02 08:58:28 -04:00
Chris Cranford
16d70a1642
DBZ-7047 Utilize closeable iterators
...
It is recommended by Infinispan that specific calls that return a collection
of elements should be treated as a closable object so that any and all the
potential resources associated with the operation are closed.
2023-11-02 08:58:28 -04:00
Chris Cranford
642b720982
DBZ-7047 Don't use parallel streaming
...
Using parallel streams, this causes a severe loss of performance.
2023-11-02 08:58:28 -04:00
rkerner
de6c1976f9
DBZ-6763 Add integration tests for the Debezium Connect REST extension for the Oracle connector / Oracle Connect REST extension
...
closes to https://issues.redhat.com/browse/DBZ-6763
2023-11-02 09:00:45 +01:00
Debezium Builder
78c5204444
[maven-release-plugin] prepare for next development iteration
2023-10-26 15:39:35 +00:00
Debezium Builder
356e488e83
[maven-release-plugin] prepare release v2.5.0.Alpha1
2023-10-26 15:39:35 +00:00
Chris Cranford
a1b88e98be
DBZ-7071 Introduce internal schema change exclusion option
...
Add a new internal `log.mining.schema_changes.username.exclude.list` to allow users
to customize the default behavior for excluding the SYS and SYSTEM usernames from
DDL changes.
2023-10-25 09:19:11 +02:00
Chris Cranford
5c9870bcfa
DBZ-7067 Improve Oracle logging for COMMITs
2023-10-24 22:39:21 -04:00
Chris Cranford
6e814e17a3
DBZ-6975 Fix test failures
2023-10-18 17:56:12 -04:00
Chris Cranford
b96be67937
DBZ-7018 Limit continuation in test to 10 rows
2023-10-13 08:40:34 -04:00
Chris Cranford
2479d811fb
DBZ-7018 Allow SQL statements greater than 40kb
...
In some corner cases, users may have unusually large SQL statements that
need to be buffered due to the number of columns paired with the data in
those columns. Previously we capped this to 4000*10 or 40kb primarily to
address situations with LOB operations that could lead to OOM scenarios.
The new code rather logs a warning when exceeding 100kb and hard faults
ony when the connector sees Integer.MAX_VALUE number of SQL lines for a
single SQL buffer.
2023-10-13 08:40:34 -04:00
Chris Cranford
472d010f7c
DBZ-7006 Properly handle escaped CLOB single-quotes
2023-10-11 14:39:12 +02:00
Chris Cranford
e2b871ae7b
DBZ-6975 Correctly handle escaped single-quotes in UNISTR arguments
2023-10-11 14:37:44 +02:00
Chris Cranford
67ede63ccc
DBZ-6899 Add test-excludes for XStream
2023-10-04 13:47:43 -04:00
Chris Cranford
a3b51b48b9
DBZ-6899 Refactor Oracle streamings metrics
2023-10-04 13:47:43 -04:00
Debezium Builder
40a8f4026d
[maven-release-plugin] prepare for next development iteration
2023-10-03 14:18:54 +00:00
Debezium Builder
412c6f7979
[maven-release-plugin] prepare release v2.4.0.Final
2023-10-03 14:18:54 +00:00
Roman Kudryashov
98c426dec4
DBZ-3642 CloudEventsConverter can retrieve metadata info from headers
2023-09-26 13:06:09 +02:00
Debezium Builder
cc1de0aa51
[maven-release-plugin] prepare for next development iteration
2023-09-22 09:00:45 +00:00
Debezium Builder
6809978083
[maven-release-plugin] prepare release v2.4.0.CR1
2023-09-22 09:00:45 +00:00
Jiri Pechanec
bbc8b5b4d6
DBZ-6945 DROP event does not contain table structure
2023-09-22 10:45:00 +02:00
rkerner
6d3528d4fb
DBZ-4395 Add connector specific Debezium Connect REST Extension/s and move logic from UI backend to the Debezium Connect REST Extension (part 2 of the re-architecturing/re-factoring)
...
DBZ-6761 Add "validate connection" endpoints in connector specific Connect REST extensions
closes to https://issues.redhat.com/browse/DBZ-4395
closes to https://issues.redhat.com/browse/DBZ-6761
2023-09-21 13:29:59 +02:00
mfvitale
68eb4b2df2
DBZ-6828 Process drop table events during blocking snapshot
2023-09-20 08:18:33 +02:00
mfvitale
ae199d2053
DBZ-6828 Blocking snapshot will generate schema events only for signaled tables
2023-09-20 08:18:33 +02:00
Chris Cranford
e282817f5c
DBZ-6869 Ignore pending transactions with zero START_SCN
...
Pending transactions with a START_SCN of 0 are considered transactions
that have started before the oldest available archive log and these
will be ignored as the entire transaction cannot be mined.
2023-09-19 15:42:56 -04:00
Chris Cranford
f429118d98
DBZ-6938 Exclude UNUSED redo logs
2023-09-19 15:36:26 -04:00
Chris Cranford
9039a12731
DBZ-6938 Make retry scn-log-check per redo thread
2023-09-19 15:36:26 -04:00
Chris Cranford
3a729932d3
DBZ-6895 Always dispatch heartbeats on commit and checkpoints
2023-09-19 09:00:26 +02:00
Chris Cranford
9bccb2baaf
DBZ-6895 Refactor to use Json deserializers
2023-09-19 09:00:26 +02:00
Chris Cranford
1b31c663ff
DBZ-6895 Fix OpenLogReplicator confirmation
2023-09-19 09:00:26 +02:00
Jiri Pechanec
f8e4e589d8
DBZ-6903 Use distinct dir for protobuf generated sources
2023-09-13 11:50:47 -04:00
Debezium Builder
65169a9315
[maven-release-plugin] prepare for next development iteration
2023-09-13 09:41:46 +00:00
Debezium Builder
845e3e9d09
[maven-release-plugin] prepare release v2.4.0.Beta2
2023-09-13 09:41:46 +00:00
Chris Cranford
6e10618c94
DBZ-2543 Checkstyle fixes
2023-09-12 18:20:07 -04:00
Chris Cranford
1d9b81b58c
DBZ-2543 Confirm only checkpoints
2023-09-12 18:20:07 -04:00
Chris Cranford
e8e11cd40a
DBZ-2543 Fix value converter failures
2023-09-12 18:20:07 -04:00
Chris Cranford
46fa3bef4d
DBZ-2543 Refactor value conversion logic
2023-09-12 18:20:07 -04:00
Chris Cranford
ca6ccbcd35
DBZ-2543 Remove unnecessary test class
2023-09-12 18:20:07 -04:00
Chris Cranford
e841e6c74d
DBZ-2543 Use protoc plugin setup like PostgreSQL
2023-09-12 18:20:07 -04:00
Chris Cranford
c076d9526b
DBZ-2543 Remove vert.x dependency
2023-09-12 18:20:07 -04:00
Chris Cranford
2cae1b396d
DBZ-2543 Add OpenLogReplicator configuration properties
...
Added source, host, and port configurable OpenLogReplicator connector properties.
2023-09-12 18:20:07 -04:00
Chris Cranford
0955d0e7d6
DBZ-2543 Disable XML tests for OpenLogReplicator
2023-09-12 18:20:07 -04:00
Chris Cranford
07c7a8236e
DBZ-2543 Disable ROWID tests for OpenLogReplicator
2023-09-12 18:20:07 -04:00
Chris Cranford
a87533e4a8
DBZ-2543 Interval data types support
2023-09-12 18:20:07 -04:00
Chris Cranford
51b95e3bfb
DBZ-2543 Add timestamp with local time zone support
2023-09-12 18:20:07 -04:00
Chris Cranford
c9f6f4596e
DBZ-2543 Clean-up value resolution logic
2023-09-12 18:20:07 -04:00
Chris Cranford
73aa4a6b35
DBZ-2543 Properly decode non-variable RAW data
2023-09-12 18:20:07 -04:00
Chris Cranford
1e803edee8
DBZ-2543 Cleanup timestamptz parsing and DDL handling
2023-09-12 18:20:07 -04:00
Chris Cranford
d130fde297
DBZ-2543 Improve timestamp handling for precision modes
2023-09-12 18:20:07 -04:00
Chris Cranford
26db0b3961
DBZ-2543 Support timestamps with time zone data types
2023-09-12 18:20:07 -04:00
Chris Cranford
748420605e
DBZ-2543 Add support for new db
field in streaming events
2023-09-12 18:20:07 -04:00
Chris Cranford
2d4d34428d
DBZ-2543 Add some streaming metrics
2023-09-12 18:20:07 -04:00
Chris Cranford
d977e9b0f1
DBZ-2543 Initial OpenLogReplicator adapter
2023-09-12 18:20:07 -04:00
Chris Cranford
dd5f1ac0f9
DBZ-6880 Include details about unsupported operations
2023-09-08 14:16:02 -04:00
Chris Cranford
b5700a62c9
DBZ-6880 Move some Oracle log messages to DEBUG
2023-09-08 14:16:02 -04:00
Chris Cranford
a0c67aa0e8
DBZ-6860 Fix NPE Oracle shouldContinueToUpdateOffsetsEvenWhenTableIsNotChanged
2023-08-31 19:13:55 -04:00
Chris Cranford
6aa6b58fdc
DBZ-6859 Restore Oracle connector test suite logging
2023-08-31 17:09:25 -04:00
Debezium Builder
d97a18e8dc
[maven-release-plugin] prepare for next development iteration
2023-08-29 11:41:08 +00:00
Debezium Builder
5ce25f2cfe
[maven-release-plugin] prepare release v2.4.0.Beta1
2023-08-29 11:41:08 +00:00
Ondrej Babec
f03eda1cc4
DBZ-6827 Testing farm onboarding
2023-08-28 10:39:30 +02:00
Chris Cranford
e093a7bdd2
[ci] Fix some failing Oracle tests
2023-08-23 12:53:19 -04:00