Chris Cranford
09af8c1e85
DBZ-7819 Fix test failure due to case
2024-05-01 08:40:19 -04:00
Chris Cranford
3127284ac3
DBZ-7819 Remove creation of temporary objects
2024-05-01 08:40:19 -04:00
Chris Cranford
f23734c5d6
DBZ-7819 Allow log.mining.flush.table.name
to optionally include schema
2024-05-01 08:40:19 -04:00
Chris Cranford
35bc10a97a
DBZ-7833 Move undocumented option to internal namespace
2024-04-30 08:53:15 +02:00
Debezium Builder
bcac0c5c7a
[maven-release-plugin] prepare for next development iteration
2024-04-25 09:21:27 +00:00
Debezium Builder
0be6e6b8fd
[maven-release-plugin] prepare release v2.7.0.Alpha1
2024-04-25 09:21:26 +00:00
mfvitale
a708ef1009
DBZ-7616 Simplify Oracle tests assertion on SQLTimeoutException
2024-04-22 09:17:44 +02:00
mfvitale
30c11242ac
DBZ-7616 Avoid to filter out every SET STATEMENT
queries and filter just the ones related to Amazon RDS
2024-04-22 09:17:44 +02:00
mfvitale
8206d2d98d
DBZ-7616 Add query timeout on JdbcConnection controlled by query.timeout.ms
property
2024-04-22 09:17:44 +02:00
mfvitale
717888f6f6
DBZ-7765 Use different query for validating log position for SQLServer
2024-04-11 07:35:49 +02:00
Chris Cranford
8d096e965f
DBZ-7761 Add orai18n.jar
to improve Oracle NLS character set support
2024-04-11 06:48:12 +02:00
Roman Kudryashov
50c9f042a3
DBZ-7755 Refactor exporting to CloudEvents: remove RecordParser
2024-04-08 13:22:31 +02:00
mfvitale
ae7628a732
DBZ-7732 Add ConnectorSpecific annotation to manage multiple SPI connector specific implementation
2024-04-08 12:46:53 +02:00
Chris Cranford
4ca0e7ffb9
DBZ-7753 Oracle converter for RAW
to STRING
conversion
2024-04-08 12:35:30 +02:00
mfvitale
ec3c68b0d8
DBZ-7717 Move LogPositionValidator outside JdbcConnection
2024-04-05 11:57:43 +02:00
Chris Cranford
c8bc496d63
DBZ-7741 Avoid write-behind cache clear error
2024-04-05 10:49:52 +02:00
mfvitale
1190389edc
DBZ-7718 Support table name escaping in BlockingSnapshotIT
2024-04-04 14:58:24 +02:00
jchipmunk
499a1f61b8
DBZ-7737 Column CON_ID is missing in view V$THREAD in Oracle 11
2024-04-03 23:23:46 -04:00
Debezium Builder
ea182d83f3
[maven-release-plugin] prepare for next development iteration
2024-04-02 07:38:53 +00:00
Debezium Builder
7dcd94d19e
[maven-release-plugin] prepare release v2.6.0.Final
2024-04-02 07:38:53 +00:00
mfvitale
213456c5af
DBZ-7497 Properties related to configuration based snapshot mode are now optional
2024-03-28 08:59:54 +01:00
mfvitale
ee5f25ccc9
DBZ-7497 Add a configuration based snapshot modes configurable via connector properties
2024-03-28 08:59:54 +01:00
Debezium Builder
4df18d9f43
[maven-release-plugin] prepare for next development iteration
2024-03-25 09:57:05 +00:00
Debezium Builder
9656da1fad
[maven-release-plugin] prepare release v2.6.0.CR1
2024-03-25 09:57:04 +00:00
mfvitale
c7a28021a6
DBZ-7308 Remove connector specific SnapshotLockProvider and SnapshotterServiceProvider
2024-03-25 05:46:46 +01:00
mfvitale
60a0b1b5c3
DBZ-7308 tableLockingStatement from SnapshotLock interface now takes just one table in input
2024-03-25 05:46:46 +01:00
mfvitale
ae9009303c
DBZ-7308 Move getSnapshottingTask to RelationalSnapshotChangeEventSource
2024-03-25 05:46:46 +01:00
mfvitale
265ba0f6e6
DBZ-7308 Check if configured snapshot mode permits streaming before starting it.
2024-03-25 05:46:46 +01:00
Chris Cranford
bad5a964c1
DBZ-6896 Enable XML tests for Oracle OpenLogReplicator
...
OpenLogReplicator 1.5 introduces XML support that works with Debezium. Removed all
skip annotations on such tests and updated documentation.
2024-03-19 18:45:58 -04:00
Chris Cranford
9f6a26b0af
DBZ-7389 Log exceptions
2024-03-19 13:53:08 +01:00
Chris Cranford
0d62e0da06
DBZ-7389 Oracle log collector with full RAC support
2024-03-19 13:53:08 +01:00
Chris Cranford
97b5e335e2
DBZ-7458 Fix PK change LOB re-query with converted values
2024-03-18 16:05:29 -04:00
Chris Cranford
200f9ed28e
DBZ-7596 Improved tests for reselect post processor
2024-03-18 13:44:14 +01:00
Chris Cranford
e26d3efedc
DBZ-7645 Revert removal of LogMiner continuous mining
2024-03-18 13:08:04 +01:00
jchipmunk
43ed0241ed
DBZ-7619 Metric ScnFreezeCount never increases
2024-03-10 22:58:15 -04:00
Chris Cranford
05df682624
DBZ-7610 Remove deprecated, undocumented continuous mining option
2024-03-07 08:55:06 -05:00
Debezium Builder
2fb8fc3004
[maven-release-plugin] prepare for next development iteration
2024-03-06 07:47:33 +00:00
Debezium Builder
cd46b2b998
[maven-release-plugin] prepare release v2.6.0.Beta1
2024-03-06 07:47:33 +00:00
Chris Cranford
0dfb66235f
DBZ-7598 Permit Infinispan nullable XML value serialization
2024-03-04 13:39:18 -05:00
jchipmunk
edac20f658
DBZ-7473 Defer transaction capture until the first DML event occurs
...
Do not handle start event from V$LOGMNR_CONTENTS view if CLOB/BLOB support is disabled and memory buffer is used.
2024-03-04 13:38:46 -05:00
Chris Cranford
cd1b8ce71c
DBZ-7597 No need to install driver, it's included by default
2024-03-04 13:37:47 -05:00
Chris Cranford
6fdaa90d22
DBZ-3401 Use SkipLongRunning on several tests
2024-03-03 00:46:23 -05:00
Chris Cranford
2c2c914589
DBZ-3401 Try and centralize many hex-to-raw checks
2024-03-03 00:46:23 -05:00
Chris Cranford
84ba268db1
DBZ-3401 Fix test failures
2024-03-03 00:46:23 -05:00
Chris Cranford
4eaed28d6d
DBZ-3401 Fix test failure due to mock method visibility
2024-03-03 00:46:23 -05:00
Chris Cranford
17faf65802
DBZ-3401 Add existing schema history tests
2024-03-03 00:46:23 -05:00
Chris Cranford
0a0e499cdc
DBZ-3401 Add TemporalPrecisionMode tests
2024-03-03 00:46:23 -05:00
Chris Cranford
31d25c6ac1
DBZ-3401 Add DecimalHandlingMode tests
2024-03-03 00:46:23 -05:00
Chris Cranford
c3dcaf92a4
DBZ-3401 Add test for NUMBER(36,-2) as VariableScaleDecimal
2024-03-03 00:46:23 -05:00
Chris Cranford
ab61686e44
DBZ-3401 Remove todos, fix typo
2024-03-03 00:46:23 -05:00
Chris Cranford
af42a22f2e
DBZ-3401 Run HybridMiningStrategyIT only for LogMiner (fix JUnit rule)
2024-03-03 00:46:23 -05:00
Chris Cranford
86799e7cef
DBZ-3401 Introduce SkipWhenLogMiningStrategyIs
rule
2024-03-03 00:46:23 -05:00
Chris Cranford
e33901959d
DBZ-3401 Don't apply attributes if table not in schema
2024-03-03 00:46:23 -05:00
Chris Cranford
d245034d1e
DBZ-3401 Fix test parameter bindings
2024-03-03 00:46:23 -05:00
Chris Cranford
648db88868
DBZ-3401 Add new hybrid
mining strategy
2024-03-03 00:46:23 -05:00
mfvitale
446931a349
DBZ-7461 validate ArchiveLogDestination for all Oracle flavors
2024-03-01 14:12:31 +01:00
mfvitale
d142c7ec52
DBZ-7461 Fix getOffsetScn for XStreamAdapter
2024-03-01 14:12:31 +01:00
mfvitale
ff1f38ae0b
DBZ-7461 Fix getOffsetScn for XStreamAdapter
2024-03-01 14:12:31 +01:00
mfvitale
e174eae235
DBZ-7461 Remove redundant log
2024-03-01 14:12:31 +01:00
mfvitale
ce17bdcce6
DBZ-7461 Add generic properties for redo log archive and retention
2024-03-01 14:12:31 +01:00
mfvitale
4ade54351f
DBZ-7461 Rename SCHEMA_ONLY_RECOVERY to RECOVERY and SCHEMA_ONLY to NO_DATA
2024-03-01 14:12:31 +01:00
mfvitale
211675a355
DBZ-7461 Rename shouldSnapshot to shouldSnapshotData
2024-03-01 14:12:31 +01:00
mfvitale
562c999da4
DBZ-7461 Move out-of-the-box snapshotters to core
2024-03-01 14:12:31 +01:00
mfvitale
371905f135
DBZ-7461 Move snapshot mode validation logic to BaseSourceTask
2024-03-01 14:12:31 +01:00
mfvitale
99eee98e9d
DBZ-7461 Align common code for Snapshotters
2024-03-01 14:12:31 +01:00
mfvitale
b1a907757d
DBZ-7461 Support WhenNeededSnapshotter for Oracle
2024-03-01 14:12:31 +01:00
mfvitale
1f52b81b6d
DBZ-7461 Support AlwaysSnapshotter for MySQL
2024-03-01 14:12:31 +01:00
James Johnston
f632fa081e
DBZ-5071 Correctly handle NULL values in incremental snapshots
...
It turns out that the existing code for chunking a table when taking
an incremental snapshot was buggy and did not correctly handle NULL
values when building the chunk query. An example of such a situation
would be when the user has specified "message.key.columns" to reference
a column that is part of a PostgreSQL UNIQUE INDEX that was created with
the NULLS NOT DISTINCT option.
This commit updates the new AbstractChunkQueryBuilder so that it checks
whether a key column is optional. If it is, then additional will
appropriately consider NULL values when generating a chunk query using
"IS [NOT] NULL" clauses.
One complication is that different database engines have different
sorting behavior of ORDER BY. It is apparently not well-defined by the
SQL standard. Some databases consider NULL values to be higher than any
non-NULL values, and others consider them to be lower.
To handle this situation, a new nullsSortLast() function is added to the
JdbcConnection class. By default, it returns an empty value, indicating
that the behavior of the database engine is unknown. When an optional
field is encountered by AbstractChunkQueryBuilder in this situation, we
throw an error because we don't actually know how to correctly chunk the
query: there's no safe assumption that can be made here.
Derived JdbcConnection classes can then override the nullsSortLast
function, and return a value indicating the actual behavior of that
database engine. When this is done, the AbstractChunkQueryBuilder then
knows how to correctly build a chunk query that can handle NULL values.
To help test this, new tests have been added to
AbstractIncrementalSnapshotTest. First, the existing insertsWithoutPks
test has been moved and deduplicated from MySQL and PostgreSQL so that
the test case can be reused on other engines. Second, a new
insertsWithoutPksAndNull test is run, which inserts data with NULL
values in the message key columns. To demonstrate that chunk queries
are being correctly generated for practically every case, the
INCREMENTAL_SNAPSHOT_CHUNK_SIZE is set to 1 so that NULL values are not
returned in the middle of a chunk, which can cause us to skip testing
the code we need to test.
2024-02-29 13:36:26 +01:00
Chris Cranford
66e23613ce
DBZ-7072 Retry Oracle Flashback-based snapshot queries
2024-02-29 13:33:10 +01:00
Chris Cranford
5c144cd0d0
DBZ-7559 Retry XStream ORA-23656 exceptions
2024-02-23 07:31:59 -05:00
rkerner
1e69e40ec2
DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Moved deduplication from Map to LinkedHashSet.
...
+ minor fixes added for cleanup and centralization of common code
closes https://issues.redhat.com/browse/DBZ-7416
2024-02-22 13:34:08 -05:00
Jiri Pechanec
68b6591142
DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Moved deduplication from Map to LinkedHashSet.
...
closes https://issues.redhat.com/browse/DBZ-7416
2024-02-22 13:34:08 -05:00
mfvitale
440c2cb509
DBZ-7302 Increase "task.management.timeout.ms" fro Oracles tests
2024-02-20 14:45:59 +01:00
mfvitale
d9deb7a944
DBZ-7302 Increase "task.management.timeout.ms" for Oracles tests
2024-02-20 14:45:59 +01:00
mfvitale
b6149d8f68
DBZ-7302 Increase "task.management.timeout.ms" fro Oracles tests
2024-02-20 14:45:59 +01:00
mfvitale
7ed5649e07
DBZ-7302 Implement Snapshotter for Oracle
2024-02-20 14:45:59 +01:00
mfvitale
6392a1b816
DBZ-7302 Implement SnapshotLock and SnapshotQuery for Oracle connector
2024-02-20 14:45:59 +01:00
Chris Cranford
2af17ac69c
DBZ-7365 Upgrade Oracle ojdbc8 from 21.6.0.0 to 21.11.0.0
2024-02-19 14:25:25 +01:00
Chris Cranford
2034d48cf9
DBZ-6960 Fail validation with config state mismatch
2024-02-19 14:02:06 +01:00
Chris Cranford
aab87365d9
DBZ-6960 Include REDO SQL in Oracle source info block
2024-02-19 14:02:06 +01:00
Chris Cranford
340234c2ac
DBZ-6679 Only advance LogMiner start position when query has data
2024-02-19 13:52:41 +01:00
Chris Cranford
a75744547a
DBZ-7364 Distribute Oracle ojdbc8 with Oracle connector
2024-02-19 08:54:50 +01:00
Chris Cranford
438f0adcde
DBZ-7504 Fix LogMinerHelperIT random test failure
2024-02-18 13:03:50 -05:00
jchipmunk
79763211cb
DBZ-7479 Refactor code to support re-selection without flashback
...
Because OracleConnection uses flashback query (AS OF SCN) to re-select row, it is potentially possible to get "ORA-01555 Snapshot too old" error, which can be solved by performing reselection without flashback to get at least its latest row state.
2024-02-17 11:21:45 -05:00
jchipmunk
f50aa7a987
DBZ-7479 PreparedStatement leak in Oracle ReselectColumnsProcessor
...
Each time, Oracle connector creates a new instance of PreparedStatement because value of commit SCN is added directly to SQL query to reselect column values.
2024-02-17 11:21:45 -05:00
Chris Cranford
e3472d8082
DBZ-7489 Correctly handle setting XML field to "NULL"
2024-02-16 12:54:19 +01:00
Chris Cranford
4978312dad
DBZ-7489 Correctly parse XML preamble SQL ending with "IS NULL"
2024-02-16 12:54:19 +01:00
Chris Cranford
928aa26cdb
DBZ-7107 Fix source info test failures
2024-02-16 12:52:20 +01:00
Debezium Builder
10e327602c
[maven-release-plugin] prepare for next development iteration
2024-02-13 09:20:04 +00:00
Debezium Builder
0c5b05738c
[maven-release-plugin] prepare release v2.6.0.Alpha2
2024-02-13 09:20:04 +00:00
Vojtech Juranek
4452e3d095
DBZ-7024 Move EmbeddedConfig into separate class
...
`EmbeddedConfig` needs to be shared with other implmentations of
`DebeziumEngine` as long as Debezium embedded depends on the Kafka
model.
2024-02-12 13:43:21 +01:00
Chris Cranford
9fbcd6baa1
DBZ-7456 Skip PK change LOB column reselect if column is excluded
2024-02-09 16:43:13 -05:00
Chris Cranford
e2d03d9936
DBZ-7425 Use adapter-specific offset loaders when copying offsets
2024-02-07 19:40:30 -05:00
Chris Cranford
a597a82c19
DBZ-7439 Remove unnecessary log/output entries
2024-02-07 15:00:27 +01:00
Chris Cranford
de9364bb4d
DBZ-7439 Fix test compatibility
2024-02-07 15:00:27 +01:00
Ondrej Babec
c806a04531
DBZ-7439 [ci] Add Oracle 23 initialisation scripts
...
* Create oracle-init dir with SQL init scripts
* Create init-db.sh to set default password and run init scripts
* Update test.sh to run init-db.sh for oracle-23
2024-02-07 15:00:27 +01:00
Chris Cranford
3d22ed30a3
DBZ-7439 [ci] Update TF with Oracle 23
...
* Introduce new InstantClient version property
Needed because ojdbc8 versions for Oracle 23 have different version semantics
Also adjusted documentation/build pipelines
* Introduce oracle-23 profile
Allows testing Oracle 23 separately from distribution builds
2024-02-07 15:00:27 +01:00
mfvitale
f78d0cbd19
DBZ-7301 Add SnapshotterService to ChangeEventSourceFactory and SnapshotChangeEventSource
2024-02-06 07:12:42 +01:00
jchipmunk
c51ef3099a
DBZ-7445 LogMiner batch size does not increase automatically
...
The condition for increasing the LogMiner batch size must depend on log.mining.batch.size.max, and not on log.mining.batch.size.min.
2024-02-05 12:28:16 -05:00
Chris Cranford
c669857760
DBZ-7446 Fix test failure
2024-02-05 12:26:39 -05:00