alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
10,806 Commits 3 Branches 0 Tags 51 MiB
40899c2269
Commit Graph

1456 Commits

Author SHA1 Message Date
rkerner
56e63b98a8 DBZ-7223 Add the MongoDB sink connector 
* add basic tests for MongoDB sink connector (MySQL to MongoDB 1 node ReplicaSet and multi-node sharded cluster)

closes https://issues.redhat.com/browse/DBZ-7223
 2024-07-04 13:21:46 +02:00
Debezium Builder
ee2c192ae0 [maven-release-plugin] prepare for next development iteration 2024-06-28 04:36:37 -04:00
Debezium Builder
d47ba9bcb0 [maven-release-plugin] prepare release v2.7.0.Final 2024-06-28 04:36:37 -04:00
Chris Cranford
4c3781a22f DBZ-7944 Revert exclude signal data collection from Oracle supplemental log check 2024-06-14 07:54:55 +02:00
Chris Cranford
7c2ca32b6b DBZ-7944 Exclude signal data collection from Oracle supplemental log check 2024-06-14 07:54:55 +02:00
Chris Cranford
8f02d9bf39 DBZ-7942 Fix test failures 2024-06-14 07:44:56 +02:00
Chris Cranford
c06b1ce0be DBZ-7942 Address parsing signal tables with special characters 2024-06-14 07:44:56 +02:00
Debezium Builder
db691fd16f [maven-release-plugin] prepare for next development iteration 2024-06-13 09:04:36 +00:00
Debezium Builder
73ce5c5058 [maven-release-plugin] prepare release v2.7.0.Beta2 2024-06-13 09:04:36 +00:00
Chris Cranford
22efd5a367 DBZ-7831 Allow streaming from tables with single-quotes in name 2024-06-13 07:07:57 +02:00
Chris Cranford
f1335d9926 DBZ-7823 Add serialization for ROW_ID for OpenLogReplicator 2024-06-10 15:14:35 -04:00
jchipmunk
673ed67a6f DBZ-7935 OlrNetworkClient does not disconnect when error occurs 
OlrNetworkClient should disconnect if an unexpected error occurs while reading events from OpenLogReplicator.
 2024-06-10 14:20:43 +02:00
Debezium Builder
5273bb1601 [maven-release-plugin] prepare for next development iteration 2024-06-06 05:44:28 +00:00
Debezium Builder
a79c9ef799 [maven-release-plugin] prepare release v2.7.0.Beta1 2024-06-06 05:44:27 +00:00
Chris Cranford
bfa664189c DBZ-7882 Apply decimal.handling.mode to 0-scale numeric values 2024-05-31 11:55:29 +02:00
Jiri Pechanec
b052d8a472 DBZ-7880 Make schema backward compatible 2024-05-22 14:20:02 +02:00
Jiri Pechanec
c09c2c6712 DBZ-7641 Update tests 2024-05-15 05:42:12 +02:00
Debezium Builder
34a7c11de6 [maven-release-plugin] prepare for next development iteration 2024-05-13 06:42:03 +00:00
Debezium Builder
ebf74a8933 [maven-release-plugin] prepare release v2.7.0.Alpha2 2024-05-13 06:42:03 +00:00
Chris Cranford
964603fa6a DBZ-7790 Reworked to use introspection to determine merge 2024-05-10 09:40:24 +02:00
Chris Cranford
e9a1e2c108 DBZ-7790 Checkstyle fixes 2024-05-10 09:40:24 +02:00
liaoyuxing
b8fb83cd90 DBZ-7790 add test in OracleClobDataTypesIT and OracleBlobDataTypesIT 2024-05-10 09:40:24 +02:00
liaoyuxing
e0b28e3c66 DBZ-7790 fix Debezium oracle connector lost data of the update operation when the table has no primary key and contains lob type 2024-05-10 09:40:24 +02:00
Chris Cranford
ed65216e4f DBZ-7847 Support in query filter mode with 1000+ tables 2024-05-10 09:37:56 +02:00
Chris Cranford
c5f42058c1 DBZ-4332 Add row_id to Oracle streaming event's source info block 2024-05-06 10:40:26 +02:00
Chris Cranford
8b029f9cc0
DBZ-7839 Increase test shouldGracefullySkipObjectBasedTables wait time 2024-05-01 08:43:55 -04:00
Chris Cranford
09af8c1e85 DBZ-7819 Fix test failure due to case 2024-05-01 08:40:19 -04:00
Chris Cranford
3127284ac3 DBZ-7819 Remove creation of temporary objects 2024-05-01 08:40:19 -04:00
Chris Cranford
f23734c5d6 DBZ-7819 Allow log.mining.flush.table.name to optionally include schema 2024-05-01 08:40:19 -04:00
Chris Cranford
35bc10a97a DBZ-7833 Move undocumented option to internal namespace 2024-04-30 08:53:15 +02:00
Debezium Builder
bcac0c5c7a [maven-release-plugin] prepare for next development iteration 2024-04-25 09:21:27 +00:00
Debezium Builder
0be6e6b8fd [maven-release-plugin] prepare release v2.7.0.Alpha1 2024-04-25 09:21:26 +00:00
mfvitale
a708ef1009 DBZ-7616 Simplify Oracle tests assertion on SQLTimeoutException 2024-04-22 09:17:44 +02:00
mfvitale
30c11242ac DBZ-7616 Avoid to filter out every SET STATEMENT queries and filter just the ones related to Amazon RDS 2024-04-22 09:17:44 +02:00
mfvitale
8206d2d98d DBZ-7616 Add query timeout on JdbcConnection controlled by query.timeout.ms property 2024-04-22 09:17:44 +02:00
mfvitale
717888f6f6 DBZ-7765 Use different query for validating log position for SQLServer 2024-04-11 07:35:49 +02:00
Chris Cranford
8d096e965f DBZ-7761 Add orai18n.jar to improve Oracle NLS character set support 2024-04-11 06:48:12 +02:00
Roman Kudryashov
50c9f042a3 DBZ-7755 Refactor exporting to CloudEvents: remove RecordParser 2024-04-08 13:22:31 +02:00
mfvitale
ae7628a732 DBZ-7732 Add ConnectorSpecific annotation to manage multiple SPI connector specific implementation 2024-04-08 12:46:53 +02:00
Chris Cranford
4ca0e7ffb9 DBZ-7753 Oracle converter for RAW to STRING conversion 2024-04-08 12:35:30 +02:00
mfvitale
ec3c68b0d8 DBZ-7717 Move LogPositionValidator outside JdbcConnection 2024-04-05 11:57:43 +02:00
Chris Cranford
c8bc496d63 DBZ-7741 Avoid write-behind cache clear error 2024-04-05 10:49:52 +02:00
mfvitale
1190389edc DBZ-7718 Support table name escaping in BlockingSnapshotIT 2024-04-04 14:58:24 +02:00
jchipmunk
499a1f61b8 DBZ-7737 Column CON_ID is missing in view V$THREAD in Oracle 11 2024-04-03 23:23:46 -04:00
Debezium Builder
ea182d83f3 [maven-release-plugin] prepare for next development iteration 2024-04-02 07:38:53 +00:00
Debezium Builder
7dcd94d19e [maven-release-plugin] prepare release v2.6.0.Final 2024-04-02 07:38:53 +00:00
mfvitale
213456c5af DBZ-7497 Properties related to configuration based snapshot mode are now optional 2024-03-28 08:59:54 +01:00
mfvitale
ee5f25ccc9 DBZ-7497 Add a configuration based snapshot modes configurable via connector properties 2024-03-28 08:59:54 +01:00
Debezium Builder
4df18d9f43 [maven-release-plugin] prepare for next development iteration 2024-03-25 09:57:05 +00:00
Debezium Builder
9656da1fad [maven-release-plugin] prepare release v2.6.0.CR1 2024-03-25 09:57:04 +00:00
mfvitale
c7a28021a6 DBZ-7308 Remove connector specific SnapshotLockProvider and SnapshotterServiceProvider 2024-03-25 05:46:46 +01:00
mfvitale
60a0b1b5c3 DBZ-7308 tableLockingStatement from SnapshotLock interface now takes just one table in input 2024-03-25 05:46:46 +01:00
mfvitale
ae9009303c DBZ-7308 Move getSnapshottingTask to RelationalSnapshotChangeEventSource 2024-03-25 05:46:46 +01:00
mfvitale
265ba0f6e6 DBZ-7308 Check if configured snapshot mode permits streaming before starting it. 2024-03-25 05:46:46 +01:00
Chris Cranford
bad5a964c1 DBZ-6896 Enable XML tests for Oracle OpenLogReplicator 
OpenLogReplicator 1.5 introduces XML support that works with Debezium. Removed all
skip annotations on such tests and updated documentation.
 2024-03-19 18:45:58 -04:00
Chris Cranford
9f6a26b0af DBZ-7389 Log exceptions 2024-03-19 13:53:08 +01:00
Chris Cranford
0d62e0da06 DBZ-7389 Oracle log collector with full RAC support 2024-03-19 13:53:08 +01:00
Chris Cranford
97b5e335e2 DBZ-7458 Fix PK change LOB re-query with converted values 2024-03-18 16:05:29 -04:00
Chris Cranford
200f9ed28e DBZ-7596 Improved tests for reselect post processor 2024-03-18 13:44:14 +01:00
Chris Cranford
e26d3efedc DBZ-7645 Revert removal of LogMiner continuous mining 2024-03-18 13:08:04 +01:00
jchipmunk
43ed0241ed DBZ-7619 Metric ScnFreezeCount never increases 2024-03-10 22:58:15 -04:00
Chris Cranford
05df682624 DBZ-7610 Remove deprecated, undocumented continuous mining option 2024-03-07 08:55:06 -05:00
Debezium Builder
2fb8fc3004 [maven-release-plugin] prepare for next development iteration 2024-03-06 07:47:33 +00:00
Debezium Builder
cd46b2b998 [maven-release-plugin] prepare release v2.6.0.Beta1 2024-03-06 07:47:33 +00:00
Chris Cranford
0dfb66235f DBZ-7598 Permit Infinispan nullable XML value serialization 2024-03-04 13:39:18 -05:00
jchipmunk
edac20f658 DBZ-7473 Defer transaction capture until the first DML event occurs 
Do not handle start event from V$LOGMNR_CONTENTS view if CLOB/BLOB support is disabled and memory buffer is used.
 2024-03-04 13:38:46 -05:00
Chris Cranford
cd1b8ce71c DBZ-7597 No need to install driver, it's included by default 2024-03-04 13:37:47 -05:00
Chris Cranford
6fdaa90d22 DBZ-3401 Use SkipLongRunning on several tests 2024-03-03 00:46:23 -05:00
Chris Cranford
2c2c914589 DBZ-3401 Try and centralize many hex-to-raw checks 2024-03-03 00:46:23 -05:00
Chris Cranford
84ba268db1 DBZ-3401 Fix test failures 2024-03-03 00:46:23 -05:00
Chris Cranford
4eaed28d6d DBZ-3401 Fix test failure due to mock method visibility 2024-03-03 00:46:23 -05:00
Chris Cranford
17faf65802 DBZ-3401 Add existing schema history tests 2024-03-03 00:46:23 -05:00
Chris Cranford
0a0e499cdc DBZ-3401 Add TemporalPrecisionMode tests 2024-03-03 00:46:23 -05:00
Chris Cranford
31d25c6ac1 DBZ-3401 Add DecimalHandlingMode tests 2024-03-03 00:46:23 -05:00
Chris Cranford
c3dcaf92a4 DBZ-3401 Add test for NUMBER(36,-2) as VariableScaleDecimal 2024-03-03 00:46:23 -05:00
Chris Cranford
ab61686e44 DBZ-3401 Remove todos, fix typo 2024-03-03 00:46:23 -05:00
Chris Cranford
af42a22f2e DBZ-3401 Run HybridMiningStrategyIT only for LogMiner (fix JUnit rule) 2024-03-03 00:46:23 -05:00
Chris Cranford
86799e7cef DBZ-3401 Introduce SkipWhenLogMiningStrategyIs rule 2024-03-03 00:46:23 -05:00
Chris Cranford
e33901959d DBZ-3401 Don't apply attributes if table not in schema 2024-03-03 00:46:23 -05:00
Chris Cranford
d245034d1e DBZ-3401 Fix test parameter bindings 2024-03-03 00:46:23 -05:00
Chris Cranford
648db88868 DBZ-3401 Add new hybrid mining strategy 2024-03-03 00:46:23 -05:00
mfvitale
446931a349 DBZ-7461 validate ArchiveLogDestination for all Oracle flavors 2024-03-01 14:12:31 +01:00
mfvitale
d142c7ec52 DBZ-7461 Fix getOffsetScn for XStreamAdapter 2024-03-01 14:12:31 +01:00
mfvitale
ff1f38ae0b DBZ-7461 Fix getOffsetScn for XStreamAdapter 2024-03-01 14:12:31 +01:00
mfvitale
e174eae235 DBZ-7461 Remove redundant log 2024-03-01 14:12:31 +01:00
mfvitale
ce17bdcce6 DBZ-7461 Add generic properties for redo log archive and retention 2024-03-01 14:12:31 +01:00
mfvitale
4ade54351f DBZ-7461 Rename SCHEMA_ONLY_RECOVERY to RECOVERY and SCHEMA_ONLY to NO_DATA 2024-03-01 14:12:31 +01:00
mfvitale
211675a355 DBZ-7461 Rename shouldSnapshot to shouldSnapshotData 2024-03-01 14:12:31 +01:00
mfvitale
562c999da4 DBZ-7461 Move out-of-the-box snapshotters to core 2024-03-01 14:12:31 +01:00
mfvitale
371905f135 DBZ-7461 Move snapshot mode validation logic to BaseSourceTask 2024-03-01 14:12:31 +01:00
mfvitale
99eee98e9d DBZ-7461 Align common code for Snapshotters 2024-03-01 14:12:31 +01:00
mfvitale
b1a907757d DBZ-7461 Support WhenNeededSnapshotter for Oracle 2024-03-01 14:12:31 +01:00
mfvitale
1f52b81b6d DBZ-7461 Support AlwaysSnapshotter for MySQL 2024-03-01 14:12:31 +01:00
James Johnston
f632fa081e DBZ-5071 Correctly handle NULL values in incremental snapshots 
It turns out that the existing code for chunking a table when taking
an incremental snapshot was buggy and did not correctly handle NULL
values when building the chunk query.  An example of such a situation
would be when the user has specified "message.key.columns" to reference
a column that is part of a PostgreSQL UNIQUE INDEX that was created with
the NULLS NOT DISTINCT option.

This commit updates the new AbstractChunkQueryBuilder so that it checks
whether a key column is optional.  If it is, then additional will
appropriately consider NULL values when generating a chunk query using
"IS [NOT] NULL" clauses.

One complication is that different database engines have different
sorting behavior of ORDER BY.  It is apparently not well-defined by the
SQL standard.  Some databases consider NULL values to be higher than any
non-NULL values, and others consider them to be lower.

To handle this situation, a new nullsSortLast() function is added to the
JdbcConnection class.  By default, it returns an empty value, indicating
that the behavior of the database engine is unknown.  When an optional
field is encountered by AbstractChunkQueryBuilder in this situation, we
throw an error because we don't actually know how to correctly chunk the
query: there's no safe assumption that can be made here.

Derived JdbcConnection classes can then override the nullsSortLast
function, and return a value indicating the actual behavior of that
database engine.  When this is done, the AbstractChunkQueryBuilder then
knows how to correctly build a chunk query that can handle NULL values.

To help test this, new tests have been added to
AbstractIncrementalSnapshotTest.  First, the existing insertsWithoutPks
test has been moved and deduplicated from MySQL and PostgreSQL so that
the test case can be reused on other engines.  Second, a new
insertsWithoutPksAndNull test is run, which inserts data with NULL
values in the message key columns.  To demonstrate that chunk queries
are being correctly generated for practically every case, the
INCREMENTAL_SNAPSHOT_CHUNK_SIZE is set to 1 so that NULL values are not
returned in the middle of a chunk, which can cause us to skip testing
the code we need to test.
 2024-02-29 13:36:26 +01:00
Chris Cranford
66e23613ce DBZ-7072 Retry Oracle Flashback-based snapshot queries 2024-02-29 13:33:10 +01:00
Chris Cranford
5c144cd0d0 DBZ-7559 Retry XStream ORA-23656 exceptions 2024-02-23 07:31:59 -05:00
rkerner
1e69e40ec2 DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Moved deduplication from Map to LinkedHashSet. 
+ minor fixes added for cleanup and centralization of common code

closes https://issues.redhat.com/browse/DBZ-7416
 2024-02-22 13:34:08 -05:00
Jiri Pechanec
68b6591142 DBZ-7416 Fix duplicate SMTs sometimes returned by Kafka Connect. Moved deduplication from Map to LinkedHashSet. 
closes https://issues.redhat.com/browse/DBZ-7416
 2024-02-22 13:34:08 -05:00
mfvitale
440c2cb509 DBZ-7302 Increase "task.management.timeout.ms" fro Oracles tests 2024-02-20 14:45:59 +01:00
mfvitale
d9deb7a944 DBZ-7302 Increase "task.management.timeout.ms" for Oracles tests 2024-02-20 14:45:59 +01:00