alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
8,365 Commits 3 Branches 0 Tags 51 MiB
30eb53c764
Commit Graph

848 Commits

Author SHA1 Message Date
Vojtech Juranek
b797201263 DBZ-5043 Rename DB_HISTORY_PATH to SCHEMA_HISTORY_PATH 2022-09-07 12:57:31 +02:00
Vojtech Juranek
b6c31f3da5 DBZ-5043 Replace 'database history' with 'database schema history' 
Reflect renaming database history to schema history in comments,
configuration descriptions and log and error messages.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
0494a16a79 DBZ-5043 Rename DatabaseHistory to SchemaHistory 
Rename also all related classes and varibales.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
0823cedf25 DBZ-5043 Replace database.server.name with topic.prefix 2022-09-07 12:57:31 +02:00
Vojtech Juranek
161a7229b6 DBZ-5043 Use topic prefix as the logical name 
With this change the user cannot direcly set connector logical name and
thus in the future it can be remove without breaking user config.
If the turn out that the logical name is useful and user should be able
to configure it, dedicated config option can be added.

Originally it was prposed in the Jira to replace it with connector name,
but it turned out that logical name defaults to `database.server.name`
and is heavily used in the tests and JMX, so it would require another
big refactoring. Thus, use topic prefix for now. Once we know further
direction (remove logical name or add new option), do this refactoring.
 2022-09-07 12:57:31 +02:00
Vojtech Juranek
25aa6c4acc DBZ-5043 Change namespace to database.history to schema.history 2022-09-07 12:57:31 +02:00
Vadzim Ramanenka
917de37a01 DBZ-5557: Supply partition when comiting offsets with source database 
Keep track of partitions offsets being committed belong to in
`BaseSourceTask`. Supply the partition along with the offset in calls to
`commitOffset`.
 2022-09-05 14:04:23 +02:00
ggaborg
244de2bd96 DBZ-3293 Add signal table automatically to include list 2022-09-01 20:16:47 +02:00
Chris Cranford
b5754a19ec DBZ-5439 Remove rsid and ssn from CommitScn format 2022-09-01 09:32:17 +02:00
Chris Cranford
c0f92e24ea DBZ-5439 Updated tests, use TreeSet trx id deterministic order 2022-09-01 09:32:17 +02:00
thangdc94
57210095d2 DBZ-5439 Fix testcase 2022-09-01 09:32:17 +02:00
thangdc94
c755720637 DBZ-5439 Backward compatible with old commit scn 2022-09-01 09:32:17 +02:00
thangdc94
cfa8b780ed DBZ-5439 Add transaction id associate with commit scn 2022-09-01 09:32:17 +02:00
harveyyue
7ccd40129c DBZ-5569 Oracle connector's max.queue.size.in.bytes is invalid 2022-08-31 09:39:29 -04:00
pkgonan
b031770300 DBZ-5544 Support BASE64_URL_SAFE in BinaryHandlingMode 2022-08-24 09:50:23 +02:00
Chris Cranford
14c9f8855b DBZ-5327 Fix Oracle/Db2 test failures 2022-08-19 11:33:01 -04:00
Chris Cranford
176529e83b DBZ-5521 Fix concatenation operator handling in LogMiner DML parser 2022-08-19 11:19:45 +02:00
Chris Cranford
37b97970a6 DBZ-5497 Skip truncate operations by default 
* Promote truncate skip default from PostgreSQL to core library
* Adjust behavior for Oracle with truncates skipped by default
 2022-08-19 11:05:29 +02:00
Vivek Wassan
d89e71ebeb DBZ-5327 Smart Backfills 2022-08-17 10:05:55 +02:00
Chris Cranford
41a9324044 DBZ-5441 Gracefully skip Oracle non-relational tables 2022-07-29 12:56:27 -04:00
harveyyue
5f430f1a8e DBZ-5457 Deprecate legacy topic selector for all connectors 2022-07-28 12:58:59 -04:00
Chris Cranford
412498db41
DBZ-5295 Avoid NPE With Xstream emitter 2022-07-26 11:05:17 -04:00
Chris Cranford
2c2cb9213a DBZ-5248 Avoid NumberFormatException in OracleConnectorIT tests 2022-07-20 13:21:59 -04:00
Chris Cranford
f8d097ebc3
DBZ-5419 Fix OracleSchemaMigrationIT (non-CDB) test failure 2022-07-19 09:23:16 -04:00
Anisha Mohanty
04941330c3 DBZ-5390 Reorder relational_property definition to give priority to out_of_line_constraint 2022-07-14 08:55:21 +02:00
Chris Cranford
f62fab4364 DBZ-5404 Use fail from fest assertions 2022-07-14 08:40:42 +02:00
harveyyue
6ea7abc8fc DBZ-4180 Pluggable topic selector 2022-07-13 14:45:25 +02:00
Jiri Pechanec
d8baac7c46 DBZ-5229 Reorganize code for inheritance use 2022-07-12 17:31:19 +02:00
Hossein Torabi
1ea02be9c2 DBZ-5229 define centralized and modular aproach for debezium storage 2022-07-12 17:31:19 +02:00
Chris Cranford
35e3eae52f DBZ-5295 Add delete row test case 2022-07-12 11:59:34 +02:00
Chris Cranford
843f133d4d DBZ-5295 Use connection quote method, rework unavailable value check 2022-07-12 11:59:34 +02:00
Chris Cranford
5f443dd9e1 DBZ-5295 Fix format errors 2022-07-12 11:59:34 +02:00
Chris Cranford
6735904075 DBZ-5295 Cache LOB column status in schema model 2022-07-12 11:59:34 +02:00
Chris Cranford
32947ae512 DBZ-5295 Remove unnecessary test output 2022-07-12 11:59:34 +02:00
Chris Cranford
d2a76a7ffc DBZ-5295 Reselect LOB columns on primary key update 2022-07-12 11:59:34 +02:00
Chris Cranford
deaa34d71e DBZ-5356 Avoid NPE during index evaluation 2022-07-12 11:01:10 +02:00
Chris Cranford
ddbd682d8d DBZ-5381 Add a pretty loggable format for commit_scn 2022-07-12 06:55:13 +02:00
Chris Cranford
f7696ffac7 DBZ-5358 Rename to "user_name", position at end of struct 2022-07-11 10:10:21 -04:00
Nathan Smit
28ce4f0aa2 DBZ-5358 added name of user who executed transaction to source info block 2022-07-11 10:10:21 -04:00
Chris Cranford
28c26741d6 DBZ-5351 Add warning for LogMiner rows with unsupported op type 2022-07-06 17:23:45 -04:00
Chris Cranford
9cca260197 DBZ-5373 Propagate Oracle Xstream commit timestamp to transaction metadata 2022-07-06 16:25:18 -04:00
Chris Cranford
30d95358b8 DBZ-5245 Track commit scn per redo thread 2022-07-01 06:39:19 +02:00
Chris Cranford
2be190bcdd DBZ-5281 Use separate connection for snapshot-based LogMiner queries 2022-06-30 10:37:32 +02:00
Chris Cranford
65fa2c73a5 DBZ-5281 Fix test compatibility with Oracle non-CDB 2022-06-30 10:37:32 +02:00
Mickael Maison
5ff71aa992 DBZ-5335 Fix formatting issues in the connectors configdefs 2022-06-30 09:35:54 +02:00
Chris Cranford
cf1e711299
DBZ-5266 Skip tests when using Oracle Xstream 2022-06-29 19:59:35 -04:00
harveyyue
20dc353262 DBZ-5170 Mysql Commit Timestamp 2022-06-27 15:50:52 +02:00
Chris Cranford
fd296557b5 DBZ-5266 Correctly advance commit_scn with LOB support 2022-06-27 15:24:11 +02:00
Chris Cranford
811ba8a4af
DBZ-4902 Fix compatibility with Oracle connector 2022-06-27 08:45:37 -04:00
Chris Cranford
cfc63776fa DBZ-5300 Exclude ADT indices as relational primary keys 2022-06-27 10:38:56 +02:00
Chris Cranford
5d4d659b37 DBZ-5285 Fix store.only.captured.tables.ddl to work during streaming 2022-06-27 09:02:22 +02:00
Chris Cranford
a1d35b402e DBZ-5303 Rework SkipOnDatabaseOption, allow for enabled/disabled toggles 2022-06-23 18:58:05 -04:00
Chris Cranford
cab3c664a1 DBZ-5301 FIx LogMinerHelperIT to work on Oracle RAC with multiple nodes 2022-06-23 18:54:57 -04:00
Jiri Pechanec
44107961e5 DBZ-4367 Align test with new schema changes behaviour 2022-06-17 08:53:47 +02:00
Jiri Pechanec
86abed2f9b DBZ-4834 Fix test 2022-06-16 14:52:47 +02:00
Chris Cranford
263ba5aad6 DBZ-5256 Add internal.log.mining.transaction.snapshot.boundary.mode option 
This internal option is meant to replace the old `log.mining.query.logs.for.snapshot.offset`.
This now enum-based setting provides much more flexibility by being able to completely
disable the in-progress transaction check (now the default), only grab transactions that are
in-progress from V$TRANSACTION, or finally be able to grab all in-progress transactions that
are both from V$TRANSACTION and by scanning the logs.
 2022-06-16 12:44:45 +02:00
Chris Cranford
3fb5bf1339 DBZ-4451 Introduce SchemaChangeEvent factory methods 2022-06-16 09:19:41 +02:00
Chris Cranford
b6549670c8 DBZ-4451 Allow test to be extended by connectors 2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081 DBZ-4451 Store previous table id in alter schema change events 
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
 2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5 DBZ-4834 Fix incremental snapshot of table added to include list 2022-06-16 08:39:16 +02:00
Chris Cranford
44159c95be DBZ-5225 Add EVENT_SCN_KEY key 2022-06-16 08:27:32 +02:00
Bob Roldan
18688077bf DBZ-5225 Align source.scn with the event's scn 2022-06-16 08:27:32 +02:00
Chris Cranford
4d30e6be47 DBZ-5257 Source info table order deterministic for rename schema changes 2022-06-16 08:19:38 +02:00
Chris Cranford
14a3b46ee5 DBZ-5258 Fix test assertions using static database names 2022-06-15 18:05:56 -04:00
Chris Cranford
46bef9c3ae DBZ-4890 BigDecimal scale match Field when column uses negative scale 2022-06-15 12:27:56 +02:00
Chris Cranford
e142203a86 DBZ-5226 Improve test wait-time performance 2022-06-09 15:15:22 -04:00
Vojtech Juranek
ec9625bd70 DBZ-5045 Remove deprecated Oracle DB ORACLE_VERSION option 2022-06-08 09:11:00 +02:00
Vojtech Juranek
ff5ddd7959 DBZ-5045 Remove deprecated FIELD_PAYLOAD_ID 
`table.field.event.payload.id` is replaced by `table.field.event.key`.
Also fix default value of `table.field.event.key` which was missing.
 2022-06-08 09:11:00 +02:00
Vojtech Juranek
01d8e4fe27 DBZ-5045 Remove deprecated Oracle LOG_MINING_BUFFER_LOCATION option 2022-06-08 09:11:00 +02:00
Vojtech Juranek
091e2f3162 DBZ-5045 Remove deprecated Oracle INFINISPAN option 2022-06-08 09:11:00 +02:00
Vojtech Juranek
f84893a8b4 DBZ-5045 Remove deprecated SCHEMA_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
261188482f DBZ-5045 Remove deprecated SCHEMA_WHITELIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
6abbcee6a3 DBZ-5045 Remove deprecated TABLE_BLACKLIST 2022-06-08 09:11:00 +02:00
Vojtech Juranek
db08e894fc DBZ-5045 Remove deprecated TABLE_WHITELIST 2022-06-08 09:11:00 +02:00
Jiri Pechanec
7d16c879f3 DBZ-5202 Extract constants; Use Java 8 API 2022-06-07 09:04:03 +02:00
Chris Cranford
310e788c22 DBZ-5202 Add retry/delay to offset transaction id resolution 2022-06-07 09:04:03 +02:00
Chris Cranford
ec940e8220 DBZ-5085 Enforce read-consistency determining snapshot offsets 2022-06-02 08:29:13 +02:00
Jiri Pechanec
b61d99d0c6 DBZ-5136 Fix Oracle tests 2022-06-02 07:41:00 +02:00
Chris Cranford
9d71296b5c DBZ-4251 Support stopping incremental snapshots 2022-06-02 07:30:07 +02:00
Chris Cranford
f3671afaf7 DBZ-5139 Apply threshold to all processors, added tests 2022-06-01 17:52:44 -04:00
Ethan Zou
1ac6abf63e DBZ-5139 Abandon transaction who has too many events according to new configuration 'log.mining.buffer.memory.transaction.events.threshold' 2022-06-01 17:52:44 -04:00
Jiri Pechanec
88b7ae2050 DBZ-5136 Move sourceInfo to common class 2022-06-01 16:56:29 +02:00
Mark Bereznitsky
c64b8af31a DBZ-5136 reintroduce the CommonOffsetContext class and refactor common functionality into it 2022-06-01 16:56:29 +02:00
Mark Bereznitsky
2d72dae649 DBZ-5136 further refining of the algorithm for determining position in snapshot and IT test fixes 2022-06-01 16:56:29 +02:00
Jiri Pechanec
08e8c3d795 DBZ-5136 Move common class to common package 2022-06-01 16:56:29 +02:00
Andrew Walker
d23735c658 DBZ-5136: Fix missing reference 2022-06-01 16:56:29 +02:00
Andrew Walker
d0759461a4 DBZ-5136: Convert RelationalOffsetContext to CommonOffsetContext 2022-06-01 16:56:29 +02:00
Andrew Walker
b6222d42bf DBZ-5136: Add first flag and refactor OffsetContext 2022-06-01 16:56:29 +02:00
Andrew Walker
593c2769e6 DBZ-5136: Add last in table flag 2022-06-01 16:56:29 +02:00
Chris Cranford
15e9231c30 DBZ-5179 Use LRUCacheMap to avoid high memory usage 2022-06-01 15:53:55 +02:00
Chris Cranford
a900ababcd DBZ-5042 Use PDB name when set in Oracle configuration 2022-05-27 14:20:30 -04:00
Vojtech Juranek
fa07f67cdb DBZ-5042 Add DB name logging support into Oracle connector 2022-05-27 14:20:30 -04:00
Chris Cranford
1ae7bec46e DBZ-5146 Remove log.mining.view.fetch.size, use query.fetch.size instead. 2022-05-26 19:26:14 -04:00
Chris Cranford
60b1b3cc1d DBZ-5147 Fix test failures 2022-05-26 18:37:24 -04:00
Chris Cranford
2978898641 DBZ-5147 Adjust log levels, apply test rules 2022-05-26 18:37:24 -04:00
Chris Cranford
7ab0cea8f1 DBZ-5147 Add event.processing.failure.handling.mode for Oracle 2022-05-26 18:37:24 -04:00
Chris Cranford
7d8c9486d6 DBZ-5119 Move Heartbeat creation to ConnectorConfig & descendants 2022-05-25 09:59:59 +02:00
Chris Cranford
7ff7826743 DBZ-5119 Pull heartbeat.action.query support into debezium-core 2022-05-24 12:54:40 +02:00
Chris Cranford
15eb7807c4 DBZ-5119 Add heartbeat.action.query support for Oracle 2022-05-24 12:54:40 +02:00
Chris Cranford
2b78fca8ce DBZ-5148 Retry starting Oracle LogMiner after ORA-01291 2022-05-23 10:30:18 +02:00
Chris Cranford
7577500696 DBZ-5140 Do not log transaction warning when lob-enabled is true 2022-05-18 14:11:36 -04:00
Chris Cranford
9a81339b54 DBZ-4782 Rework if-branches 2022-05-13 12:51:38 +02:00
Chris Cranford
e1579f8738 DBZ-4782 Flush SCN values upon schema changes 2022-05-13 12:51:38 +02:00
Chris Cranford
d0c6b59527 DBZ-4973 Cleanup logging calls 2022-05-10 11:33:33 -04:00
Chris Cranford
db9a104ea4 DBZ-4973 Remove unused code 2022-05-10 11:33:33 -04:00
Chris Cranford
035d6b4c14 DBZ-5090 Improve test use case 2022-05-10 09:45:34 -04:00
Chris Cranford
0d595025fd DBZ-5090 Undo events across multiple mining sessions 2022-05-10 09:45:34 -04:00
Chris Cranford
43ae067ef8 DBZ-5006 Add suggested test coverage 2022-04-27 08:52:29 -04:00
Chris Cranford
7e845f83cb DBZ-3318 Scope tests to Oracle LogMiner adapter 2022-04-27 08:52:29 -04:00
Chris Cranford
cea4b51a3b DBZ-4963 Scope tests to Oracle LogMiner adapter 2022-04-27 08:52:29 -04:00
Chris Cranford
09e58c5a5c DBZ-5006 Allow "/" characters in Oracle table names 2022-04-27 08:52:29 -04:00
Jan Doms
f698a59481 DBZ-5005 adjust LogMiner batch size based on comparison with currently used batch size 2022-04-26 14:08:00 -04:00
Chris Cranford
9e08093900 DBZ-5029 Scope several OracleConnectorIT tests to LogMiner only 
* shouldCaptureChangeForTableWithUnsupportedColumnType
* shouldContinueToUpdateOffsetsEvenWhenTableIsNotChanged
 2022-04-21 16:10:39 +02:00
Chris Cranford
cee77e5b96 DBZ-5028 Scope OracleOffsetContextTest to the LogMiner adapter 2022-04-21 15:51:02 +02:00
Chris Cranford
5f9b2fa822 DBZ-4963 Use Optional & Stopwatch, disable lengthy test by default 2022-04-21 08:05:59 +02:00
Chris Cranford
c804509673 DBZ-4963 Introduce log.mining.session.max.ms config option for Oracle 2022-04-21 08:05:59 +02:00
Chris Cranford
f24be1f533 DBZ-4994 Fix LOB locator event parsing with escaped single quotes 2022-04-19 09:07:36 +02:00
Chris Cranford
f667402fe0 DBZ-4987 Propagate up all exceptions from LCR handler 2022-04-19 07:23:39 +02:00
Chris Cranford
89fb497602 DBZ-4976 Add unit test 2022-04-14 11:58:50 +02:00
Chris Cranford
0ffd2e2ec3 DBZ-4976 Rework null checks & add nullability comments 2022-04-14 11:58:50 +02:00
Anisha Mohanty
736a3b149b DBZ-4976 Handle NPEs for column definitions of Oracle DDL statements 2022-04-14 11:58:50 +02:00
Chris Cranford
1d68c8ed53 DBZ-4978 Avoid NPE during Oracle connector shutdown 2022-04-14 10:18:40 +02:00
Chris Cranford
7b95aa7bcb DBZ-4953 Correctly handle TRUNCATE when LOB is enabled 2022-04-11 07:45:12 +02:00
Chris Cranford
e984754c2a DBZ-4932 Use LCR v1 format with Oracle 12.0 and 12.1 2022-04-11 07:09:39 +02:00
Chris Cranford
a793d9e2ca DBZ-4936 Restrict test to only memory-based buffer 2022-04-08 14:21:17 -04:00
Chris Cranford
22d300731f DBZ-3318 Add integration test for database.url 2022-04-08 11:14:30 -04:00
Chris Cranford
c7e1006dd3 DBZ-4879 Correctly resolve archive-log-only SCN mining range 2022-04-08 12:05:57 +02:00
Chris Cranford
3740d27366 DBZ-4936 Don't re-emit transaction on restart with commit SCN equal to offset commit SCN 2022-04-08 09:21:38 +02:00
Chris Cranford
785623658c DBZ-4937 Correctly read/write snapshot_scn and snapshot_pending_trx offsets 2022-04-01 13:56:04 -04:00
Chris Cranford
13a96549bc DBZ-4907 Increment Commit SCN when any COMMIT is detected 2022-03-31 10:02:38 -04:00
Chris Cranford
36ad6f23db
DBZ-4891 Remove unused variable 2022-03-30 16:06:04 -04:00
Chris Cranford
6f7fa743ed
DBZ-4862 Correctly parse double single-quotes in SEL_LOB_LOCATOR events 2022-03-30 16:06:04 -04:00
asibanovs
1fe8dbb1cd
DBZ-4891 Fix Oracle Logminer's escaped single quotes parsing in DMLs, related unit test and fix for the old DBZ-3414 test. 2022-03-30 16:06:04 -04:00
Sergei Morozov
b258674782 DBZ-3535: Introduce schema.name.adjustment.mode 2022-03-24 09:37:05 +01:00
Jiri Pechanec
60b80f6365 DBZ-4869 Add hint to warn message 2022-03-23 10:50:03 +01:00
Chris Cranford
64f22e3498 DBZ-4869 Log warning rather than exception on incorrect table-level logging level 2022-03-23 10:50:03 +01:00
Jiri Pechanec
71ffde5eaa DBZ-4900 Use Duration instead of millis 2022-03-23 09:01:59 +01:00
Chris Cranford
2dd488effb DBZ-4900 Use exponential back-off for log availability check 2022-03-23 09:01:59 +01:00
Ethan Zou
efdb5d21fc DBZ-4850 Add several retriable SQL error codes 2022-03-22 11:52:27 -04:00
Chris Cranford
80cda74e63 DBZ-4897 Suggested edits 2022-03-22 16:47:21 +01:00
Chris Cranford
337305f0e6 DBZ-4897 Parse Oracle character-based column default values with parens 2022-03-22 16:47:21 +01:00
Chris Cranford
45648632f3 DBZ-4880 Correctly interpret NCLOB in Oracle DDL parser 2022-03-18 11:05:36 +01:00
Chris Cranford
c90993a652 DBZ-4852 DBZ-4853 DDL parser pass unsupported column types as OracleTypes.OTHER 2022-03-18 11:05:36 +01:00
Jiri Pechanec
341ec1260e DBZ-4878 Add partition to Oracle xstream 2022-03-18 07:55:43 +01:00
kate
d2899a276a DBZ-4878 Add partition to Oracle logminer 2022-03-18 07:55:43 +01:00
Chris Cranford
c8ecea8df9 DBZ-4842 Relax supplemental log check when table does not exist 2022-03-17 09:52:47 +01:00
Gunnar Morling
d20a6f00e7 DBZ-4801 Using JdbcConfiguration in JdbcConnection 2022-03-07 17:53:45 +01:00
Chris Cranford
fd38f771fa DBZ-4819 Correct timing concerns to get FlushStrategyIT stable 2022-03-04 11:28:27 -05:00
Chris Cranford
252549c674 DBZ-4287 Remove redundant getCurrentRedoLogSequences usage 2022-03-03 07:48:12 -05:00
Willie Zhu
66007df053 DBZ-4287 set the currentRedoLogSequences at the time when we update the redo log file list 2022-03-03 07:48:12 -05:00