Chris Cranford
79ca0becae
DBZ-5635 Reduce undo trx log verbosity when LOB enabled
2022-09-22 13:18:06 +02:00
Ismail Simsek
b966f4b65b
DBZ-5591 Snapshot process setting source.ts_ms
2022-09-20 17:12:54 -04:00
Chris Cranford
13bfcc8270
DBZ-5581 Extend test cases with NULL assignment
2022-09-20 08:35:59 +02:00
Chris Cranford
5484948127
DBZ-5581 Apply unavailable value placeholder on when parsing SELECT_LOB_LOCATOR event types
2022-09-20 08:35:59 +02:00
harveyyue
288c93434f
DBZ-5585 Remove classloader parameter from JdbcConnection
2022-09-19 17:15:29 +02:00
harveyyue
20d56734d4
DBZ-5585 Modify the Instantiator to not require classloader
2022-09-19 17:15:29 +02:00
Debezium Builder
218d414552
[maven-release-plugin] prepare for next development iteration
2022-09-16 06:36:40 +00:00
Debezium Builder
567f57c47b
[maven-release-plugin] prepare release v2.0.0.Beta2
2022-09-16 06:36:39 +00:00
Jiri Pechanec
3dc16b722b
DBZ-4842 LogMiner specific test
2022-09-16 05:27:43 +02:00
harveyyue
f21ebc98fe
DBZ-5609 Oracle connector couldn't parse type_name mdsys.sdo_geometry
2022-09-14 15:57:41 -04:00
Vojtech Juranek
acc6be6df3
DBZ-5043 Move TOPIC_PREFIX into CommonConnectorConfig
...
`TOPIC_PREFIX` is now mandatory to all connectors therefore it make
sense to have it in common config. Beside that, it also makes it more
easy to use it in Debezium UI without any workarounds - if the field is
not member of the given connector config, the field has to be
explicitely added into known fields otherwise is invisible for UI.
2022-09-13 09:25:54 +02:00
Anisha Mohanty
9fd204b383
DBZ-5044 Introduce respective SchemaFactory for all connectors
2022-09-08 10:35:20 +02:00
Vojtech Juranek
402df78381
DBZ-5043 Rename schema.history to schema.history.internal
2022-09-07 12:57:31 +02:00
Vojtech Juranek
b797201263
DBZ-5043 Rename DB_HISTORY_PATH to SCHEMA_HISTORY_PATH
2022-09-07 12:57:31 +02:00
Vojtech Juranek
b6c31f3da5
DBZ-5043 Replace 'database history' with 'database schema history'
...
Reflect renaming database history to schema history in comments,
configuration descriptions and log and error messages.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0494a16a79
DBZ-5043 Rename DatabaseHistory to SchemaHistory
...
Rename also all related classes and varibales.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0823cedf25
DBZ-5043 Replace database.server.name with topic.prefix
2022-09-07 12:57:31 +02:00
Vojtech Juranek
161a7229b6
DBZ-5043 Use topic prefix as the logical name
...
With this change the user cannot direcly set connector logical name and
thus in the future it can be remove without breaking user config.
If the turn out that the logical name is useful and user should be able
to configure it, dedicated config option can be added.
Originally it was prposed in the Jira to replace it with connector name,
but it turned out that logical name defaults to `database.server.name`
and is heavily used in the tests and JMX, so it would require another
big refactoring. Thus, use topic prefix for now. Once we know further
direction (remove logical name or add new option), do this refactoring.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
25aa6c4acc
DBZ-5043 Change namespace to database.history
to schema.history
2022-09-07 12:57:31 +02:00
Vadzim Ramanenka
917de37a01
DBZ-5557: Supply partition when comiting offsets with source database
...
Keep track of partitions offsets being committed belong to in
`BaseSourceTask`. Supply the partition along with the offset in calls to
`commitOffset`.
2022-09-05 14:04:23 +02:00
ggaborg
244de2bd96
DBZ-3293 Add signal table automatically to include list
2022-09-01 20:16:47 +02:00
Chris Cranford
b5754a19ec
DBZ-5439 Remove rsid and ssn from CommitScn format
2022-09-01 09:32:17 +02:00
Chris Cranford
c0f92e24ea
DBZ-5439 Updated tests, use TreeSet trx id deterministic order
2022-09-01 09:32:17 +02:00
thangdc94
57210095d2
DBZ-5439 Fix testcase
2022-09-01 09:32:17 +02:00
thangdc94
c755720637
DBZ-5439 Backward compatible with old commit scn
2022-09-01 09:32:17 +02:00
thangdc94
cfa8b780ed
DBZ-5439 Add transaction id associate with commit scn
2022-09-01 09:32:17 +02:00
harveyyue
7ccd40129c
DBZ-5569 Oracle connector's max.queue.size.in.bytes is invalid
2022-08-31 09:39:29 -04:00
Chris Cranford
92ca57d731
DBZ-5229 Restore Kafka-based database history options in SchemaGenerator
2022-08-30 10:32:12 -04:00
pkgonan
b031770300
DBZ-5544 Support BASE64_URL_SAFE in BinaryHandlingMode
2022-08-24 09:50:23 +02:00
Chris Cranford
14c9f8855b
DBZ-5327 Fix Oracle/Db2 test failures
2022-08-19 11:33:01 -04:00
Chris Cranford
176529e83b
DBZ-5521 Fix concatenation operator handling in LogMiner DML parser
2022-08-19 11:19:45 +02:00
Chris Cranford
37b97970a6
DBZ-5497 Skip truncate operations by default
...
* Promote truncate skip default from PostgreSQL to core library
* Adjust behavior for Oracle with truncates skipped by default
2022-08-19 11:05:29 +02:00
Vivek Wassan
d89e71ebeb
DBZ-5327 Smart Backfills
2022-08-17 10:05:55 +02:00
Chris Cranford
41a9324044
DBZ-5441 Gracefully skip Oracle non-relational tables
2022-07-29 12:56:27 -04:00
harveyyue
5f430f1a8e
DBZ-5457 Deprecate legacy topic selector for all connectors
2022-07-28 12:58:59 -04:00
Debezium Builder
1ab51ae8d0
[maven-release-plugin] prepare for next development iteration
2022-07-26 18:13:10 +00:00
Debezium Builder
d247a48b79
[maven-release-plugin] prepare release v2.0.0.Beta1
2022-07-26 18:13:09 +00:00
Chris Cranford
412498db41
DBZ-5295 Avoid NPE With Xstream emitter
2022-07-26 11:05:17 -04:00
Chris Cranford
2c2cb9213a
DBZ-5248 Avoid NumberFormatException in OracleConnectorIT tests
2022-07-20 13:21:59 -04:00
Chris Cranford
f8d097ebc3
DBZ-5419 Fix OracleSchemaMigrationIT (non-CDB) test failure
2022-07-19 09:23:16 -04:00
Anisha Mohanty
04941330c3
DBZ-5390 Reorder relational_property definition to give priority to out_of_line_constraint
2022-07-14 08:55:21 +02:00
Chris Cranford
f62fab4364
DBZ-5404 Use fail from fest assertions
2022-07-14 08:40:42 +02:00
harveyyue
6ea7abc8fc
DBZ-4180 Pluggable topic selector
2022-07-13 14:45:25 +02:00
Jiri Pechanec
d8baac7c46
DBZ-5229 Reorganize code for inheritance use
2022-07-12 17:31:19 +02:00
Hossein Torabi
1ea02be9c2
DBZ-5229 define centralized and modular aproach for debezium storage
2022-07-12 17:31:19 +02:00
Chris Cranford
35e3eae52f
DBZ-5295 Add delete row test case
2022-07-12 11:59:34 +02:00
Chris Cranford
843f133d4d
DBZ-5295 Use connection quote method, rework unavailable value check
2022-07-12 11:59:34 +02:00
Chris Cranford
5f443dd9e1
DBZ-5295 Fix format errors
2022-07-12 11:59:34 +02:00
Chris Cranford
6735904075
DBZ-5295 Cache LOB column status in schema model
2022-07-12 11:59:34 +02:00
Chris Cranford
32947ae512
DBZ-5295 Remove unnecessary test output
2022-07-12 11:59:34 +02:00
Chris Cranford
d2a76a7ffc
DBZ-5295 Reselect LOB columns on primary key update
2022-07-12 11:59:34 +02:00
Chris Cranford
deaa34d71e
DBZ-5356 Avoid NPE during index evaluation
2022-07-12 11:01:10 +02:00
Chris Cranford
ddbd682d8d
DBZ-5381 Add a pretty loggable format for commit_scn
2022-07-12 06:55:13 +02:00
Chris Cranford
f7696ffac7
DBZ-5358 Rename to "user_name", position at end of struct
2022-07-11 10:10:21 -04:00
Nathan Smit
28ce4f0aa2
DBZ-5358 added name of user who executed transaction to source info block
2022-07-11 10:10:21 -04:00
Chris Cranford
28c26741d6
DBZ-5351 Add warning for LogMiner rows with unsupported op type
2022-07-06 17:23:45 -04:00
Chris Cranford
9cca260197
DBZ-5373 Propagate Oracle Xstream commit timestamp to transaction metadata
2022-07-06 16:25:18 -04:00
Debezium Builder
6fd1441af3
[maven-release-plugin] prepare for next development iteration
2022-07-01 09:21:49 +00:00
Debezium Builder
99402af634
[maven-release-plugin] prepare release v2.0.0.Alpha3
2022-07-01 09:21:48 +00:00
Chris Cranford
30d95358b8
DBZ-5245 Track commit scn per redo thread
2022-07-01 06:39:19 +02:00
Chris Cranford
406e060df4
DBZ-5234 Rename oracle-ide
to oracle-all
2022-07-01 06:08:31 +02:00
Chris Cranford
c651996861
DBZ-5250 Update Oracle README.md, clarifying 11g requirements
2022-06-30 17:09:45 -04:00
Chris Cranford
2be190bcdd
DBZ-5281 Use separate connection for snapshot-based LogMiner queries
2022-06-30 10:37:32 +02:00
Chris Cranford
65fa2c73a5
DBZ-5281 Fix test compatibility with Oracle non-CDB
2022-06-30 10:37:32 +02:00
Mickael Maison
5ff71aa992
DBZ-5335 Fix formatting issues in the connectors configdefs
2022-06-30 09:35:54 +02:00
Chris Cranford
cf1e711299
DBZ-5266 Skip tests when using Oracle Xstream
2022-06-29 19:59:35 -04:00
harveyyue
20dc353262
DBZ-5170 Mysql Commit Timestamp
2022-06-27 15:50:52 +02:00
Chris Cranford
fd296557b5
DBZ-5266 Correctly advance commit_scn
with LOB support
2022-06-27 15:24:11 +02:00
Chris Cranford
811ba8a4af
DBZ-4902 Fix compatibility with Oracle connector
2022-06-27 08:45:37 -04:00
Chris Cranford
cfc63776fa
DBZ-5300 Exclude ADT indices as relational primary keys
2022-06-27 10:38:56 +02:00
Chris Cranford
5d4d659b37
DBZ-5285 Fix store.only.captured.tables.ddl to work during streaming
2022-06-27 09:02:22 +02:00
Chris Cranford
ced5cc6827
DBZ-5234 Add oracle-ide
profile, simplifies IDE integrations
...
Some IDE environments do not support maven-compiler-plugin include/exclude
filter configurations well and this profile is meant to bridge that gap by
enabling all sources and dependencies for IDE compiles.
2022-06-27 06:04:58 +02:00
Chris Cranford
f4b6cfafe6
DBZ-5234 Update documentation references
2022-06-24 12:19:23 +02:00
Chris Cranford
124bc6a515
DBZ-5234 Align GitHub actions with Oracle profile changes
...
* Removed oracle-ci profile entirely, no longer required
The default build with no profiles explicitly uses the previous oracle-ci behavior.
* Removed the use of the oracle profile, the Oracle connector enabled by default
* Removed the xstream-dependency usage, Xstream dependencies aren't enabled by default.
2022-06-24 12:19:23 +02:00
Chris Cranford
35f4c232a3
DBZ-5234 Rework Oracle build profiles
...
* Oracle will always be built with LogMiner by default
* Oracle tests are always compiled, integration tests skipped by default
* Integration tests require oracle-tests profile to be executed
* Xstream requires oracle-xstream profile
* Instant client dependencies are excluded by default
* The assembly profile enables Xstream bits automatically
2022-06-24 12:19:23 +02:00
Chris Cranford
a1d35b402e
DBZ-5303 Rework SkipOnDatabaseOption, allow for enabled/disabled toggles
2022-06-23 18:58:05 -04:00
Chris Cranford
cab3c664a1
DBZ-5301 FIx LogMinerHelperIT to work on Oracle RAC with multiple nodes
2022-06-23 18:54:57 -04:00
Jiri Pechanec
44107961e5
DBZ-4367 Align test with new schema changes behaviour
2022-06-17 08:53:47 +02:00
Jiri Pechanec
86abed2f9b
DBZ-4834 Fix test
2022-06-16 14:52:47 +02:00
Chris Cranford
263ba5aad6
DBZ-5256 Add internal.log.mining.transaction.snapshot.boundary.mode
option
...
This internal option is meant to replace the old `log.mining.query.logs.for.snapshot.offset`.
This now enum-based setting provides much more flexibility by being able to completely
disable the in-progress transaction check (now the default), only grab transactions that are
in-progress from V$TRANSACTION, or finally be able to grab all in-progress transactions that
are both from V$TRANSACTION and by scanning the logs.
2022-06-16 12:44:45 +02:00
Chris Cranford
3fb5bf1339
DBZ-4451 Introduce SchemaChangeEvent factory methods
2022-06-16 09:19:41 +02:00
Chris Cranford
b6549670c8
DBZ-4451 Allow test to be extended by connectors
2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081
DBZ-4451 Store previous table id in alter schema change events
...
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5
DBZ-4834 Fix incremental snapshot of table added to include list
2022-06-16 08:39:16 +02:00
Chris Cranford
44159c95be
DBZ-5225 Add EVENT_SCN_KEY
key
2022-06-16 08:27:32 +02:00
Bob Roldan
18688077bf
DBZ-5225 Align source.scn
with the event's scn
2022-06-16 08:27:32 +02:00
Chris Cranford
4d30e6be47
DBZ-5257 Source info table order deterministic for rename schema changes
2022-06-16 08:19:38 +02:00
Chris Cranford
14a3b46ee5
DBZ-5258 Fix test assertions using static database names
2022-06-15 18:05:56 -04:00
Chris Cranford
46bef9c3ae
DBZ-4890 BigDecimal scale match Field when column uses negative scale
2022-06-15 12:27:56 +02:00
Chris Cranford
e142203a86
DBZ-5226 Improve test wait-time performance
2022-06-09 15:15:22 -04:00
Debezium Builder
6fff3a2876
[maven-release-plugin] prepare for next development iteration
2022-06-09 11:49:26 +00:00
Debezium Builder
b2e2e351df
[maven-release-plugin] prepare release v2.0.0.Alpha2
2022-06-09 11:49:26 +00:00
Vojtech Juranek
bbb625f29e
DBZ-5045 Use correct buffer type value after deprecated infinispan
was removed
2022-06-08 14:32:07 +02:00
Vojtech Juranek
ec9625bd70
DBZ-5045 Remove deprecated Oracle DB ORACLE_VERSION option
2022-06-08 09:11:00 +02:00
Vojtech Juranek
ff5ddd7959
DBZ-5045 Remove deprecated FIELD_PAYLOAD_ID
...
`table.field.event.payload.id` is replaced by `table.field.event.key`.
Also fix default value of `table.field.event.key` which was missing.
2022-06-08 09:11:00 +02:00
Vojtech Juranek
01d8e4fe27
DBZ-5045 Remove deprecated Oracle LOG_MINING_BUFFER_LOCATION option
2022-06-08 09:11:00 +02:00
Vojtech Juranek
091e2f3162
DBZ-5045 Remove deprecated Oracle INFINISPAN option
2022-06-08 09:11:00 +02:00
Vojtech Juranek
f84893a8b4
DBZ-5045 Remove deprecated SCHEMA_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
261188482f
DBZ-5045 Remove deprecated SCHEMA_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
6abbcee6a3
DBZ-5045 Remove deprecated TABLE_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
db08e894fc
DBZ-5045 Remove deprecated TABLE_WHITELIST
2022-06-08 09:11:00 +02:00
Jiri Pechanec
7d16c879f3
DBZ-5202 Extract constants; Use Java 8 API
2022-06-07 09:04:03 +02:00
Chris Cranford
310e788c22
DBZ-5202 Add retry/delay to offset transaction id resolution
2022-06-07 09:04:03 +02:00
Chris Cranford
ec940e8220
DBZ-5085 Enforce read-consistency determining snapshot offsets
2022-06-02 08:29:13 +02:00
Jiri Pechanec
b61d99d0c6
DBZ-5136 Fix Oracle tests
2022-06-02 07:41:00 +02:00
Chris Cranford
9d71296b5c
DBZ-4251 Support stopping incremental snapshots
2022-06-02 07:30:07 +02:00
Chris Cranford
f3671afaf7
DBZ-5139 Apply threshold to all processors, added tests
2022-06-01 17:52:44 -04:00
Ethan Zou
1ac6abf63e
DBZ-5139 Abandon transaction who has too many events according to new configuration 'log.mining.buffer.memory.transaction.events.threshold'
2022-06-01 17:52:44 -04:00
Jiri Pechanec
88b7ae2050
DBZ-5136 Move sourceInfo to common class
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
c64b8af31a
DBZ-5136 reintroduce the CommonOffsetContext class and refactor common functionality into it
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
2d72dae649
DBZ-5136 further refining of the algorithm for determining position in snapshot and IT test fixes
2022-06-01 16:56:29 +02:00
Jiri Pechanec
08e8c3d795
DBZ-5136 Move common class to common package
2022-06-01 16:56:29 +02:00
Andrew Walker
d23735c658
DBZ-5136: Fix missing reference
2022-06-01 16:56:29 +02:00
Andrew Walker
d0759461a4
DBZ-5136: Convert RelationalOffsetContext to CommonOffsetContext
2022-06-01 16:56:29 +02:00
Andrew Walker
b6222d42bf
DBZ-5136: Add first flag and refactor OffsetContext
2022-06-01 16:56:29 +02:00
Andrew Walker
593c2769e6
DBZ-5136: Add last in table flag
2022-06-01 16:56:29 +02:00
Chris Cranford
15e9231c30
DBZ-5179 Use LRUCacheMap
to avoid high memory usage
2022-06-01 15:53:55 +02:00
Chris Cranford
a900ababcd
DBZ-5042 Use PDB name when set in Oracle configuration
2022-05-27 14:20:30 -04:00
Vojtech Juranek
fa07f67cdb
DBZ-5042 Add DB name logging support into Oracle connector
2022-05-27 14:20:30 -04:00
Chris Cranford
1ae7bec46e
DBZ-5146 Remove log.mining.view.fetch.size
, use query.fetch.size
instead.
2022-05-26 19:26:14 -04:00
Chris Cranford
60b1b3cc1d
DBZ-5147 Fix test failures
2022-05-26 18:37:24 -04:00
Chris Cranford
2978898641
DBZ-5147 Adjust log levels, apply test rules
2022-05-26 18:37:24 -04:00
Chris Cranford
7ab0cea8f1
DBZ-5147 Add event.processing.failure.handling.mode
for Oracle
2022-05-26 18:37:24 -04:00
Chris Cranford
7d8c9486d6
DBZ-5119 Move Heartbeat creation to ConnectorConfig & descendants
2022-05-25 09:59:59 +02:00
Chris Cranford
7ff7826743
DBZ-5119 Pull heartbeat.action.query
support into debezium-core
2022-05-24 12:54:40 +02:00
Chris Cranford
15eb7807c4
DBZ-5119 Add heartbeat.action.query
support for Oracle
2022-05-24 12:54:40 +02:00
Chris Cranford
2b78fca8ce
DBZ-5148 Retry starting Oracle LogMiner after ORA-01291
2022-05-23 10:30:18 +02:00
Chris Cranford
7577500696
DBZ-5140 Do not log transaction warning when lob-enabled is true
2022-05-18 14:11:36 -04:00
Chris Cranford
9a81339b54
DBZ-4782 Rework if-branches
2022-05-13 12:51:38 +02:00
Chris Cranford
e1579f8738
DBZ-4782 Flush SCN values upon schema changes
2022-05-13 12:51:38 +02:00
Chris Cranford
d0c6b59527
DBZ-4973 Cleanup logging calls
2022-05-10 11:33:33 -04:00
Chris Cranford
db9a104ea4
DBZ-4973 Remove unused code
2022-05-10 11:33:33 -04:00
Chris Cranford
035d6b4c14
DBZ-5090 Improve test use case
2022-05-10 09:45:34 -04:00
Chris Cranford
0d595025fd
DBZ-5090 Undo events across multiple mining sessions
2022-05-10 09:45:34 -04:00
Debezium Builder
901825810a
[maven-release-plugin] prepare for next development iteration
2022-04-28 13:51:13 +00:00
Debezium Builder
05dcb45647
[maven-release-plugin] prepare release v2.0.0.Alpha1
2022-04-28 13:51:12 +00:00
Chris Cranford
43ae067ef8
DBZ-5006 Add suggested test coverage
2022-04-27 08:52:29 -04:00
Chris Cranford
7e845f83cb
DBZ-3318 Scope tests to Oracle LogMiner adapter
2022-04-27 08:52:29 -04:00
Chris Cranford
cea4b51a3b
DBZ-4963 Scope tests to Oracle LogMiner adapter
2022-04-27 08:52:29 -04:00
Chris Cranford
09e58c5a5c
DBZ-5006 Allow "/" characters in Oracle table names
2022-04-27 08:52:29 -04:00
Jan Doms
f698a59481
DBZ-5005 adjust LogMiner batch size based on comparison with currently used batch size
2022-04-26 14:08:00 -04:00
Chris Cranford
9e08093900
DBZ-5029 Scope several OracleConnectorIT tests to LogMiner only
...
* shouldCaptureChangeForTableWithUnsupportedColumnType
* shouldContinueToUpdateOffsetsEvenWhenTableIsNotChanged
2022-04-21 16:10:39 +02:00
Chris Cranford
cee77e5b96
DBZ-5028 Scope OracleOffsetContextTest to the LogMiner adapter
2022-04-21 15:51:02 +02:00
Chris Cranford
5f9b2fa822
DBZ-4963 Use Optional & Stopwatch, disable lengthy test by default
2022-04-21 08:05:59 +02:00
Chris Cranford
c804509673
DBZ-4963 Introduce log.mining.session.max.ms
config option for Oracle
2022-04-21 08:05:59 +02:00
Chris Cranford
f24be1f533
DBZ-4994 Fix LOB locator event parsing with escaped single quotes
2022-04-19 09:07:36 +02:00
Chris Cranford
f667402fe0
DBZ-4987 Propagate up all exceptions from LCR handler
2022-04-19 07:23:39 +02:00
Chris Cranford
89fb497602
DBZ-4976 Add unit test
2022-04-14 11:58:50 +02:00
Chris Cranford
0ffd2e2ec3
DBZ-4976 Rework null checks & add nullability comments
2022-04-14 11:58:50 +02:00
Anisha Mohanty
736a3b149b
DBZ-4976 Handle NPEs for column definitions of Oracle DDL statements
2022-04-14 11:58:50 +02:00