Commit Graph

1015 Commits

Author SHA1 Message Date
Debezium Builder
7bd1551c99 [maven-release-plugin] prepare release v2.0.0.Final 2022-10-14 11:26:32 +00:00
thangdc94
f6be36612f DBZ-5276 Remove redundant import 2022-10-14 08:22:56 +02:00
thangdc94
c53af41deb DBZ-5276 Fix wrong log switching condition 2022-10-14 08:22:56 +02:00
thangdc94
4a9539266d DBZ-5276 Fix Group by condition for thread 2022-10-14 08:22:56 +02:00
thangdc94
d233e969e7 DBZ-5276 Prevent add duplicate log file 2022-10-14 08:22:56 +02:00
Debezium Builder
884254aba1 [maven-release-plugin] prepare for next development iteration 2022-10-07 09:27:52 +00:00
Debezium Builder
a0f3263fa6 [maven-release-plugin] prepare release v2.0.0.CR1 2022-10-07 09:27:51 +00:00
Chris Cranford
8ec0846deb
DBZ-5541 Fix Oracle test compatibility failure 2022-10-05 13:25:08 -04:00
harveyyue
f6b72f060c DBZ-5671 Remove logic name parameter from sub connector config 2022-10-03 11:40:17 -04:00
Vojtech Juranek
97532042b3 DBZ-5657 Allow to use Docker maven plugin external properties
Allow to override Docker maven plugin properties [1] from command line
to be able to change various Docker parameters more easily when starting
the container.

[1] https://dmp.fabric8.io/#combining-property-config
2022-09-29 05:41:09 +02:00
Chris Cranford
4e3b4ebae7 DBZ-5645 Skip rollback trx cache when LOB is disabled 2022-09-22 18:09:33 -04:00
Chris Cranford
79ca0becae DBZ-5635 Reduce undo trx log verbosity when LOB enabled 2022-09-22 13:18:06 +02:00
Ismail Simsek
b966f4b65b DBZ-5591 Snapshot process setting source.ts_ms 2022-09-20 17:12:54 -04:00
Chris Cranford
13bfcc8270 DBZ-5581 Extend test cases with NULL assignment 2022-09-20 08:35:59 +02:00
Chris Cranford
5484948127 DBZ-5581 Apply unavailable value placeholder on when parsing SELECT_LOB_LOCATOR event types 2022-09-20 08:35:59 +02:00
harveyyue
288c93434f DBZ-5585 Remove classloader parameter from JdbcConnection 2022-09-19 17:15:29 +02:00
harveyyue
20d56734d4 DBZ-5585 Modify the Instantiator to not require classloader 2022-09-19 17:15:29 +02:00
Debezium Builder
218d414552 [maven-release-plugin] prepare for next development iteration 2022-09-16 06:36:40 +00:00
Debezium Builder
567f57c47b [maven-release-plugin] prepare release v2.0.0.Beta2 2022-09-16 06:36:39 +00:00
Jiri Pechanec
3dc16b722b DBZ-4842 LogMiner specific test 2022-09-16 05:27:43 +02:00
harveyyue
f21ebc98fe DBZ-5609 Oracle connector couldn't parse type_name mdsys.sdo_geometry 2022-09-14 15:57:41 -04:00
Vojtech Juranek
acc6be6df3 DBZ-5043 Move TOPIC_PREFIX into CommonConnectorConfig
`TOPIC_PREFIX` is now mandatory to all connectors therefore it make
sense to have it in common config. Beside that, it also makes it more
easy to use it in Debezium UI without any workarounds - if the field is
not member of the given connector config, the field has to be
explicitely added into known fields otherwise is invisible for UI.
2022-09-13 09:25:54 +02:00
Anisha Mohanty
9fd204b383 DBZ-5044 Introduce respective SchemaFactory for all connectors 2022-09-08 10:35:20 +02:00
Vojtech Juranek
402df78381 DBZ-5043 Rename schema.history to schema.history.internal 2022-09-07 12:57:31 +02:00
Vojtech Juranek
b797201263 DBZ-5043 Rename DB_HISTORY_PATH to SCHEMA_HISTORY_PATH 2022-09-07 12:57:31 +02:00
Vojtech Juranek
b6c31f3da5 DBZ-5043 Replace 'database history' with 'database schema history'
Reflect renaming database history to schema history in comments,
configuration descriptions and log and error messages.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0494a16a79 DBZ-5043 Rename DatabaseHistory to SchemaHistory
Rename also all related classes and varibales.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
0823cedf25 DBZ-5043 Replace database.server.name with topic.prefix 2022-09-07 12:57:31 +02:00
Vojtech Juranek
161a7229b6 DBZ-5043 Use topic prefix as the logical name
With this change the user cannot direcly set connector logical name and
thus in the future it can be remove without breaking user config.
If the turn out that the logical name is useful and user should be able
to configure it, dedicated config option can be added.

Originally it was prposed in the Jira to replace it with connector name,
but it turned out that logical name defaults to `database.server.name`
and is heavily used in the tests and JMX, so it would require another
big refactoring. Thus, use topic prefix for now. Once we know further
direction (remove logical name or add new option), do this refactoring.
2022-09-07 12:57:31 +02:00
Vojtech Juranek
25aa6c4acc DBZ-5043 Change namespace to database.history to schema.history 2022-09-07 12:57:31 +02:00
Vadzim Ramanenka
917de37a01 DBZ-5557: Supply partition when comiting offsets with source database
Keep track of partitions offsets being committed belong to in
`BaseSourceTask`. Supply the partition along with the offset in calls to
`commitOffset`.
2022-09-05 14:04:23 +02:00
ggaborg
244de2bd96 DBZ-3293 Add signal table automatically to include list 2022-09-01 20:16:47 +02:00
Chris Cranford
b5754a19ec DBZ-5439 Remove rsid and ssn from CommitScn format 2022-09-01 09:32:17 +02:00
Chris Cranford
c0f92e24ea DBZ-5439 Updated tests, use TreeSet trx id deterministic order 2022-09-01 09:32:17 +02:00
thangdc94
57210095d2 DBZ-5439 Fix testcase 2022-09-01 09:32:17 +02:00
thangdc94
c755720637 DBZ-5439 Backward compatible with old commit scn 2022-09-01 09:32:17 +02:00
thangdc94
cfa8b780ed DBZ-5439 Add transaction id associate with commit scn 2022-09-01 09:32:17 +02:00
harveyyue
7ccd40129c DBZ-5569 Oracle connector's max.queue.size.in.bytes is invalid 2022-08-31 09:39:29 -04:00
Chris Cranford
92ca57d731 DBZ-5229 Restore Kafka-based database history options in SchemaGenerator 2022-08-30 10:32:12 -04:00
pkgonan
b031770300 DBZ-5544 Support BASE64_URL_SAFE in BinaryHandlingMode 2022-08-24 09:50:23 +02:00
Chris Cranford
14c9f8855b DBZ-5327 Fix Oracle/Db2 test failures 2022-08-19 11:33:01 -04:00
Chris Cranford
176529e83b DBZ-5521 Fix concatenation operator handling in LogMiner DML parser 2022-08-19 11:19:45 +02:00
Chris Cranford
37b97970a6 DBZ-5497 Skip truncate operations by default
* Promote truncate skip default from PostgreSQL to core library
* Adjust behavior for Oracle with truncates skipped by default
2022-08-19 11:05:29 +02:00
Vivek Wassan
d89e71ebeb DBZ-5327 Smart Backfills 2022-08-17 10:05:55 +02:00
Chris Cranford
41a9324044 DBZ-5441 Gracefully skip Oracle non-relational tables 2022-07-29 12:56:27 -04:00
harveyyue
5f430f1a8e DBZ-5457 Deprecate legacy topic selector for all connectors 2022-07-28 12:58:59 -04:00
Debezium Builder
1ab51ae8d0 [maven-release-plugin] prepare for next development iteration 2022-07-26 18:13:10 +00:00
Debezium Builder
d247a48b79 [maven-release-plugin] prepare release v2.0.0.Beta1 2022-07-26 18:13:09 +00:00
Chris Cranford
412498db41
DBZ-5295 Avoid NPE With Xstream emitter 2022-07-26 11:05:17 -04:00
Chris Cranford
2c2cb9213a DBZ-5248 Avoid NumberFormatException in OracleConnectorIT tests 2022-07-20 13:21:59 -04:00
Chris Cranford
f8d097ebc3
DBZ-5419 Fix OracleSchemaMigrationIT (non-CDB) test failure 2022-07-19 09:23:16 -04:00
Anisha Mohanty
04941330c3 DBZ-5390 Reorder relational_property definition to give priority to out_of_line_constraint 2022-07-14 08:55:21 +02:00
Chris Cranford
f62fab4364 DBZ-5404 Use fail from fest assertions 2022-07-14 08:40:42 +02:00
harveyyue
6ea7abc8fc DBZ-4180 Pluggable topic selector 2022-07-13 14:45:25 +02:00
Jiri Pechanec
d8baac7c46 DBZ-5229 Reorganize code for inheritance use 2022-07-12 17:31:19 +02:00
Hossein Torabi
1ea02be9c2 DBZ-5229 define centralized and modular aproach for debezium storage 2022-07-12 17:31:19 +02:00
Chris Cranford
35e3eae52f DBZ-5295 Add delete row test case 2022-07-12 11:59:34 +02:00
Chris Cranford
843f133d4d DBZ-5295 Use connection quote method, rework unavailable value check 2022-07-12 11:59:34 +02:00
Chris Cranford
5f443dd9e1 DBZ-5295 Fix format errors 2022-07-12 11:59:34 +02:00
Chris Cranford
6735904075 DBZ-5295 Cache LOB column status in schema model 2022-07-12 11:59:34 +02:00
Chris Cranford
32947ae512 DBZ-5295 Remove unnecessary test output 2022-07-12 11:59:34 +02:00
Chris Cranford
d2a76a7ffc DBZ-5295 Reselect LOB columns on primary key update 2022-07-12 11:59:34 +02:00
Chris Cranford
deaa34d71e DBZ-5356 Avoid NPE during index evaluation 2022-07-12 11:01:10 +02:00
Chris Cranford
ddbd682d8d DBZ-5381 Add a pretty loggable format for commit_scn 2022-07-12 06:55:13 +02:00
Chris Cranford
f7696ffac7 DBZ-5358 Rename to "user_name", position at end of struct 2022-07-11 10:10:21 -04:00
Nathan Smit
28ce4f0aa2 DBZ-5358 added name of user who executed transaction to source info block 2022-07-11 10:10:21 -04:00
Chris Cranford
28c26741d6 DBZ-5351 Add warning for LogMiner rows with unsupported op type 2022-07-06 17:23:45 -04:00
Chris Cranford
9cca260197 DBZ-5373 Propagate Oracle Xstream commit timestamp to transaction metadata 2022-07-06 16:25:18 -04:00
Debezium Builder
6fd1441af3 [maven-release-plugin] prepare for next development iteration 2022-07-01 09:21:49 +00:00
Debezium Builder
99402af634 [maven-release-plugin] prepare release v2.0.0.Alpha3 2022-07-01 09:21:48 +00:00
Chris Cranford
30d95358b8 DBZ-5245 Track commit scn per redo thread 2022-07-01 06:39:19 +02:00
Chris Cranford
406e060df4 DBZ-5234 Rename oracle-ide to oracle-all 2022-07-01 06:08:31 +02:00
Chris Cranford
c651996861 DBZ-5250 Update Oracle README.md, clarifying 11g requirements 2022-06-30 17:09:45 -04:00
Chris Cranford
2be190bcdd DBZ-5281 Use separate connection for snapshot-based LogMiner queries 2022-06-30 10:37:32 +02:00
Chris Cranford
65fa2c73a5 DBZ-5281 Fix test compatibility with Oracle non-CDB 2022-06-30 10:37:32 +02:00
Mickael Maison
5ff71aa992 DBZ-5335 Fix formatting issues in the connectors configdefs 2022-06-30 09:35:54 +02:00
Chris Cranford
cf1e711299
DBZ-5266 Skip tests when using Oracle Xstream 2022-06-29 19:59:35 -04:00
harveyyue
20dc353262 DBZ-5170 Mysql Commit Timestamp 2022-06-27 15:50:52 +02:00
Chris Cranford
fd296557b5 DBZ-5266 Correctly advance commit_scn with LOB support 2022-06-27 15:24:11 +02:00
Chris Cranford
811ba8a4af
DBZ-4902 Fix compatibility with Oracle connector 2022-06-27 08:45:37 -04:00
Chris Cranford
cfc63776fa DBZ-5300 Exclude ADT indices as relational primary keys 2022-06-27 10:38:56 +02:00
Chris Cranford
5d4d659b37 DBZ-5285 Fix store.only.captured.tables.ddl to work during streaming 2022-06-27 09:02:22 +02:00
Chris Cranford
ced5cc6827 DBZ-5234 Add oracle-ide profile, simplifies IDE integrations
Some IDE environments do not support maven-compiler-plugin include/exclude
filter configurations well and this profile is meant to bridge that gap by
enabling all sources and dependencies for IDE compiles.
2022-06-27 06:04:58 +02:00
Chris Cranford
f4b6cfafe6 DBZ-5234 Update documentation references 2022-06-24 12:19:23 +02:00
Chris Cranford
124bc6a515 DBZ-5234 Align GitHub actions with Oracle profile changes
* Removed oracle-ci profile entirely, no longer required
  The default build with no profiles explicitly uses the previous oracle-ci behavior.
* Removed the use of the oracle profile, the Oracle connector enabled by default
* Removed the xstream-dependency usage, Xstream dependencies aren't enabled by default.
2022-06-24 12:19:23 +02:00
Chris Cranford
35f4c232a3 DBZ-5234 Rework Oracle build profiles
* Oracle will always be built with LogMiner by default
* Oracle tests are always compiled, integration tests skipped by default
* Integration tests require oracle-tests profile to be executed
* Xstream requires oracle-xstream profile
* Instant client dependencies are excluded by default
* The assembly profile enables Xstream bits automatically
2022-06-24 12:19:23 +02:00
Chris Cranford
a1d35b402e DBZ-5303 Rework SkipOnDatabaseOption, allow for enabled/disabled toggles 2022-06-23 18:58:05 -04:00
Chris Cranford
cab3c664a1 DBZ-5301 FIx LogMinerHelperIT to work on Oracle RAC with multiple nodes 2022-06-23 18:54:57 -04:00
Jiri Pechanec
44107961e5 DBZ-4367 Align test with new schema changes behaviour 2022-06-17 08:53:47 +02:00
Jiri Pechanec
86abed2f9b DBZ-4834 Fix test 2022-06-16 14:52:47 +02:00
Chris Cranford
263ba5aad6 DBZ-5256 Add internal.log.mining.transaction.snapshot.boundary.mode option
This internal option is meant to replace the old `log.mining.query.logs.for.snapshot.offset`.
This now enum-based setting provides much more flexibility by being able to completely
disable the in-progress transaction check (now the default), only grab transactions that are
in-progress from V$TRANSACTION, or finally be able to grab all in-progress transactions that
are both from V$TRANSACTION and by scanning the logs.
2022-06-16 12:44:45 +02:00
Chris Cranford
3fb5bf1339 DBZ-4451 Introduce SchemaChangeEvent factory methods 2022-06-16 09:19:41 +02:00
Chris Cranford
b6549670c8 DBZ-4451 Allow test to be extended by connectors 2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081 DBZ-4451 Store previous table id in alter schema change events
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5 DBZ-4834 Fix incremental snapshot of table added to include list 2022-06-16 08:39:16 +02:00
Chris Cranford
44159c95be DBZ-5225 Add EVENT_SCN_KEY key 2022-06-16 08:27:32 +02:00
Bob Roldan
18688077bf DBZ-5225 Align source.scn with the event's scn 2022-06-16 08:27:32 +02:00
Chris Cranford
4d30e6be47 DBZ-5257 Source info table order deterministic for rename schema changes 2022-06-16 08:19:38 +02:00
Chris Cranford
14a3b46ee5 DBZ-5258 Fix test assertions using static database names 2022-06-15 18:05:56 -04:00
Chris Cranford
46bef9c3ae DBZ-4890 BigDecimal scale match Field when column uses negative scale 2022-06-15 12:27:56 +02:00