Ivan San Jose
cc8dd160f5
DBZ-1617 Blacklisted columns are not being filtered out when generating a Kafka message from a CDC event
2019-11-25 13:23:03 +01:00
Brandon Brown
a0f7f6ec6b
DBZ-1402 Support for JDK 11
2019-11-25 11:53:19 +01:00
Jiri Pechanec
a56ab78b3f
DBZ-825 Unify decimal upscaling
2019-11-20 18:23:55 +01:00
pan3793
33384098fd
DBZ-1621 Remove useless null check
2019-11-19 07:35:03 +01:00
pan3793
91f0c2a520
DBZ-1621 Fix IoUtil.createDirectory parameter removeExistingContent is not take effect
2019-11-19 07:35:03 +01:00
pan3793
89fde5ce06
DBZ-1622 Fix VariableLatch.create parameter initialValue is not take effect
2019-11-19 07:33:52 +01:00
Debezium Builder
38279cc2dd
[maven-release-plugin] prepare for next development iteration
2019-11-14 02:56:07 -05:00
Debezium Builder
868faf4412
[maven-release-plugin] prepare release v1.0.0.Beta3
2019-11-14 02:56:07 -05:00
David Feinblum
2b9fbcd5f6
DBZ-1519 Avoid erroneously logging there are no monitored tables
2019-11-14 05:44:37 +01:00
Jiri Pechanec
2b9536f24d
DBZ-1607 use Instant for envelope timestamps
2019-11-12 16:36:11 -05:00
Gunnar Morling
da2b6c2c78
DBZ-1555 JavaDoc fix; formatting
2019-11-08 15:09:46 +01:00
Luis Garcés-Erice
0198bceffa
DBZ-1555 Enabling configuration of table parse mode to support DB2
2019-11-08 15:07:25 +01:00
Gunnar Morling
2c86b1c84f
DBZ-1604 Documentation update
2019-11-08 10:23:23 +01:00
Yang Yang
884def88d7
DBZ-1604 Added a new converter to enable byte[] payload
2019-11-08 10:19:21 +01:00
Jiri Pechanec
d1ee65d1b2
DBZ-1569 Don't compute index statistics
2019-11-06 15:43:21 +01:00
Jiri Pechanec
c179076710
DBZ-1592 Generate heartbeats for filtred events
2019-11-05 14:54:56 -05:00
Chris Cranford
f97be3272c
DBZ-1590 Fix LogInterceptor to be thread-safe
2019-11-04 16:23:43 -05:00
Gunnar Morling
2c569977d7
DBZ-1392 Applying formatter;
...
* Keeping indentation formatting for JSON
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
Co-authored-by: Chris Cranford <chris@hibernate.org>
2019-11-01 13:20:43 +02:00
Mike Graham
13c6c67a1d
fix spelling error in exception message
2019-10-24 22:00:16 +02:00
Debezium Builder
b08f523ed7
[maven-release-plugin] prepare for next development iteration
2019-10-24 07:42:49 -04:00
Debezium Builder
8ecd9e28cf
[maven-release-plugin] prepare release v1.0.0.Beta2
2019-10-24 07:42:49 -04:00
Gunnar Morling
e06d25dd92
DBZ-1336 Moving Ltree type to PG connector module
2019-10-21 12:17:08 +02:00
Mincong HUANG
1e3eb8c840
DBZ-1336 Support Postgres LTREE columns
2019-10-21 12:10:46 +02:00
Jiri Pechanec
a92b93182e
DBZ-1560 Removed dropped PK from the list
2019-10-20 22:13:02 +02:00
Gunnar Morling
4ea85d7fd3
DBZ-1565 Raise warning about growing WAL backlog
2019-10-18 13:53:53 +03:00
Debezium Builder
519dc011ff
[maven-release-plugin] prepare for next development iteration
2019-10-17 04:20:13 -04:00
Debezium Builder
51ef6b9470
[maven-release-plugin] prepare release v1.0.0.Beta1
2019-10-17 04:20:12 -04:00
Grant Cooksey
18ca6ec72e
DBZ-1546 Comma-separated regular expressions remove whitespace.
...
The table whitespace parameter should remove whitespace between regular expressions
since tables cannot start with whitespace characters.
2019-10-17 10:13:44 +03:00
Jiri Pechanec
6d8412e5de
DBZ-1401 Support for pgoutput
2019-10-10 09:37:52 +02:00
Jiri Pechanec
7c4203cf60
DBZ-1401 Handle tables without explicit PK
2019-10-10 09:37:52 +02:00
Jiri Pechanec
cf7eccb152
DBZ-1498 Docs added; microseconds changed to numeric; nanos supported
2019-10-09 12:14:35 +02:00
Jiri Pechanec
20363f24bb
DBZ-1498 Support for precise interval value
2019-10-09 12:14:35 +02:00
Jiri Pechanec
74fd88afca
DBZ-1513 Improve start logging
2019-10-07 18:38:35 +02:00
Jiri Pechanec
2c4cb83259
DBZ-1513 Centralize handling of non-data messages in SMT
2019-10-07 18:38:35 +02:00
Gunnar Morling
92885ebe99
DBZ-1392 Removing trailing whitespace from Java files
2019-10-07 12:11:43 +03:00
Gunnar Morling
8d6f5409b1
DBZ-1392 Moving static imports up and applying this to all files
2019-10-07 12:11:43 +03:00
Chris Cranford
3594ff639a
DBZ-1341 Rebased against master and applied necessary checkstyle fixes
2019-10-03 19:44:43 +02:00
Chris Cranford
ca5517746a
DBZ-1341 Checkstyle flag missing whitespace
2019-10-03 19:44:43 +02:00
Chris Cranford
7442670692
DBZ-1355 Fix checkstyle problems detected after upgrade to 8.19
2019-10-03 19:44:43 +02:00
Chris Cranford
5ac2318d84
DBZ-1391 Do not allow code after closing brace ( #1065 )
...
* DBZ-1391 Do not allow code after closing brace
2019-10-02 20:44:40 +02:00
Jiri Pechanec
f868cb87c0
DBZ-1280 Lock timeout for SQL Server
2019-10-02 10:34:25 +02:00
pushpavanthar
891c8b90f4
DBZ-1492 - MySQL Connector Fails while parsing AWS RDS internal events
2019-10-02 10:31:09 +02:00
Gunnar Morling
d94b37c693
DBZ-1507 Misc. clean-up;
...
* Removing unused Table methods
* Re-using primaryKeyColumnNames() instead of custom predicate
2019-10-02 10:25:37 +02:00
Chris Cranford
d906096820
DBZ-1507 Ensure message keys have correct field order
2019-10-02 10:25:37 +02:00
Debezium Builder
2ff77b993b
[maven-release-plugin] prepare for next development iteration
2019-10-01 15:34:23 -04:00
Debezium Builder
e09d584b11
[maven-release-plugin] prepare release v0.10.0.Final
2019-10-01 15:34:23 -04:00
Jiri Pechanec
410662c4aa
DBZ-1517 Skip unknow records while operation header is used
2019-09-30 20:25:44 +02:00
Debezium Builder
2f5960c5d2
[maven-release-plugin] prepare for next development iteration
2019-09-27 01:03:50 -04:00
Debezium Builder
e3b44ffb12
[maven-release-plugin] prepare release v0.10.0.CR2
2019-09-27 01:03:50 -04:00
Gunnar Morling
2fba51863e
DBZ-1015 Unifying the retrieval of custom key mappers
2019-09-26 10:14:48 +02:00
Guillaume Rosauro
1405889bef
DBZ-1015-allow-user-to-customize-key
2019-09-26 10:14:21 +02:00
Gunnar Morling
236bae4265
DBZ-1508 Avoiding assertion based on platform-specific line endings
2019-09-26 09:25:24 +02:00
Gunnar Morling
442408be02
DBZ-1497 Using int64 for INTERVAL microseconds mapping
2019-09-25 11:21:40 +03:00
Chris Cranford
b20e689b2c
DBZ-1495 Avoid NPE by throwing more meaningful error messages
2019-09-25 08:08:16 +03:00
Jiri Pechanec
8f308746d3
DBZ-1438 Interrupt only after interrupt affected operations
2019-09-24 12:51:48 +02:00
jerrinot
3064a4af06
DBZ-1438 Fix InterruptedException handling
...
It appears there is an intent to restore the interruption flag
when catching InterruptedException. However the old code was just
checking the current state of the flag instead of raising it.
2019-09-24 12:51:48 +02:00
Gunnar Morling
9de7c25d60
DBZ-1405 Misc adjustments;
...
* Adding test
* Removing remaining double variables
2019-09-24 12:08:55 +02:00
Bingqin Zhou
d6353dd519
DBZ-1405 Add NanoDuration.
2019-09-24 12:08:55 +02:00
Gunnar Morling
c5d0b18ea3
DBZ-1044 "AVRO" -> "Avro", as it's not an abbreviation
2019-09-23 12:21:54 +02:00
Chris Cranford
074cbf01a3
DBZ-1044 Checkstyle fix: remove unused import
2019-09-20 07:40:23 +03:00
Chris Cranford
6dc37fc8f6
DBZ-1044 Fix broken test caused by previous commit
2019-09-20 07:40:23 +03:00
Chris Cranford
910d00f6a2
DBZ-1044 Fixed documentation typo, added tests, adjusted FieldNameSelector
2019-09-20 07:40:23 +03:00
Chris Cranford
25086e82e3
DBZ-1044 Add auto-Enable sanitize.field.names
support
...
When the connector detects that its configuration explicitly specifies
that "key.converter" or "value.converter" are using the AvroConverter,
the "sanitize.field.names" option will be automatically enabled.
2019-09-20 07:40:23 +03:00
Chris Cranford
130210b5ad
DBZ-1044 Add config option sanitize.field.names
and FieldNameSelector
2019-09-20 07:40:23 +03:00
Josh Arenberg
6573738fe9
DBZ-1044 Sanitize column/topic names to always be valid Avro Names
...
This patch fixes the issue where more lenient standards for name conventions in source databases cause the
snapshot to fail when the schema for the table is created. This patch will prepend any column names that start
with a digit with an underscore, and will replace any subsequent invalid characters with an underscore. This should
conform to the Avro Name standard here: https://avro.apache.org/docs/1.7.7/spec.html#Names
2019-09-20 07:40:23 +03:00
Debezium Builder
9b7e09fdda
[maven-release-plugin] prepare for next development iteration
2019-09-10 09:12:12 -04:00
Debezium Builder
a1c2861458
[maven-release-plugin] prepare release v0.10.0.CR1
2019-09-10 09:12:11 -04:00
Renato Mefi
405a3b5b43
DBZ-1457 Do not commit when offset is null
2019-09-03 22:39:37 +02:00
Gunnar Morling
3215a3ce0a
DBZ-1448 Misc. clean-up
2019-09-03 10:39:41 +02:00
garrett528
1aa7d70b0d
DBZ-1448 Added support for source fields to be added to delete messages rewrites
2019-09-03 10:21:56 +02:00
Gunnar Morling
2a065686bd
DBZ-1255 Exporting null value when temporal precision mode is "connect" and the value is out of the supported range
2019-08-21 07:50:40 +02:00
Gunnar Morling
fe9fc1c0dc
DBZ-1255 Adjustments after rebase
2019-08-21 07:50:40 +02:00
Gunnar Morling
08ef1418d9
DBZ-DBZ-1255 Preventing value overflows for TIMESTAMP;
...
* Generally avoiding conversion to nano-seconds to accomodate for larger values
* Generally avoiding the usage of long to convey temporal values internally:
* Instant is used for TIMESTAMP
* OffsetDateTime for TIMESTAMPTZ
* Duration for TIME (LocalTime cannot be used as it doesn't support 24:00:00 as possible with Postgres)
* OffsetTime for TIMETZ
* Avoiding usage of methods under test for calculating expected values in tests
2019-08-21 07:50:40 +02:00
Gunnar Morling
94eb75257b
DBZ-1255 Using Instant for representing timestamps
2019-08-21 07:50:40 +02:00
Jiri Pechanec
51c12d930d
DBZ-1419 Support for Connect temporal types in SQL Server
2019-08-20 18:43:03 +02:00
Debezium Builder
4ae105bdce
[maven-release-plugin] prepare for next development iteration
2019-08-16 03:42:55 -04:00
Debezium Builder
945ccf68cb
[maven-release-plugin] prepare release v0.10.0.Beta4
2019-08-16 03:42:54 -04:00
Jiri Pechanec
540021cf68
DBZ-777 Minor fixes
2019-08-15 20:43:31 +02:00
Gunnar Morling
5ffe8123f1
DBZ-777 Further fixes
2019-08-15 20:43:31 +02:00
Gunnar Morling
63250d0c13
DBZ-777 Misc. adjustments
...
* Avoiding repeated parsing of table id
* Making 'inconsistentSchemaHandler' field final in EventDispatcher
* Not exposing synchronizeTableSchema() on SPI level
2019-08-15 20:43:31 +02:00
Jiri Pechanec
037d581844
DBZ-777 Fix heartbeats
2019-08-15 20:43:31 +02:00
Jiri Pechanec
d72afa8e9b
DBZ-777 Skip empty messages
2019-08-15 20:43:31 +02:00
Jiri Pechanec
f4246df6e4
DBZ-777 PostgreSQL connector rewrite to the standard framework
2019-08-15 20:43:31 +02:00
Jiri Pechanec
8adb5e48ca
DBZ-1385 Elevate payload for only headers setup
2019-08-12 10:36:52 +02:00
Jiri Pechanec
0991c70ac9
DBZ-1385 eventType can be configured as add field
2019-08-12 10:36:52 +02:00
Guillaume Rosauro
4b44e5f925
[DBZ-1254] Re-order the captured tables in the same order as the
...
'table.whitelist' parameter
2019-07-25 07:33:22 +02:00
Jiri Pechanec
0742e5775f
Revert "[maven-release-plugin] prepare release v0.10.0.Beta4"
...
This reverts commit c7340e5152
.
2019-07-23 14:54:55 +02:00
Jiri Pechanec
f7d20635ce
Revert "[maven-release-plugin] prepare for next development iteration"
...
This reverts commit 22879777e4
.
2019-07-23 14:54:49 +02:00
Jiri Pechanec
22879777e4
[maven-release-plugin] prepare for next development iteration
2019-07-23 14:10:03 +02:00
Jiri Pechanec
c7340e5152
[maven-release-plugin] prepare release v0.10.0.Beta4
2019-07-23 14:10:02 +02:00
Debezium Builder
71333fda98
[maven-release-plugin] prepare for next development iteration
2019-07-23 02:55:08 -04:00
Debezium Builder
20d006fac7
[maven-release-plugin] prepare release v0.10.0.Beta3
2019-07-23 02:55:08 -04:00
Adrian Kreuziger
00843cb0fb
DBZ-1395 Update to use BoundedConcurrentHashMap for schema cache
2019-07-18 10:26:05 +02:00
Adrian Kreuziger
6c63c7daa0
DBZ-1395 Throw exception for missing source field
2019-07-18 10:26:05 +02:00
Adrian Kreuziger
63299022d7
DBZ-1395 Review feedback (keep field type, minor tweaks)
2019-07-18 10:26:05 +02:00
Adrian Kreuziger
dda17b9aa2
DBZ-1395 Undo accidental change to existing test
2019-07-18 10:26:05 +02:00
Adrian Kreuziger
c7684c351c
DBZ-1395 Add ability to insert field from original source struct to unrwrapped record
2019-07-18 10:26:05 +02:00
Jiri Pechanec
3008339142
DBZ-916 Keyless tables have null key by default
2019-07-17 14:41:37 -04:00
Gunnar Morling
3b6e6b7719
Merge pull request #955 from jpechane/DBZ-1198
...
DBZ-1198 Heartbeat for MongoDB connector
2019-07-11 07:44:04 +02:00
Gunnar Morling
287551294f
DBZ-1198 Adding comments
2019-07-11 07:43:32 +02:00
Renato Mefi
1b03fd3c46
DBZ-1388 Ensure Outbox SMT skips non CDC records
2019-07-10 22:32:23 +02:00
Jiri Pechanec
0316ac7fc0
DBZ-1198 Heartbeat for MongoDB connector
2019-07-10 11:16:26 +02:00
Gunnar Morling
369f74d875
DBZ-1320 Addressing review remarks
2019-07-09 21:07:18 +02:00
Brandon Brown
efff4f0809
DBZ-1320 Adding support for deletion events via new configurable property route.tombstone.on.empty.payload
2019-07-09 21:07:18 +02:00
Jiri Pechanec
67ca8a2d76
DBZ-794 Filter out savepoints from database history
2019-07-08 12:54:01 +02:00
Gunnar Morling
e2d8736a79
DBZ-1380 Misc. clean-up;
...
* reducing visibility of some methods
* resetting interrupted state
2019-07-05 21:08:35 +02:00
Gunnar Morling
02427a054b
DBZ-1222 EventsInError -> ErroneousEvents
2019-07-04 21:31:32 +02:00
Jiri Pechanec
c08a5374fe
DBZ-1222 Improve error event method name
2019-07-04 21:31:32 +02:00
Jiri Pechanec
2b365be63c
DBZ-1222 EventsInError metric added
2019-07-04 21:31:32 +02:00
Jiri Pechanec
13d638fd43
DBZ-1356 Metrics for schema history ( #947 )
...
* DBZ-1356 Metrics for schema history
2019-07-02 13:26:17 +02:00
Jiri Pechanec
b1c1c05d1f
DBZ-1363 Generate heartbeat periodically not upon event
2019-07-02 12:02:33 +02:00
Gunnar Morling
08c5d36789
Merge pull request #942 from jpechane/stab-lsn-flush-test
...
DBZ-1362 Stabilize test executions on CI
2019-07-01 09:34:24 +02:00
Jiri Pechanec
b546dc886e
DBZ-1362 Fix Snapshot/streaming switch race condition
2019-06-28 09:09:18 +02:00
Gunnar Morling
254dba97ee
DBZ-1358 Avoiding instantiation of SystemTime
2019-06-28 08:54:04 +02:00
Jiri Pechanec
a0df2c140b
Use Kafka SystemTime implementation
2019-06-28 08:54:04 +02:00
Jiri Pechanec
53f4db06b9
DBZ-1358 Update to Kafka 2.3.0
2019-06-28 08:54:04 +02:00
Jiri Pechanec
7e03dbe87e
DBZ-1362 Log test method executions
2019-06-28 05:41:02 +02:00
Jiri Pechanec
82d245acb1
DBZ-1362 Set longer ZooKeeper timeout
2019-06-28 05:33:49 +02:00
Gunnar Morling
a41fb30eaa
DBZ-1361 Restoring compatibility with Apache Kafka 1.x
2019-06-28 05:07:52 +02:00
Jenkins user
e95d656bca
[maven-release-plugin] prepare for next development iteration
2019-06-27 07:36:13 +00:00
Jenkins user
d95fd87311
[maven-release-plugin] prepare release v0.10.0.Beta2
2019-06-27 07:36:12 +00:00
Chris Cranford
d518fa59fe
DBZ-1345 Should not snapshot materialized views
2019-06-25 12:44:00 +02:00
Cheng Pan
186babeabf
DBZ-1346 Fixing description for state of snapshot
2019-06-24 10:52:16 +02:00
Gunnar Morling
d3e399e167
DBZ-1340 Avoiding repeated case conversion of incoming DML statements;
...
Also adding startsWithIgnoreCase() method.
2019-06-20 13:33:12 +02:00
Jiri Pechanec
3012524ea1
DBZ-1278 Rename to non-whitelisted table sends warn
2019-06-19 17:05:26 +02:00
Gunnar Morling
ee87f40f0c
DBZ-1270 Avoiding usage of deprecated method; adding Stathis Souris to authors
2019-06-14 11:35:53 +02:00
Gunnar Morling
bb360e472c
DBZ-1270 Logging improvements
2019-06-14 11:35:53 +02:00
ssouris
d513e2c757
DBZ-1270 - Fix KafkaDatabaseHistory throwing even after creating the topic in older brokers
2019-06-14 11:35:53 +02:00
Jiri Pechanec
9b5640ddde
DBZ-835 Emit tombstone optionally
2019-06-12 21:39:13 +02:00
Jenkins user
5fdb0ca81b
[maven-release-plugin] prepare for next development iteration
2019-06-11 11:20:46 +00:00
Jenkins user
61729b9e00
[maven-release-plugin] prepare release v0.10.0.Beta1
2019-06-11 11:20:46 +00:00
Gunnar Morling
5d01f409f0
DBZ-1317 Separating update and query methods for MongoDB's SourceInfo;
...
* Avoids access to collection id after events which don't have one
* Pulling up struct() and schema() methods to common base class
2019-06-11 10:07:36 +02:00
Chris Cranford
0aadaa10dc
DBZ-1242 Fixed typo and added negative test checks
2019-06-05 13:31:25 -04:00
Chris Cranford
b3ca33287a
DBZ-1242 Issue warning when filters produce an empty schema
2019-06-05 13:31:25 -04:00
Jenkins user
056730d06a
[maven-release-plugin] prepare for next development iteration
2019-06-03 10:24:08 +00:00
Jenkins user
7aeaec236e
[maven-release-plugin] prepare release v0.10.0.Alpha2
2019-06-03 10:24:08 +00:00
Gunnar Morling
246c1be305
DBZ-1283 Enforcing presence of database.server.name globally
2019-06-03 10:58:14 +02:00
Chris Cranford
5f0b44b4ec
DBZ-1283 Only validate history topic name for affected connectors
2019-06-03 10:58:14 +02:00
Jiri Pechanec
ff51e99b6f
DBZ-1295 Base SourceInfor for snapshotting
2019-05-31 10:56:05 +02:00
Jiri Pechanec
4b64c8e447
DBZ-1295 3state snapshot for MySQL
2019-05-31 10:56:05 +02:00
Jiri Pechanec
7f35d70d0e
DBZ-1295 3state snapshot for SQL Server
2019-05-31 10:56:05 +02:00
Jiri Pechanec
348b2415d5
DBZ-1295 3state snapshot for PostgreSQL
2019-05-31 10:56:05 +02:00
Gunnar Morling
ca361bd122
DBZ-871 Clearly separating between DB and table events
2019-05-30 08:51:28 +02:00
Jiri Pechanec
e826599e2e
DBZ-871 Add table(s) name to schema change event
2019-05-30 08:51:27 +02:00
Gunnar Morling
7759333fe3
DBZ-1092 Using ColumnNameFilter instead of Predicate<Column>
2019-05-29 15:49:49 +02:00
Jenkins user
ac5c76e431
[maven-release-plugin] prepare for next development iteration
2019-05-28 14:38:32 +00:00
Jenkins user
35f59ba460
[maven-release-plugin] prepare release v0.10.0.Alpha1
2019-05-28 14:38:32 +00:00
Gunnar Morling
eef37cb567
DBZ-1235 Misc adjustments;
...
* Removing superfluous parameter from getSourceInfoStructMaker()
* JavaDoc
* Renaming field to "source.struct.version"
2019-05-28 09:35:15 +02:00
Gunnar Morling
8f28f323e7
DBZ-1235 Using Instant to represent timestamps internally;
...
Only "rendered" as epoch milli-seconds at the system boundaries.
2019-05-28 09:35:15 +02:00
Jiri Pechanec
6f6c8ddf06
DBZ-1235 Refactored to unify the code
2019-05-28 09:35:15 +02:00
Jiri Pechanec
ea63dbf291
DBZ-1235 TableId added to SQL server connector
2019-05-28 09:35:15 +02:00
Jiri Pechanec
e6ed5f867a
DBZ-1235 Make struct maker configured via enum
2019-05-28 09:35:15 +02:00
Jiri Pechanec
49910dcabb
DBZ-1235 Pluggable source maker for SqlServer
2019-05-28 09:35:15 +02:00
Jiri Pechanec
f5757dbd9f
DBZ-1235 Extract serverName to common source
2019-05-28 09:35:15 +02:00
Gunnar Morling
6f36eede61
DBZ-677 Adding missing header
2019-05-27 13:52:58 +02:00
Gunnar Morling
cf4db8c1a1
DBZ-677 Extracting shared configuration options for both new state extraction SMTs
2019-05-27 13:52:58 +02:00
Gunnar Morling
cb9e916616
DBZ-677 Renaming record state SMTs;
...
Keeping old names in deprecated sub-classes for the time being.
2019-05-27 13:52:58 +02:00
Gunnar Morling
4ed8b4636c
DBZ-1224 Completing implementation;
...
* Further unification of override handling;
* Undoing change related to SnapshotStatementFactory as it seems not needed
* Adding test
2019-05-27 08:44:42 +02:00
Gunnar Morling
034739ab06
DBZ-1224 Making defaultSnapshotFetchSize a primitive
2019-05-27 08:44:42 +02:00
mokkara
b3de0250a9
DBZ-1224 Snapshot SELECT support for SQL Server
2019-05-27 08:44:42 +02:00
Gunnar Morling
a0e30b7c90
DBZ-1289 Making outbox router options more consistent
2019-05-24 13:18:55 +02:00
Jiri Pechanec
c1c4d3d149
DBZ-683 After review changes
2019-05-22 14:52:06 +02:00
Jiri Pechanec
540196091c
DBZ-683 Snapshot supports storing only whitelisted tables setting
2019-05-22 14:52:06 +02:00
Chris Cranford
e2c74111ad
DBZ-1234 Removed deprecated configuration options
2019-05-22 10:19:05 +02:00
Gunnar Morling
1ed8615434
DBZ-1234 Removing deprecated "drop.deletes" SMT option
2019-05-21 14:11:32 +02:00
Jiri Pechanec
a8e7104f18
DBZ-1209 Remove events skipped metric form common metrics
2019-05-21 10:26:22 +02:00
Jiri Pechanec
547ac819df
DBZ-1257 Fixed typo
2019-05-21 10:11:57 +02:00
Jiri Pechanec
8c376d1795
DBZ-1257 Validate data against schema
2019-05-21 10:11:57 +02:00
Gunnar Morling
4751b266f3
DBZ-1164 Misc. improvements;
...
* Moving stringToDuration() for the sake of re-use and testability
* Making pattern and durations constants
2019-05-21 09:38:23 +02:00
Jenkins user
7748be6056
[maven-release-plugin] prepare for next development iteration
2019-05-02 16:04:56 +00:00
Jenkins user
5af083b86e
[maven-release-plugin] prepare release v0.9.5.Final
2019-05-02 16:04:55 +00:00
Jiri Pechanec
b53f456912
DBZ-1245 Heartbeat only when offsets ready
2019-05-02 08:22:28 +02:00
Chris Cranford
c6aa02f813
DBZ-1226 Add support for enum/set values that contain commas or escaped characters.
2019-05-01 21:17:42 +02:00
Gunnar Morling
a84230447e
DBZ-1247 Making defaultSnapshotFetchSize() abstract
2019-04-30 09:57:11 +02:00
jchipmunk
72b06366fe
Removing extra initialisation of "config" variable in PG config
2019-04-30 09:57:11 +02:00
jchipmunk
9fe033ce24
DBZ-1247 Specifying default snapshot fetch size for each connector config
2019-04-30 09:57:11 +02:00
jchipmunk
a5cc83c0d1
DBZ-1247 Adding property to specify snapshot fetch size
2019-04-30 09:57:11 +02:00
krizhan
70c564b4c2
DBZ-212 Moved SERVER_NAME to RelationalDatabaseConnectorConfig
2019-04-23 12:29:44 +02:00
Jenkins user
cefc5ccfb9
[maven-release-plugin] prepare for next development iteration
2019-04-11 12:07:16 +00:00
Jenkins user
fc3026adc3
[maven-release-plugin] prepare release v0.9.4.Final
2019-04-11 12:07:15 +00:00
Steviep
94f71ea96e
DBZ-1206 Adding new connector metric "numberOfFilteredEvents"
2019-04-09 14:46:54 +02:00
jchipmunk
7c0ae3ee20
DBZ-1212 SLF4J usage issues
2019-04-04 21:32:12 +02:00
Joy Gao
f3220e9766
DBZ-1207 MySQL connection timeout after bootstrapping a new table
2019-04-02 12:52:39 +02:00
Jiri Pechanec
b90274cc63
DBZ-1194 Convert nulls even for key
2019-03-29 21:20:53 +01:00
ShubhamRwt
540a951211
DBZ- 362 Adding WhitespaceAfter check to Checkstyle
2019-03-28 09:24:11 +01:00
Jenkins user
8b0edd1871
[maven-release-plugin] prepare for next development iteration
2019-03-25 12:16:31 +00:00
Jenkins user
56f98a48b2
[maven-release-plugin] prepare release v0.9.3.Final
2019-03-25 12:16:30 +00:00
Jiri Pechanec
7524b1ec89
DBZ-1169 Mark outbox as incubating
2019-03-25 11:27:44 +01:00
Renato Mefi
0847870d21
DBZ-1169 Test for invalid Outbox operations
...
Also removed unused code
2019-03-25 11:27:44 +01:00
Renato Mefi
be9c70db64
DBZ-1169 Ensure SMT configuration is valid
...
Also had to update a validator
2019-03-25 11:27:44 +01:00
Renato Mefi
87a3278dfe
DBZ-1169 Support user set Schema version
...
The cache mechanism had to be adapted in order to support non-versioned
and versioned schemas, a test now confirms it's the same valueSchema
instance created once.
2019-03-25 11:27:44 +01:00
Renato Mefi
4aec4d5e92
DBZ-1169 Schema should inherit from original type
...
To allow more flexibility we should proxy the new Event Schema fields
based on the original table columns which Debezium is capturing.
The Schema is now built one time during the first Record in order to
detect those types which are only available within the Record.
2019-03-25 11:27:44 +01:00
Renato Mefi
c14ce54a6f
DBZ-1169 Support additional user fields
2019-03-25 11:27:44 +01:00
Renato Mefi
2a8806e15c
DBZ-1169 Support setting Record timestamp
...
It's a good practice to depend on the Kafka metadata instead of custom
dates in the payload, this way for instance when using KStreams with
Tumbling Window the dates are correctly matched.
2019-03-25 11:27:44 +01:00
Renato Mefi
8b689d7f03
DBZ-1169 Implement RegexRouter based on config
2019-03-25 11:27:44 +01:00
Renato Mefi
f0f13ed4a4
DBZ-1169 Implement basic behavior
2019-03-25 11:27:44 +01:00
Renato Mefi
ebf36e1ddb
DBZ-1169 Support skipping deletions and tombstones
2019-03-25 11:27:44 +01:00
Renato Mefi
6512c0b387
DBZ-1169 Add Configuration for Outbox Transform
2019-03-25 11:27:44 +01:00
Jiri Pechanec
9b405aa84d
DBZ-1039 Add missing spaces
2019-03-25 10:23:14 +01:00
shubham
8cca21e969
DBZ-1039 Make all ifs with braces
2019-03-25 10:23:14 +01:00
Addison Higham
8e21139ca3
DBZ-1082 Add new custom recovery mode, more metadata
...
This commit does a few things:
- Refactors snapshot modes to be encapsulated by an interface and
to only use that interface in determining when to snapshot and in
determing the type of the `RecordProducer` interface to instantiate
- Refactors the configuration of existing snapshot modes to tie the
existing snapshot modes to their aligned implementation
- Adds a new snapshot.mode, custom, and a new configuration option to
specify a custom implementation that will be loaded by the class loader
- Changes the visibility of some classes to allow for custom snapshot
modes to get enough context to make an informed choice
- Adds some metadata about slots (the catalog_xmin) to give a full idea
of the state of slots which can be useful in implementing snapshot
modes (which is also configurable, as it can add some overhead)
Together, these changes allow for a much broader flexibility got end
users to implement a snapshot mode that can do more advanced snapshots,
such as partial recovery or for partial snapshots for tables where not
all records are needed.
This could also be seen as superseeding the
`snapshot.select.statement.overrides` to allow for users to dynamically
build queries based on the state of the slot and the offsets consumed.
2019-03-24 11:56:40 +01:00
Jon Casstevens
2efdae1ffd
DBZ-1179 Set the default replication factor property for history topics when none is provided
2019-03-21 12:07:10 +01:00
Gunnar Morling
2ea87727cf
DBZ-1174 Using Instant instead of long for representing timestamps
2019-03-12 17:19:13 +01:00
Jiri Pechanec
71765a2ab2
DBZ-1170 Detect and handle errors like ULE
2019-03-11 10:29:54 +01:00
Josh Stanfield
e4b7b90637
DBZ-1162 fix to ensure hstore snapshots behave the same as streams
2019-03-04 13:39:50 +01:00
Jenkins user
50a7c568fa
[maven-release-plugin] prepare for next development iteration
2019-02-23 09:53:58 +00:00
Jenkins user
28f3839804
[maven-release-plugin] prepare release v0.9.2.Final
2019-02-23 09:53:58 +00:00
Gunnar Morling
6787d17d54
DBZ-1141 Misc. clean-up;
...
* Removing redundant check for date mapping type
* Always using String as fallback value for temporal values where needed
* Pulling fallback temporal values up to JdbcValueConverters
2019-02-21 15:02:51 +01:00
Jiri Pechanec
41ba7128e3
DBZ-1141 Correct default falback values for connect date types
2019-02-21 15:02:51 +01:00
Keith Barber
95b2290df2
DBZ-1149 Ensure consistent Java types for fallback values when converting
2019-02-21 15:02:51 +01:00
Jiri Pechanec
ff5aaf2e9a
DBZ-1153 Support for special char names
2019-02-20 15:25:45 +01:00
Gunnar Morling
510e9415f9
DBZ-1137 Some clean-up;
...
* Using existing justifyLeft() method
* Giving expected values as Duration
* Removing unused methods
2019-02-19 18:41:09 +01:00
Jiri Pechanec
0d60e60243
DBZ-1137 Default for TIME can be generic interval value
2019-02-19 13:40:01 +01:00
Jenkins user
b96e5fee0d
[maven-release-plugin] prepare for next development iteration
2019-02-13 11:13:44 +00:00
Jenkins user
3de2d04bcd
[maven-release-plugin] prepare release v0.9.1.Final
2019-02-13 11:13:44 +00:00
Jiri Pechanec
9049078607
DBZ-997 Handle de-sync of message and JDBC PK info
2019-02-13 10:13:12 +01:00
tomaz.fernandes
4e6b44d925
DBZ-1122 Fixed a potential NPE in FileDatabaseHistory
2019-02-12 13:17:09 +01:00
Gunnar Morling
2a486ecfff
DBZ-1019 Removing legacy methods from Metronome
2019-02-11 18:11:34 +01:00
Jiri Pechanec
fa6fcd8487
DBZ-1019 Guard against wall-clock change
2019-02-11 18:11:34 +01:00
Jiri Pechanec
4d76243227
DBZ-1121 Emit ALTER event for updated tables
2019-02-09 20:03:57 +01:00
Gunnar Morling
7076deba5a
DBZ-1118 Adding JUnit rule to skip tests depending on Postgres version
2019-02-04 14:43:19 +01:00
Jiri Pechanec
16620f0bf9
DBZ-1040 Cleaning up event formatting
2019-02-04 12:39:07 +01:00
Gunnar Morling
99caad60d8
DBZ-1040 Changing resolution of "lag behind source" to milli-seconds;
...
Also using Duration/Instant types internally.
2019-02-04 12:39:07 +01:00
Gunnar Morling
f1c4234248
DBZ-1040 Removing value from metrics events
2019-02-04 12:39:07 +01:00
Gunnar Morling
913e0bc5c8
DBZ-1040 Making metadata provider a member of Metrics instead of passing it for each onEvent() call
2019-02-04 12:39:07 +01:00
Gunnar Morling
399f91ce84
DBZ-1040 Typo fix
2019-02-04 12:39:07 +01:00
Gunnar Morling
27274d664f
DBZ-1040 Passing source as DataCollectionId instead of Object
2019-02-04 12:39:07 +01:00
Jiri Pechanec
de171917dc
DBZ-1040 Improve event to string formatting
2019-02-04 12:39:07 +01:00
Jiri Pechanec
a621e43d8b
DBZ-1040 Expose transaction id
2019-02-04 12:39:07 +01:00
Jiri Pechanec
213d4a42ca
DBZ-1040 Metrics for internal queue
2019-02-04 12:39:07 +01:00
Jiri Pechanec
d993ee265e
DBZ-1040 Metrics for lag behind source, offset and tx count
2019-02-04 12:39:07 +01:00
jchipmunk
acbc94e8b0
DBZ-1112 rename prepareQuery() to prepareQueryWithBlockingConsumer()
...
Reason:
reference to prepareQuery is ambiguous
[ERROR] both method prepareQuery(java.lang.String,io.debezium.jdbc.JdbcConnection.StatementPreparer,io.debezium.jdbc.JdbcConnection.BlockingResultSetConsumer) in io.debezium.jdbc.JdbcConnection and method prepareQuery(java.lang.String,io.debezium.jdbc.JdbcConnection.StatementPreparer,io.debezium.jdbc.JdbcConnection.ResultSetConsumer) in io.debezium.jdbc.JdbcConnection match
2019-01-30 09:36:09 +01:00
jchipmunk
ff4e38cc46
DBZ-1112 Strings.join() doesn't apply conversation for first element
...
Replace:
- Integer.parseInt() to Integer.valueOf()
- Short.parseShort() and new Short() to Short.valueOf()
- Long.parseLong() to Long.valueOf()
2019-01-30 09:36:09 +01:00
Gunnar Morling
2a80931d03
DBZ-962 Avoiding Short constructor
2019-01-29 10:28:48 +01:00
Gunnar Morling
e84c6e2238
DBZ-962 Avoiding dependency to JAXB
2019-01-29 10:28:48 +01:00
Jenkins user
c5b7d21d3e
[maven-release-plugin] prepare for next development iteration
2019-01-28 11:07:04 +00:00
Jenkins user
f1ae79ff73
[maven-release-plugin] prepare release v0.9.0.CR1
2019-01-28 11:07:04 +00:00
Gunnar Morling
ccff79e07d
DBZ-1105 Removing TableImpl#columnNames field
2019-01-28 10:23:56 +01:00
Gunnar Morling
a27c29765a
DBZ-947 Formatting
2019-01-28 09:32:57 +01:00
Grzegorz Kołakowski
64c45c7490
DBZ-947 Reset initial transaction isolation level at snapshot end
2019-01-28 09:32:57 +01:00
Gunnar Morling
ebe5fd7e85
DBZ-1067 Adding SourceRecordAssert
2019-01-23 08:31:22 +01:00
Gunnar Morling
9665af3ebc
DBZ-1086 Adding Ivan Kovbas to COPYRIGHT.txt
2019-01-18 21:43:45 +01:00
Ivan Kovbas
81f031cb7c
DBZ-1086 Added functionality to gracefully handle key-less messages in ByLogicalTableRouter transformation.
2019-01-18 21:38:52 +01:00
Gunnar Morling
cedef89a48
DBZ-1089 Using Strings#isNullOrEmpty(); adding test
2019-01-18 10:07:20 +01:00
Jiri Pechanec
28c9734685
DBZ-1089 Use only schema name when both catalog and schema is present
2019-01-18 10:07:20 +01:00
Jiri Pechanec
25156730f0
DBZ-1088 use correct key for update of PK insert event
2019-01-17 14:28:18 +01:00
Jiri Pechanec
ceb6c62721
DBZ-1064 Remove lazy fallback values
2019-01-16 13:01:36 +01:00
Jiri Pechanec
536e517c08
DBZ-1064 Geometry default value changes; fixes for default suppliers
2019-01-16 13:01:36 +01:00
Gunnar Morling
532c17f8aa
DBZ-1064 Using dedicated functional interface instead of Consumer;
...
* Commenting
* Moving type from "util" to "jdbc"
2019-01-16 13:01:36 +01:00
Jiri Pechanec
bff7eea734
DBZ-1064 Use convert value template
2019-01-16 13:01:36 +01:00
Jiri Pechanec
920e0cbd3f
DBZ-1064 Set default value only for non-nullable fields
2019-01-16 13:01:36 +01:00
Gunnar Morling
de356896c0
DBZ-1029 Adding one more test
2019-01-15 17:41:42 +01:00
Amit Sela
b0b7d942a3
DBZ-1073 Adding scale as schema parameter, if present
2019-01-11 15:40:30 +01:00
Grzegorz Kołakowski
91c72738c0
DBZ-1067 Add column blacklist field to RelationalDatabaseConnectorConfig
2019-01-09 18:13:26 +01:00
Jenkins user
5275f73424
[maven-release-plugin] prepare for next development iteration
2018-12-19 13:06:12 +00:00
Jenkins user
b6569c18ae
[maven-release-plugin] prepare release v0.9.0.Beta2
2018-12-19 13:06:12 +00:00
Jiri Pechanec
0e98bb428b
DBZ-978 Make offset readable in last event
2018-12-19 11:03:01 +01:00
luobo
fcfc019b6b
fix the wrong words
2018-12-14 13:05:14 +01:00
Gunnar Morling
47f22feddc
DBZ-978 Misc. clean-up
2018-12-13 21:17:23 +01:00
Jiri Pechanec
edec1c3090
DBZ-978 Added last event and captured tables metrics
2018-12-13 20:52:11 +01:00
Gunnar Morling
f89fc0bb14
DBZ-966 Awaiting snapshot delay only when actually doing a snapshot
2018-12-13 14:29:54 +01:00
Gunnar Morling
5542c0de29
DBZ-966 Making sure Postgres connector can be stopped while awaiting initial snapshot delay;
...
* Also skipping snapshotting when requested to stop during initial delay
* Showing remaining delay time in logs
* Adding missing config
2018-12-13 14:29:54 +01:00
Gunnar Morling
7a577161da
DBZ-966 Pushing snapshot delay logic into coordinator;
...
Using timer + metronome to ensure the delay phase will be cancelled upon connector shutdown
2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
7f8c0e8ff1
DBZ-966 Enable to delay initial snapshot in HistorizedRelationalSnapshotChangeEventSource
2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
2a1934eeb1
DBZ-966 Make snapshot.delay.ms global property
2018-12-13 14:29:54 +01:00
Jiri Pechanec
2b2ff0754d
DBZ-960 Run SnapshotReader in a single transaction
2018-12-11 14:15:53 +01:00
Jiri Pechanec
284ae98adf
DBZ-1003 Initialize database history
2018-12-03 21:52:05 +01:00
Renato Mefi
8c1ebc2a8f
DBZ-971 Unwrap can add data operation header
2018-11-30 12:06:18 +01:00
Renato Mefi
0af8dcfde0
DBZ-971 Ensure Envelope unwrap propagates headers
2018-11-30 12:06:18 +01:00
Jenkins user
db1d3a7fb8
[maven-release-plugin] prepare for next development iteration
2018-11-20 16:15:14 +00:00
Jenkins user
f83db82cea
[maven-release-plugin] prepare release v0.9.0.Beta1
2018-11-20 16:15:14 +00:00
Gunnar Morling
6bd4024618
DBZ-776 Misc. improvements;
...
* Renaming getTimeSinceLastEvent() to getMilliSecondsSinceLastEvent()
* Further unifying metrics implementation across connectors
* Emitting event in EventDispatcher also if event is filtered out
* Typo fixes
2018-11-16 08:52:07 +01:00
Jiri Pechanec
4510e2bbe6
DBZ-776 Cleanup after review
2018-11-16 08:52:07 +01:00
Gunnar Morling
360714a5a1
DBZ-776 Formatting and minor clean-up
2018-11-16 08:52:07 +01:00
Jiri Pechanec
b44c9fd04f
DBZ-776 A first set of metrics implemented in new framework
2018-11-16 08:52:07 +01:00
Jiri Pechanec
4cdc3e669b
DBZ-776 Extract metrics to a common code
2018-11-16 08:52:07 +01:00
Gunnar Morling
9436778a1d
DBZ-812 Misc. clean-up
2018-11-13 10:06:22 +01:00
Jiri Pechanec
33f6529614
DBZ-812 PK list is not-null for no PKs on table
2018-11-13 09:41:03 +01:00
Jiri Pechanec
5930c580f7
DBZ-812 Change extract column metadata
2018-11-13 09:41:03 +01:00
Jiri Pechanec
839d7c80eb
DBZ-812 Prep statements with different parameters; Extract table schema code
2018-11-13 09:41:03 +01:00
Gunnar Morling
92704ce394
DBZ-963 Removing hard-coded references to "target" dir;
...
Also injecting actual build dir via system variable.
Making DATA_DIR a simple string.
2018-11-08 10:35:07 +01:00
Anton Martynov
3401cd23ea
DBZ-963 Allowing to customize the directory where tests are storing their data
2018-11-08 10:34:10 +01:00
Gunnar Morling
9994172860
DBZ-953 Pulling decimal mode handling up to RelationalDatabaseConnectorConfig;
...
Also using it in MySQL connector.
2018-11-08 09:23:28 +01:00
Grzegorz Kołakowski
2d5295805c
DBZ-941 Add connectionCreated template method in HistorizedRelationalSnapshotChangeEventSource
...
The method allows to define steps which have to be taken just after the
database connection is created (e.g. setting snapshot isolation level).
By default no operation is executed.
2018-10-30 13:35:57 +01:00
Jenkins user
d952f5dfb0
[maven-release-plugin] prepare for next development iteration
2018-10-04 11:59:14 +00:00
Jenkins user
ff9b70278b
[maven-release-plugin] prepare release v0.9.0.Alpha2
2018-10-04 11:59:13 +00:00
jchipmunk
05ba99bb0a
DBZ-918 Adding Debezium connector field to source info
...
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
2018-10-04 09:00:13 +02:00
Gunnar Morling
39a47fabfa
DBZ-899 Using custom statement to obtain SQL type info in order to avoid N+1 issues with PgDatabaseMetaData#getTypeInfo()
2018-09-19 09:51:38 +02:00
Gunnar Morling
3331d4c1a9
DBZ-899 Removing unused field;
...
Also inlining method only used in one place.
2018-09-19 09:51:38 +02:00
Jiri Pechanec
2253eaa6a6
DBZ-819 JdbcConnection is not thread safe
2018-09-18 09:05:36 +02:00
Jiri Pechanec
05ce2e3063
DBZ-819 Cache prepared statements in connection
2018-09-18 09:05:36 +02:00
jchipmunk
b0feaf496a
DBZ-633 Adding "field.blacklist" property for MongoDB connector
...
The "field.blacklist" configuration property is an optional comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form "databaseName.collectionName.fieldName.nestedFieldName", where "databaseName" and "collectionName" may contain the wildcard (*) which matches any characters.
Although the "field.blacklist" configuration property allows you to remove fields from the event values, the "_id" field is always included in the event’s key.
2018-09-14 12:54:11 +02:00
Gunnar Morling
5878e5105f
DBZ-878 Using LRU eviction strategy
...
[As recommended](4961277637 (commitcomment-30412629)
) by Ben Manes, the LIRS implementation seems to provide no advantage and isn't well tested.
2018-09-06 11:21:31 +02:00
Gunnar Morling
4961277637
DBZ-878 Making topic name cache size bound;
...
* Imported BoundedConcurrentHashMap from Hibernate ORM as a thread-safe concurrent cache.
2018-09-05 12:56:53 +02:00
Gunnar Morling
e1bbbd7992
DBZ-865 DBZ-878 Replacing invalid topic name characters with "_";
2018-09-05 12:56:53 +02:00
Gunnar Morling
eba134817f
DBZ-894 Making TableIdToStringMapper dedicated functional interface with specifically named method
2018-09-04 12:43:50 +02:00
Jiri Pechanec
1373ac8475
DBZ-894 Introduce explicit mapping functional type
2018-09-04 12:39:10 +02:00
Jiri Pechanec
3e0648a42c
DBZ-894 Customize TableId comparison per-connector in filters
2018-09-04 09:18:15 +02:00
Gunnar Morling
985999b4af
DBZ-872 Making MySqlAntlrDdlParser#tableFilter final
2018-08-24 08:12:54 +02:00
Jiri Pechanec
63ee65bb7d
DBZ-872 Honor filters in ALTER TABLE parsing
2018-08-24 08:12:54 +02:00
Jiri Pechanec
82a93cdac5
DBZ-859 Heartbeat is sent after pipeline snapshot
2018-08-23 11:44:53 +02:00
Jiri Pechanec
985e03e7b5
DBZ-859 Force heartbeat after snapshot completion
2018-08-23 11:44:53 +02:00
Maciej Bryński
947abf9815
DBZ-857 Ability to rewrites deleted records
2018-08-14 10:37:39 +02:00
artiship
57fa31e1bc
DBZ-854 Correct param name for excludeColumns(String fullyQualifiedTableNames)
2018-08-14 08:39:28 +02:00
LiuHanlin
41e16f4a4f
DBZ-853 Fix kafka database history storage miscount attemp number even if there are more records to consume
2018-08-13 20:26:48 +02:00
Jenkins user
e00dad127f
[maven-release-plugin] prepare for next development iteration
2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700
[maven-release-plugin] prepare release v0.9.0.Alpha1
2018-07-26 08:00:12 +00:00
Gunnar Morling
3061c1dc0b
DBZ-815 Passing offset to Heartbeat#heartbeat();
...
That's simpler to grasp than the approach of passing a supplier lambda to the constructor.
Also it allows to pass on the offset via local variables instead of instance fields in some cases.
2018-07-25 14:49:52 +02:00
Gunnar Morling
0a659c0159
DBZ-815 Ensuring latest offset is propagated also if change events relate to non-whitelisted tables
2018-07-25 14:49:52 +02:00
Gunnar Morling
fa68c1a158
DBZ-816 Preparing code for database history set-up for re-use
2018-07-25 11:54:15 +02:00
Jiri Pechanec
e86ddc0ab2
DBZ-40 Add blocking version of MultiResultSetConsumer
2018-07-25 11:51:09 +02:00
Gunnar Morling
bd4f3d60fa
DBZ-804 Simplifying construction of VariableScaleDecimal
2018-07-24 20:48:52 +02:00
Gunnar Morling
98c329aabe
DBZ-40 Adding SnapshottingTask#toString()
2018-07-24 18:53:32 +02:00
Gunnar Morling
0362b333bf
DBZ-804 Better schema comparison in tests; preparing byte conversion
2018-07-24 05:18:52 +02:00
Gunnar Morling
9814a5c82c
DBZ-803 Finalizing snapshot in offsets also for schema-only snapshots
2018-07-23 06:44:04 +02:00
Gunnar Morling
2905cc498b
DBZ-800 Don't expect that partition is a Map<String, String>
2018-07-20 13:37:54 +02:00
Gunnar Morling
fe53360f30
DBZ-800 Avoiding "mysql" in commonly used heartbeat key schema
2018-07-20 13:37:54 +02:00
Gunnar Morling
86a4731587
DBZ-800 Emitting heartbeat events in generic event dispatcher
2018-07-20 13:37:54 +02:00
Gunnar Morling
26106cdd8e
DBZ-800 Removing unused method from Heartbeat
2018-07-20 13:37:54 +02:00
Jiri Pechanec
0e35439019
DBZ-40 Snapshotting and connector restart
2018-07-18 14:39:11 +02:00
Gunnar Morling
a5b5e7d9eb
DBZ-627 Using unfified TopicSelector for MongoDB, too
2018-07-18 12:36:07 +02:00
Gunnar Morling
20db9299c5
DBZ-627 Unifying TopicSelector implementations
2018-07-18 12:36:07 +02:00
Gunnar Morling
610da70cc5
DBZ-801 Parameterizing ChangeEventQueue<DataChangeEvent>
2018-07-18 10:30:22 +02:00
Gunnar Morling
80085567ae
DBZ-720 Proper check for previous unfinished snapshot
2018-07-18 10:07:50 +02:00
Gunnar Morling
33170d4c3d
DBZ-40 Pulling up some common code to HistorizedRelationalDatabaseSchema
2018-07-18 10:07:50 +02:00
Gunnar Morling
422a303f87
DBZ-720 Signaling cancellation of snapshot through InterruptedException
2018-07-17 13:11:02 +02:00
Jiri Pechanec
f118e37da0
DBZ-40 Changes to support SQL Server connector
2018-07-13 22:23:00 +02:00
Gunnar Morling
c5e2d5f9e7
DBZ-720 Making scan of single table interruptable;
...
Also better logging.
2018-07-13 19:42:18 +02:00
Gunnar Morling
648d695c08
DBZ-720 Preparing HistorizedRelationalSnapshotChangeEventSource to support data snapshotting
2018-07-13 19:42:18 +02:00
Gunnar Morling
aa1272cc21
DBZ-720 Re-using single instance of streaming event receiver in EventDispatcher
2018-07-13 09:51:15 +02:00
Gunnar Morling
657650c581
DBZ-720 Passing change event emitter directly to dispatch methods;
...
The indirection of going through a supplier wasn't really necessary
2018-07-13 09:51:15 +02:00
Gunnar Morling
a565361e56
DBZ-720 Making SnapshotContext auto-closeable
2018-07-13 09:51:15 +02:00
Gunnar Morling
986df43649
DBZ-720 Extracting HistorizedRelationalSnapshotChangeEventSource from Oracle connector
2018-07-13 09:51:15 +02:00
Gunnar Morling
1523f230ca
DBZ-793 Using TableFilter consistently;
...
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8
[maven-release-plugin] prepare for next development iteration
2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f
[maven-release-plugin] prepare release v0.8.0.Final
2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d
[maven-release-plugin] prepare for next development iteration
2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0
[maven-release-plugin] prepare release v0.8.0.CR1
2018-07-04 07:07:44 +00:00
Gunnar Morling
99048365c9
DBZ-771 Unsetting default value when changing a column's type
2018-07-03 11:55:20 +02:00
Gunnar Morling
c7d5288d40
DBZ-771 Making sure causing DDL statement is logged also if default value can't be parsed
2018-07-03 11:55:20 +02:00
Gunnar Morling
dcd6b01134
DBZ-644 Simplifying RelationalDatabaseSchema constructor;
...
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
2018-07-03 07:03:59 +02:00
Gunnar Morling
b0980b994c
DBZ-644 Adding test for Postgres
2018-07-03 07:03:59 +02:00
Gunnar Morling
068aa85bd6
DBZ-644 Misc. clean-up and tests;
...
* Using parameter keys under "__debezium" namespace
* More expressive names
* Adding tests
2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a
DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema
2018-07-03 07:03:59 +02:00
orr.ganani
ac515b8064
DBZ-644 change Original Data Type mapper and add a test
2018-07-03 07:03:59 +02:00
orr.ganani
747184c572
DBZ-644 add JDBC data type as additional metadata to DBZ events, if configuration is opted in
2018-07-03 07:03:59 +02:00
Gunnar Morling
c4c8cbc3ab
DBZ-773 Moving management of Tables to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab
DBZ-773 Moving management of table schemas to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
a292b05a96
DBZ-751 Propagating DECIMAL column precision to Avro schemas
2018-06-27 14:48:59 +02:00
Gunnar Morling
b27421ddd0
DBZ-759 Testing via SchemaUtil;
...
* Making RecordWriter private
* Removing quotes around string representation of byte arrays
2018-06-26 12:03:53 +02:00
Andreas Bergmeier
82bac78ba8
DBZ-759 Add Test to ensure correct serialization of byte arrays
...
Needed to make RecordWriter public to have access in tests.
2018-06-26 11:50:54 +02:00
Andreas Bergmeier
71abca2afb
DBZ-759 Fix unusable array information in Serialization
...
According to ErrorProne, (implicitly) calling toString on the array does
not give useful information.
Also applying Base64 encoding is strange here.
2018-06-26 11:50:54 +02:00
Gunnar Morling
8d0d35762e
DBZ-759 Removing superfluous (and broken) ColumnEditorImpl#compareTo() implementation
2018-06-25 17:04:45 +02:00
Gunnar Morling
bf7a5018ca
Setting POM version back to 0.8.0-SNAPSHOT
2018-06-22 12:21:03 +02:00
Gunnar Morling
357102158a
DBZ-759 Fixing broken Document#setArray() method
2018-06-22 11:03:00 +02:00
Gunnar Morling
ce81f37990
DBZ-759 Adding unit test
2018-06-22 09:19:45 +02:00
Andreas Bergmeier
7417baea4d
DBZ-759 Fix equality comparison for BinaryValue
...
If value is binary or string, it should be better to compare the content
of the actual byte arrays.
According to ErrorProne you use reference equality when calling equals on
an array.
Part of https://issues.jboss.org/browse/DBZ-759
2018-06-22 09:19:45 +02:00
Andreas Bergmeier
333d6cb57d
DBZ-759 Fix hashCode of BinaryValue
...
According to ErrorProne, the hashCode of a byte array returns useless
information.
Part of https://issues.jboss.org/browse/DBZ-759
2018-06-22 09:19:41 +02:00
Andreas Bergmeier
d374ca0464
DBZ-759 Fix ColumnImpl.scale equality check
...
According to ErrorProne, checking Optional via operator == leads to
reference equality check. Probably want rather to check for contained
values.
2018-06-22 09:08:27 +02:00
Jenkins user
db42e4657a
[maven-release-plugin] prepare for next development iteration
2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99
[maven-release-plugin] prepare release v0.8.0.Beta1
2018-06-21 14:07:45 +00:00
Gunnar Morling
96c59a1568
DBZ-20 Moving debezium-connector-oracle from main to incubator repo
2018-06-20 13:05:37 +02:00
Gunnar Morling
b83f1493a2
DBZ-20 Avoiding Optional as method parameter
2018-06-20 13:05:37 +02:00
Gunnar Morling
dce1aa29f1
DBZ-20 Oracle precision supports negative numbers
2018-06-20 13:05:37 +02:00
Gunnar Morling
f4e3668ebf
DBZ-20 Tune up timeouts/connector/oracle/OracleStreamingChangeEventSource.java
2018-06-20 13:05:37 +02:00
Gunnar Morling
cf6219ccda
DBZ-20 Committing processed offsets in the DB, allowing log files to be released
2018-06-20 13:05:37 +02:00
Jiri Pechanec
661c34fc6b
DBZ-20 Use fluent API in ColumnEditor
2018-06-20 13:05:37 +02:00
Jiri Pechanec
232a1d5573
DBZ-20 Support for day to second interval
2018-06-20 13:05:37 +02:00
Jiri Pechanec
89694ffbbf
DBZ-20 Support for year to month interval
2018-06-20 13:05:37 +02:00
Jiri Pechanec
28b844bf99
DBZ-20 Added ANSI double precision, real and variable scale number datatypes
2018-06-20 13:05:37 +02:00
Jiri Pechanec
10ddb69dea
DBZ-20 A subset of string, time and number datatypes
2018-06-20 13:05:37 +02:00
Gunnar Morling
603f02e70d
DBZ-20 Excluding generated sources from CheckStyle
2018-06-20 13:05:37 +02:00
Gunnar Morling
e8614c7890
DBZ-20 Adding table whitelist/blacklist connector options;
...
* Restructuring OracleSnapshotChangeEventSource into smaller methods
2018-06-20 13:05:37 +02:00
Gunnar Morling
1e87fae023
DBZ-20 Recovering database history
2018-06-20 13:05:37 +02:00
Gunnar Morling
0733110696
DBZ-20 Recording schema history;
...
This records the DDL for DDL events captured during streaming. For the
initial schema snapshot, a JSON-style representation of the captured
Table objects is used in a new field of HistoryRecord, as the DDL
returned by dbms_metadata.get_ddl() isn't fully parseable by our
grammar.
2018-06-20 13:05:37 +02:00
Gunnar Morling
1a7786d254
DBZ-20 Further simplifying threading model and stopping logic
2018-06-20 13:05:37 +02:00
Gunnar Morling
d8a9865e51
DBZ-20 Some more docs
2018-06-20 13:05:37 +02:00
Gunnar Morling
225c0f05a6
DBZ-20 Extracting RelationalChangeRecordEmitter base class
2018-06-20 13:05:37 +02:00
Gunnar Morling
979c60d956
DBZ-20 Handling update events
2018-06-20 13:05:37 +02:00
Gunnar Morling
24e22f4985
DBZ-20 Simpler and more robust interruption handling in ChangeEventSourceCoordinator
2018-06-20 13:05:37 +02:00
Gunnar Morling
d7e196a18e
DBZ-20 Initial import of Oracle connector based on XStream
2018-06-20 13:05:37 +02:00
Gunnar Morling
9eb4b90ec9
DBZ-191 Adding record validations
2018-06-15 20:23:17 +02:00
Gunnar Morling
84dd36df46
DBZ-252 Misc typo fixes and clean-up
2018-06-15 16:43:30 +02:00
Gunnar Morling
54ca30624d
DBZ-252 Misc. improvements;
...
* Dedicated getter for DDL mode
* Using Objects#equal()
* Typo fixes
2018-06-15 16:43:29 +02:00
Jiri Pechanec
9580c8c290
DBZ-252 Rebase to master
2018-06-15 11:42:24 +02:00
rkuchar
7d7f740721
DBZ-252 refactor parser listener
2018-06-15 11:42:24 +02:00
rkuchar
5c9e718436
DBZ-252 uncomment getting ddl parser instance from configuration + improve/refactor dataType resolver for ANTLR
2018-06-15 11:42:24 +02:00
rkuchar
99eac9a8e2
DBZ-252 some test fixes + separate ddl test for antlr parser
2018-06-15 11:42:24 +02:00
rkuchar
22151250fa
DBZ-252 refactor proxy listener delegation
2018-06-15 11:42:24 +02:00
rkuchar
07212043b5
DBZ-252 test fixes + grammar fixes + some bug repairs in antlr parser
2018-06-15 11:42:23 +02:00
rkuchar
1124f81d94
DBZ-252 parse set and use statements
2018-06-15 11:42:23 +02:00
rkuchar
b4480c55b7
DBZ-252 parse database queries (create, alter, delete), parse name tokens without quotes, add new Mysql connector configuration field for choosing ddl parsing mode
2018-06-15 11:42:23 +02:00
rkuchar
5d911d511f
DBZ-252 signal event for truncate table into schema changes
2018-06-15 11:42:23 +02:00
rkuchar
32d181480d
DBZ-252 fix troubles with generalized system variables + add new data types that was missing in grammar
2018-06-15 11:42:23 +02:00
rkuchar
85e8146d1f
DBZ-252 implementation of scope synonym for local and session scope in MySQL
2018-06-15 11:42:23 +02:00
rkuchar
5851d1f07d
DBZ-252 fix initialization of system variables object + add default scope for variales in case that scope will no be set
2018-06-15 11:42:23 +02:00
rkuchar
9311c82bc5
DBZ-252 reformat code to not have anything after closing "}"
2018-06-15 11:42:23 +02:00
rkuchar
782ab75160
DBZ-252 move listener support from base to only legacy implementation, refactor system variables for possible use with other DBMSs + introduce data type resolver for antlr mysql parser
2018-06-15 11:42:23 +02:00
rkuchar
b92e8ef760
DBZ-252 exclude logging of parsed comment
2018-06-15 11:42:23 +02:00
rkuchar
30e158aa71
DBZ-252 create table and drop table listeners
2018-06-15 11:42:23 +02:00
rkuchar
785c346bcb
DBZ-252 Exclude reusable logic from LegacyDdlParser
2018-06-15 11:42:23 +02:00
rkuchar
36e2f33bf6
DBZ-252 - extract creation of DdlParser interface
2018-06-15 11:42:23 +02:00
Jiri Pechanec
29f8891b4f
DBZ-578 Process timestamps without timezones as in server timezone
2018-06-11 15:27:08 +02:00
Jiri Pechanec
de0e1ad4b5
DBZ-702 Mask SASL configuration
2018-06-08 08:57:50 +02:00
Jiri Pechanec
176dcbbc0b
DBZ-709 PostgreSQL uses scale for time precision not length
2018-06-05 15:51:35 +02:00
luobo
5b33d81f00
DBZ-191 support default value of column in mysql
2018-06-04 10:43:53 +02:00
Gunnar Morling
61f576bb90
DBZ-706 Misc. clean-up
2018-06-01 11:23:30 +02:00
Stephen Powis
40d872f48c
DBZ-706 General cleanup
2018-06-01 11:23:30 +02:00
Stephen Powis
867202e0ca
DBZ-706 Slight refactoring / shuffling of code to validate source query field in snapshot tests
2018-06-01 11:23:12 +02:00
Jiri Pechanec
764a1d0d58
DBZ-693 SQL parsing in separate method
2018-05-28 12:35:47 +02:00
Jiri Pechanec
45bc7b1536
DBZ-693 On connect statements supported for PostgreSQL
2018-05-28 12:35:47 +02:00
Jiri Pechanec
f74a9fe059
DBZ-693 Initial SQL statements
2018-05-28 12:35:47 +02:00
Jiri Pechanec
f844054f50
DBZ-696 Support nanoseconds in timestamp data
2018-05-22 11:33:50 +02:00
Jiri Pechanec
09b28c0913
DBZ-694 Support milliseconds for pre-epoch dates
2018-05-22 11:22:06 +02:00
Rao
15a90548fb
DBZ-666 Supporting ordered snapshot using tables.whitelist config
2018-05-17 06:23:31 +02:00
Jiri Pechanec
0f0a5d4cd4
DBZ-687 Kafka 1.1.0
2018-05-14 09:37:33 +02:00
Omar Al-Safi
8ef9267808
DBZ-668 Added unit test and changed the key field name
2018-03-23 10:50:42 +01:00
Omar Al-Safi
21d2e0b8a9
DBZ-668 Changed the key schema for the heartebat messages from STRING to STRUCT
2018-03-23 10:50:42 +01:00
Jiri Pechanec
2b5370de1a
DBZ-667 Sceintific numbers parsing
2018-03-20 14:42:55 +01:00
Gunnar Morling
00b63bb808
DBZ-669 Closing AdminClient instance after usage
2018-03-20 13:07:42 +01:00
Jenkins user
f4e151b23a
[maven-release-plugin] prepare for next development iteration
2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332
[maven-release-plugin] prepare release v0.7.5
2018-03-20 08:14:19 +00:00
Gunnar Morling
5f61cb167b
DBZ-655 Using AssertJ asserts in UnwrapFromEnvelopTest
2018-03-19 09:28:30 +01:00
Gunnar Morling
5358167446
DBZ-655 Adding test
2018-03-19 09:28:30 +01:00
Maciej Bryński
d5abb8415c
DBZ-655 Use cache to avoid using regexp replace
2018-03-19 09:28:23 +01:00
Gunnar Morling
d73c27d9bd
DBZ-657 Adding test
2018-03-18 18:26:27 +01:00
Jiri Pechanec
87bd3e9c2f
DBZ-657 Handle JSON with control chars not according to specs
2018-03-18 18:03:56 +01:00
Jiri Pechanec
9884a5ca1d
DBZ-663 disable cleaning of history topic based on size
2018-03-16 16:14:12 +01:00
Jiri Pechanec
62c7fe12f9
Do not log FLUSH RELAY LOGS statements
2018-03-16 09:51:48 +01:00
Jenkins user
daf27207be
[maven-release-plugin] prepare for next development iteration
2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928
[maven-release-plugin] prepare release v0.7.4
2018-03-07 08:31:07 +00:00
Jiri Pechanec
327085164e
DBZ-611 Special values supported for double
2018-03-06 16:55:24 +01:00
Gunnar Morling
1f385c21f9
DBZ-611 Typo fix and formatting
2018-03-06 16:55:24 +01:00
Jiri Pechanec
424aefbbb4
DBZ-611 Decimal-> String supported for MySQL
2018-03-06 16:55:24 +01:00
Gunnar Morling
401cbae852
DBZ-632 Removing methods from HistoryRecord which where only used in tests;
...
Also adding a new test for HistoryRecord (de-)serialization
2018-03-02 05:28:16 +01:00
Gunnar Morling
d040ec4427
DBZ-351, DBZ-606 Misc. clean-up;
...
* Renaming DebeziumDecimal to SpecialValueDecimal
* Simplifying logic in PostgresValueConverter
2018-02-27 16:36:21 +01:00
Jiri Pechanec
2534696b21
DBZ-606 Exception thrown for special values when mode handling is not string
2018-02-27 16:36:21 +01:00
Jiri Pechanec
5827a7ac1b
DBZ-611 Decimals can be encoded as strings
2018-02-27 16:36:21 +01:00
Jiri Pechanec
c71707afc9
DBZ-351 wal2json usses unlimited precision too
2018-02-27 16:36:21 +01:00
Jiri Pechanec
0c0ac72568
DBZ-606 Type for handling special BigDecimals
2018-02-27 16:36:21 +01:00
Jiri Pechanec
ceaa7f8efe
DBZ-351 Numeric is passed with unlimited precision
2018-02-27 16:36:21 +01:00
Gunnar Morling
d89f4e91c6
DBZ-632 Removing unused method parameter
2018-02-23 21:54:41 +01:00
Gunnar Morling
eef1ad7c2a
DBZ-630 Refactoring around MongoDB task context;
...
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8
DBZ-630 Pulling up getClock() to ConnectorTaskContext
2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6
DBZ-630 Unifying common start-up logic across connectors
2018-02-21 12:04:41 +01:00
Gunnar Morling
7fe45db831
DBZ-626 Renaming AvroValidator to SchemaNameAdjuster;
...
Making apparent that this functionality doesn't solely validate but returns a new name actually
2018-02-20 09:26:47 +01:00
Gunnar Morling
cddd5fcc8a
DBZ-626 Passing schema validator as AvroValidator instance instead of Function
2018-02-20 09:26:47 +01:00
Jiri Pechanec
6e00924ee9
DBZ-625 Fix parsing of float without decimals
2018-02-20 08:47:43 +01:00
Gunnar Morling
b5856f37c7
DBZ-628 Unrelated typo fix
2018-02-20 06:34:42 +01:00
Gunnar Morling
8369f10d52
DBZ-628 Adding MAX_QUEUE_SIZE, MAX_BATCH_SIZE and POLL_INTERVAL_MS to CommonConnectorConfig;
...
Also using these options in PG connector
2018-02-20 06:34:42 +01:00
Gunnar Morling
5bc62dc862
DBZ-620 Removing method only used in tests
2018-02-19 12:18:37 +01:00
Gunnar Morling
ab11f48843
DBZ-620 Creating envelope schemas only once per table type
2018-02-19 12:18:37 +01:00
Jenkins user
6d0cd88e12
[maven-release-plugin] prepare for next development iteration
2018-02-15 04:15:34 +00:00
Jenkins user
7d1e1a989e
[maven-release-plugin] prepare release v0.7.3
2018-02-15 04:15:34 +00:00
Gunnar Morling
b1362f9db9
DBZ-278 Extracting broker config retrieval into its own method
2018-02-13 17:12:11 +01:00
Jiri Pechanec
90e4aa0dea
DBZ-278 Fail if topic is not created
2018-02-13 16:48:42 +01:00
Jiri Pechanec
4541fc53c9
DBZ-278 Explicit history storage initialization
2018-02-13 16:48:42 +01:00
Jiri Pechanec
ad7b92310e
DBZ-278 Test for broker without auto-created topics
2018-02-13 16:48:42 +01:00
Jiri Pechanec
55e81c458b
DBZ-278 Create database history topic
2018-02-13 16:48:42 +01:00
Jiri Pechanec
5a0c7c07b0
DBZ-609 DBZ-571 Refactor Postgres type handling
...
TypeRegistry introduced for Postgres connector
JDBC column does not have a special componentType
JDBC column provide a database specific type id
OID is a primary type identifier to be used in Postgres connector code - dropping JDB/OID dichotomy
2018-02-13 14:54:42 +01:00
Jiri Pechanec
dbac1429b2
DBZ-581 Improve error handling for replicators
2018-02-13 14:40:54 +01:00
Jiri Pechanec
ecdce6529d
DBZ-220 Null Heartbeat object. Messages contains key
2018-02-13 12:26:48 +01:00
Jiri Pechanec
682cabb84a
DBZ-220 Generate heartbeat events in binlog thread
2018-02-13 12:24:51 +01:00
Gunnar Morling
96b4023839
DBZ-220 Misc. improvements;
...
* interpreting heartbeat interval as MS
* adding Configuration#getDuration()
2018-02-13 12:24:51 +01:00
Jiri Pechanec
3c1ea8206b
DBZ-220 Refactor heartbeat to a separate class
2018-02-13 12:24:51 +01:00
Gunnar Morling
2e0b9e5e33
DBZ-582 Moving TOMBSTONES_ON_DELETE to CommonsConnectorConfig for re-use
2018-02-13 10:19:18 +01:00
Jiri Pechanec
d0bd1560ff
DBZ-576 Make DDL filter internal parameter
2018-02-12 11:22:28 +01:00
Jiri Pechanec
3740aee4e8
DBZ-576 Internal parameters
2018-02-12 11:22:28 +01:00
Gunnar Morling
a77c8d733a
DBZ-580, DBZ-586 Making enqueue() and thus snapshots in Postgres interruptable;
...
* ChangeEventQueue#enqueue() checks the interrupted state of the calling
thread now, raising an InterruptedException in case the interrupted flag
has been set (because the producer's thread executor has been stopped)
* RecordSnapshotProducer has been adjusted to check for the interrupted
regularly, aborting if it has been set
2018-02-11 10:05:06 +01:00
Gunnar Morling
45a9847d42
DBZ-580 Initial implementation of dedicated change message queue:
...
* To be used across connectors, unifying the enqueuing/polling logic and handling of related config options
* Using it for Postgres connector
2018-02-11 10:05:06 +01:00
Gunnar Morling
2a724d9611
DBZ-537 Misc. adjustments:
...
* Renaming ConfigurationHelper to Instantiator
* Doc improvements and typo fixes
* Bringing getInstance() methods into consistent order
* Raising exception instead of logging error if instantion fails
2018-02-09 15:45:57 +01:00
Jiri Pechanec
5be55ae8ff
DBZ-537 OffsetCommitPolicy supports Configuration constructor initialization
2018-02-09 15:19:48 +01:00
Gunnar Morling
7b6db00db7
DBZ-593 JavaDoc
2018-02-02 13:16:43 +01:00
Gunnar Morling
712824e43f
DBZ-593 Passing version explicitly from SourceInfos to base class instead of relying specific packaging structure
2018-02-02 13:16:43 +01:00
Jiri Pechanec
7ebee94169
DBZ-593 Add Debezium version to source in envelope
2018-02-02 13:16:43 +01:00
Gunnar Morling
3a281d1185
DBZ-595 Making sure resources are cleaned up when snapshotting fails;
...
* shutting down the snapshotting thread and the DB history producer client
if the connector is stopped while trying to write to the history topic
* reducing the time that KafkaProducer#send() will block if Kafka isn't
available; this will release the producer thread quicker in case the
connector is stopped during snapshotting
* not returning from finally block (!) in case the TX is rolled back; This
prevented the failed state to be set by the outer catch clause in execute()
2018-02-02 09:06:32 +01:00
Jiri Pechanec
92740a3626
DBZ-587 Fix thread leak, thread names
2018-02-01 10:04:20 +01:00
Jiri Pechanec
9b592204ac
DBZ-587 Centralize and unify thread management
2018-02-01 10:04:20 +01:00
Jenkins user
04624341f5
[maven-release-plugin] prepare for next development iteration
2018-01-25 09:39:44 +00:00
Jenkins user
898f6884e1
[maven-release-plugin] prepare release v0.7.2
2018-01-25 09:39:44 +00:00
Gunnar Morling
8edcf9f3d8
DBZ-507 Making "wkb" optional in Point schema to keep compatability with previous version
2018-01-24 16:51:54 +01:00
Robert Coup
f9d90a482f
DBZ-507 Expanding support for geometry types;
...
* Adding support for PostGIS geometry types
* Adding support for GEOMETRY, POLYGON and more in MySQL
* For all newly supported types, changes are represented using two new schema types Geometry and Geography, containing the WKB (binary geo data) ans srid (coord system identifier)
* The existing Point type also contains the new (optional) srid field
2018-01-24 16:51:49 +01:00
Jiri Pechanec
01577b40c3
DBZ-541 Only whitelisted tables are recorded in history
2018-01-24 14:43:57 +01:00
Gunnar Morling
ea4b366b7f
DBZ-538 Capturing unparseable DDL for logging in one more place
2018-01-24 11:30:51 +01:00
Peter Goransson
7d78357087
DBZ-443 Adding db-history recovery mode: schema_only_recovery
2018-01-23 09:17:39 +01:00
Gunnar Morling
0c4190c493
DBZ-516 Using Duration instead of long in a few more places
2018-01-18 14:13:58 +01:00
Jiri Pechanec
24bdcaf059
DBZ-516 Return control to Connect periodically
2018-01-18 14:13:58 +01:00
Gunnar Morling
9f433b1e24
DBZ-544 Comment fix
2018-01-17 15:16:37 +01:00
Jiri Pechanec
cdf44faba4
DBZ-544 Log matching DDL filter
2018-01-17 15:16:37 +01:00
Gunnar Morling
4d1d016acb
DBZ-392 Making table map and set dedicated classes just with the needed API
2018-01-17 11:21:02 +01:00
Jiri Pechanec
9f0f0fe5d4
DBZ-392 Support for MySQL running on lowercase filesystems
2018-01-17 11:21:02 +01:00
Gunnar Morling
b99bdf7fdc
DBZ-543 Removing a few unused methods
2018-01-15 14:37:46 +01:00
Gunnar Morling
5c88431c07
DBZ-494 Making tests more lenient towards specific List implementations;
...
also fixing a few typos.
2018-01-15 10:40:50 +01:00
Tom Bentley
5b839d3665
DBZ-540 Adding KafkaCluster#zkPort()
...
If you start a cluster (e.g. in a test) without specifying a port
you get a random port. Sometimes you might want to connect to the
embedded zookeeper instance (for instance, to make an assertion about
a znode). To do this you need to know the port number. So let's expose it.
2018-01-12 18:25:41 +01:00
Jiri Pechanec
0b269a6e41
DBZ-538 Improve invalid DDL statement reporting
2018-01-12 12:26:30 +01:00
Jiri Pechanec
2a7377a833
DBZ-455 Use valueOf instead of constructors
2018-01-05 02:32:37 +01:00
Jenkins user
6bb34b42f9
[maven-release-plugin] prepare for next development iteration
2017-12-20 07:15:12 +00:00
Jenkins user
16dcd4c980
[maven-release-plugin] prepare release v0.7.1
2017-12-20 07:15:12 +00:00
Jenkins user
5e09932cb9
[maven-release-plugin] prepare for next development iteration
2017-12-15 05:10:23 +00:00
Jenkins user
6c1d61e03b
[maven-release-plugin] prepare release v0.7.0
2017-12-15 05:10:23 +00:00
Gunnar Morling
4e8cedd094
DBZ-379 Postgres connector minimizes use of JDBC metadata
2017-12-13 12:20:37 +01:00
Jiri Pechanec
5ae236241b
DBZ-469 Fixed a backslash regex in testcase
2017-12-11 16:58:23 +01:00
Gunnar Morling
8e99dc3abd
DBZ-469 Adding one more test case
2017-12-11 16:58:23 +01:00
Gunnar Morling
1c55e41941
DBZ-469 Renaming listOfRegex() to setOfRegex();
...
* Simplifying test of that method
* Adding test to ensure correctness of default DDL pattern
2017-12-11 16:58:23 +01:00
Jiri Pechanec
86d9e109fc
DBZ-469 Filter our RDS heartbeat INSERT statements
2017-12-11 16:58:23 +01:00
rkerner
c7ac481c43
[DBZ-342] fix broken MySQL data type "TIME" handling
2017-11-29 20:34:12 +01:00
Gunnar Morling
a55227aa83
DBZ-464 Don't stop after reaching max retry count, but raise an exception instead;
...
Also increasing default value, as the connector can't start its work without fully recovering the history
2017-11-28 08:47:27 +01:00
Gunnar Morling
6537d904ce
DBZ-464 Reading until end offset of history topic
2017-11-28 08:47:27 +01:00
Gunnar Morling
bc2d0e5956
DBZ-464 Removing unused method parameters from AbstractDatabaseHistory#recoverRecords()
2017-11-28 08:47:27 +01:00
Jiri Pechanec
20a2cdfdea
DBZ-476 Doubled quotes are parsed as escaped
2017-11-23 14:51:51 +01:00
Jiri Pechanec
57e7f84163
DBZ-479 Forced fsync slows down tests
2017-11-22 14:34:59 +01:00
Gunnar Morling
2b3276be1d
DBZ-478 Correctly handling null value converters;
...
* using simple for loop for the sake of easier debugging
* log info about unsupported column type on WARN rather than TRACE level
2017-11-22 09:43:10 +01:00
David Szabo
1c07ff4775
DBZ-466 Remove hardcoding of schema version number, leaving it empty instead
2017-11-16 17:43:14 +01:00
Jiri Pechanec
4d253d2987
DBZ-437 String tokens and SQL words are handled separately in procedure parsing
2017-11-14 09:40:54 +01:00
Jiri Pechanec
5e7714eb00
DBZ-456 Reduce text execution time
2017-11-14 06:47:32 +01:00
Henryk Konsek
be40502cc7
DBZ-323 Cluster and server properties should be added into config, not set as default ones
2017-11-13 16:35:57 +01:00
Gunnar Morling
2744f54e9d
DBZ-438 Renaming BufferedBlockingConsumer#flush() to close() to make clear it's onyl meant to be called once at the end
2017-11-13 14:19:39 +01:00
Gunnar Morling
2b62943e4f
DBZ-438 Avoiding issues due to concurrent usage of BufferedBlockingConsumer
2017-11-13 14:19:39 +01:00
Gunnar Morling
5fbe742be8
DBZ-285 Specifying scope of dependencies in the individual POMs for the sake of comprehensibility
2017-11-10 16:48:32 +01:00
Gunnar Morling
580647b226
DBZ-285 Making more dependencies "provided"
2017-11-10 16:33:02 +01:00
Ewen Cheslack-Postava
8826669b43
DBZ-285: Use provided or test dependencies for Connect and Kafka dependencies
2017-11-04 12:01:24 -07:00
Jiri Pechanec
a6bd883857
DBZ-432 Rebased to Kafka 1.0.0
2017-11-03 11:06:18 +01:00
Jiri Pechanec
e841cbd609
DBZ-429 Fixed signed default value handling
2017-11-03 06:13:05 +01:00
Gunnar Morling
3f410c20e5
DBZ-422 Using existing constants BigDecimal.ONE and ZERO;
...
Also using valueOf() methods to benefit from internal caching for most numeric types
2017-11-02 12:23:06 +01:00
pmx
3c91ad47ec
DBZ-422 Extracting numeric constants and boolean conversions into NumberConversions class
2017-11-02 12:12:54 +01:00
pmx
c3ce7c571a
DBZ-422 Making sure convertSmallInt() always returns a short
2017-11-02 12:12:54 +01:00
Jiri Pechanec
f93e1e9bcd
DBZ-395 Make temp table regex stricter
2017-10-26 11:11:22 +02:00
Jiri Pechanec
1c6e652c71
DBZ-395 Filter out DROP TEMPORARY TABLE statements
2017-10-26 11:11:22 +02:00
Ben Williams
a3b4fedd5f
DBZ-363 Add support for BIGINT UNSIGNED handling for MySQL
2017-10-18 10:20:03 +02:00
Gunnar Morling
73189892b3
DBZ-258 Misc. improvements while reviewing the change;
...
* Removing superfluous config option
* Making loggers static
* JavaDoc fixes
* Extracting hexStringToByteArray() to helper and adding test
* Removing superfluous super() invocations
2017-10-18 09:21:22 +02:00
Jiri Pechanec
e47b4cb81c
DBZ-258 Changes after first review
2017-10-18 09:21:22 +02:00
Jiri Pechanec
0bc8129961
DBZ-258 Support for wal2json plugin
2017-10-18 09:21:22 +02:00
Jenkins user
75937711fa
[maven-release-plugin] prepare for next development iteration
2017-09-21 04:42:02 +00:00
Jenkins user
a89b9332e4
[maven-release-plugin] prepare release v0.6.0
2017-09-21 04:42:02 +00:00
Gunnar Morling
7f20aca03d
DBZ-226 Minor wording and formatting improvements
2017-09-20 15:11:13 +02:00
Jiri Pechanec
0afb687303
DBZ-226 SMT to change CDC to simple record
2017-09-20 12:05:12 +02:00
Jiri Pechanec
ba3d7d762b
DBZ-318 Support for a Decimale with variable scale schema
2017-09-19 08:39:07 +02:00
Gunnar Morling
99f39038bb
DBZ-347 Formatting
2017-09-18 17:04:14 +02:00
Jiri Pechanec
61ccefcc5c
DBZ-347 Configurable behaviour for unknown DDL, failing by default
2017-09-18 17:02:21 +02:00
Jiri Pechanec
f7736cd0ab
DBZ-341 Ignore malformed messages coming from database history
2017-09-18 17:02:21 +02:00
Jiri Pechanec
01980cf29b
DBZ-341 Ignore malformed messages coming from database history
2017-09-15 11:52:18 +02:00
Jiri Pechanec
b53b4f67d2
DBZ-319 Views are now ignored in snapshots
2017-09-15 11:37:38 +02:00
Jiri Pechanec
e4bc6670c8
DBZ-305 Rebase build process against Kafka 0.11
2017-08-17 18:51:01 +02:00
Jenkins user
214696ef0c
[maven-release-plugin] prepare for next development iteration
2017-08-17 11:51:05 +00:00
Jenkins user
c867e6fea6
[maven-release-plugin] prepare release v0.5.2
2017-08-17 11:51:05 +00:00
Gunnar Morling
1665026c8a
DBZ-298 Adding support for quoted identifiers for schemas, tables and
...
columns; expanding test
2017-08-17 09:08:17 +02:00
Gunnar Morling
564d90dc09
DBZ-298 Formatting
2017-08-17 09:03:16 +02:00
Gunnar Morling
a4dc7620fe
DBZ-327 Simplifying validation implementation a bit via Strings#isNullOrEmpty()
2017-08-16 14:37:01 +02:00
Mario Mueller
2a2f911e74
DBZ-327 Fix broken SMT validation and add some tests for the
...
configuration combinations
2017-08-16 11:57:48 +02:00
Emrul Islam
bb79b2f60f
DBZ-297 Initial support for some Postgres array types (one-dimensional only)
2017-07-27 10:04:13 +02:00
Gunnar Morling
79fbc028a8
DBZ-311 Precompiling and simplifying some regular expressions
2017-07-25 21:43:26 +02:00
Gunnar Morling
825530256e
DBZ-311 Removing trailing whitespace
2017-07-25 21:43:26 +02:00
Gunnar Morling
a8d1817c22
[maven-release-plugin] prepare for next development iteration
2017-06-09 16:14:31 +00:00
Gunnar Morling
3f512aace7
[maven-release-plugin] prepare release v0.5.1
2017-06-09 16:14:31 +00:00
Gunnar Morling
2a22f3d4b0
DBZ-254 Right-padding values for fixed-length BINARY columns with 0x00 (zero byte) characters for MySQL;
...
Also fixing JDBC types for binary data types for MySQL.
2017-05-30 15:04:38 +02:00
Tom Bentley
96eda35da5
Respect any existing auto.create.topics.enable when creating a server
2017-05-23 16:58:12 +01:00
dleibovic
a683629519
Change ByLogicalTableRouter logging from info to debug
...
This can be a bit spammy in the info level
2017-05-19 12:09:40 -04:00
Randall Hauch
02e655fa53
DBZ-198 Add another DDL parser test case
...
Added an additional test that is unable to reproduce the problem reported on April 4.
2017-05-19 15:50:03 +02:00
Jiri Pechanec
ec915c559c
[DBZ-246] Correctly handle enumeratedvale type
2017-05-09 11:24:28 +02:00
Gunnar Morling
1158833524
DBZ-238 Removing some trailing whitespace
2017-05-05 21:59:06 +02:00
Gunnar Morling
64958d11db
DBZ-238 Moving EnumeratedValue to its own file
2017-05-05 21:55:46 +02:00
Brendan Maguire
928c9cb5f0
DBZ-238: Fix Postgres database.sslmode = "require" bug
...
* Also added EnumeratedValue interface for Configuration so getValue is used instead of toString when getting the config value
2017-05-05 21:55:45 +02:00
Gunnar Morling
1074de4efa
DBZ-222 Some more clean-up
2017-05-04 09:25:25 +02:00
Omar Al-Safi
ea092f1baf
DBZ-222 Add null to Point comment
2017-05-04 08:53:46 +02:00
Gunnar Morling
5630b61be6
DBZ-222 dependency clean-up
2017-05-04 08:53:05 +02:00
Omar Al-Safi
791545c5f4
DBZ-222 Added support for MySQL POINT type
2017-05-04 08:53:05 +02:00
Randall Hauch
d42736e065
DBZ-121 Corrected JavaDoc and reorganized imports
2017-04-04 16:53:35 -05:00
dleibovic
8e304c4cba
rename fields and simplify descriptions
2017-04-03 11:35:33 -04:00
dleibovic
fa157b0159
make the name of the physical table identifier field added to the key schema configurable
2017-03-31 13:09:32 -04:00
dleibovic
d7acda04c4
DBZ-121: routing to custom topic names via Kafka Connect Transformations.
2017-03-30 15:02:59 -04:00
Randall Hauch
709cd8f3fe
[maven-release-plugin] prepare for next development iteration
2017-03-27 11:28:12 -05:00
Randall Hauch
2bc3d45954
[maven-release-plugin] prepare release v0.5.0
2017-03-27 11:28:11 -05:00
Randall Hauch
81f62b6961
DBZ-205 Corrected MySQL connector to handle 2-digit years
...
MySQL has special handling of 2-digit years that it deems are ambiguous, such as the year value `17` that is actually treated as `2017`. Apparently the 2-digit values are stored in MySQL and the interpretation is performed when the data is extracted, so therefore the connector needs to also perform this adjustment of the year values. This commit uses the JDK’s `TemporalAdjuster` interface and passes this down to the requisite temporal-related datatype handling code. The MySQL connector then provides its own `TemporalAdjuster` implementation that adjusts the year values via the excellend JDK `Temporal` methods.
A row in one of the MySQL test databases was changed to use a 2-digit year of `16` while the test method still checks that the year is still 2016`, verifying that the year value is properly adjusted.
2017-03-27 10:58:21 -05:00
Randall Hauch
613e6b6340
Fixed compilation warnings
2017-03-27 10:37:15 -05:00
Randall Hauch
7a72ed6ae6
Merge pull request #202 from don41382/upgrade-kafka-version-to-0.10.2.0
...
DBZ-203 Upgrade kafka version from 0.10.1.1 to 0.10.2.0
2017-03-17 17:21:48 -05:00
Randall Hauch
430d756062
[maven-release-plugin] prepare for next development iteration
2017-03-17 15:41:58 -05:00
Randall Hauch
536cbf6300
[maven-release-plugin] prepare release v0.4.1
2017-03-17 15:41:57 -05:00
Randall Hauch
e5ee3847dd
DBZ-195 Added tests to try to replicate a reported issue
...
Added a table and inserted rows that tries to replicate the problem reported in DBZ-195, but the test was unable to replicate the problem. In fact, this really is no different than existing tests. Changed the log messages so that if/when this happens again it will be possible to know which row was problematic.
2017-03-17 10:47:32 -05:00
Felix Eckhardt
5d414c521a
upgraded kafka version from 0.10.1.1 to 0.10.2.0
2017-03-17 11:36:15 +11:00
Randall Hauch
cf5391482a
DBZ-198 Improved MySQL DDL parser to better handle blocks
...
The MySQL parser now properly handles control blocks such as `BEGIN…END`, `IF…END IF`, `REPEAT…END REPEAT`, and `LOOP…END LOOP`, even in cases where the block is preceded by and terminated by a label.
2017-03-16 13:32:21 -05:00
Randall Hauch
b48ccce4b5
DBZ-200 Corrected MySQL DDL parser to better handle column definitions
...
Apparently not all reserved words must be quoted when using them as colum names, so refactored MySQL’s DDL parser to better handle a variety of unquoted colum names that are reserved words.
2017-03-08 12:12:27 -06:00
Randall Hauch
d2986710a5
DBZ-188 More efficient GTID source filters for MySQL Connector
...
Changed the GTID source filters in the MySQL connector to be far more efficient when the filters specify literal UUIDs rather than regex patterns. In these cases, the predicate just checks whether a supplied value is in a hash set, and no regular expression patterns are used.
The GTID source filters can still be a combination of UUID literals and regular expressions, and the predicate will use the best implementation for each. For example, if the filters include all UUID literals, then regular expressions will never be used.
2017-02-10 11:34:24 -06:00
Randall Hauch
8c60c29883
[maven-release-plugin] prepare for next development iteration
2017-02-07 14:22:12 -06:00
Randall Hauch
20134286e9
[maven-release-plugin] prepare release v0.4.0
2017-02-07 14:22:11 -06:00
Randall Hauch
74e5ba6448
DBZ-176 Corrected MySQL DDL parser to support creating triggers with definers
...
The MySQL DDL parser was not correclty handling `DEFINER` clauses within `CREATE TRIGGER` or `CREATE EVENT` statements. Support for `DEFINER` clauses was recently added for the various forms of `CREATE PROCEDURE`, `CREATE FUNCTION` and `CREATE VIEW` statements. These are the only kinds of statements that have the definer attribute, per the [MySQL documentation](https://dev.mysql.com/doc/refman/5.7/en/stored-programs-security.html ).
2017-02-02 12:44:28 -06:00
Randall Hauch
972cfbe2c4
DBZ-173 Additional fixes to KafkaDatabaseHistory class for Kafka 0.10.1.0
...
The KafkaDatabaseHistory class was not behaving well in tests using my local development environment. When restoring from the persisted Kafka topic, the class would set up a Kafka consumer and see repeated messages. It is unclear whether the repeats were due to our test environment and very short poll timeouts. Regardless, the restore logic was refactored to track offsets so as to only process messages once.
2017-02-01 14:47:41 -06:00
Horia Chiorean
d035c4bc8d
DBZ-173 Changes the MySQL ITs to not use TZ information for expected dates and fixes the character set for parsing test files
2017-01-27 14:53:10 +02:00
Horia Chiorean
a2154d3d32
DBZ-173 Changes the MySQL ITs to use the database.hostname system property instead of always hardcoding 'localhost'
2017-01-27 09:19:57 +02:00
Horia Chiorean
7dfdef3558
DBZ-173 Upgrades the Kafka artifact versions to 0.10.1.1
2017-01-27 09:19:57 +02:00
Horia Chiorean
031c4a1552
DBZ-183 Fixes the BinlogReader's handling of TIMESTAMP columns to correctly account for timezones
2017-01-25 16:39:36 +02:00
Randall Hauch
f0db8d1b1f
DBZ-179 Corrected JavaDoc in PostgreSQL connector
...
Corrected the JavaDoc and removed trailing spaces in the PostgreSQL connector code.
2017-01-20 11:51:17 -06:00
Randall Hauch
a73f85a80f
Merge pull request #162 from rareddy/DBZ-177
...
DBZ-177: Providing an alternative way to create JDBC connection based …
2017-01-13 13:37:38 -06:00
Ramesh Reddy
a9aace3480
DBZ-177: Providing an alternative way to create JDBC connection based on the configured JDBC driver class name and supplied classloader. The loading/creating the JDBC connections is not reliable when driver libraries in a different classloader than the DriverManager.
2017-01-13 12:58:14 -06:00
Horia Chiorean
ae85656851
DBZ-3 Fixes topic naming to include the name of the server
2016-12-30 09:42:48 +02:00
Horia Chiorean
737614a555
DBZ-3 Implements a connector for streaming changes from a Postgres database
...
The version of the DB server required for this to work is at least 9.4. To be able to stream logical changes, the code relies on enhancements to the JDBC driver which are not yet public. Therefore, the current codebase includes the sources for the JDBC driver.
The commit also updates the general DBZ build system for:
* custom checkstyle package exclusions - required by the Postgres driver the protobuf code for now
* adds support for debugging Surefire and Failsafe
2016-12-27 14:44:32 +02:00
Horia Chiorean
23e3f59fa1
DBZ-3 Implements a connector for streaming changes from a Postgres database
...
The version of the DB server required for this to work is at least 9.4
The commit also updates the general DBZ build system for:
* custom checkstyle package exclusions - required by the Postgres driver the protobuf code for now
* adds support for debugging Surefire and Failsafe
2016-12-27 14:44:32 +02:00
Horia Chiorean
8e14f150db
DBZ-3 Adds the structure for a Postgres connector which uses a Debezium Postgres docker image that has the decoderbufs plugin enabled to read WAL changes
2016-12-27 14:44:29 +02:00
Randall Hauch
5dceb05f69
DBZ-151 Additional changes to improve test framework and MySQL integration tests
2016-12-20 10:58:56 -06:00
Randall Hauch
a3bece4472
DBZ-151 Added new integration test framework for easily comparing output of connectors to expected results.
2016-12-20 09:18:09 -06:00
Randall Hauch
0851d8280c
DBZ-166 Corrected shutdown logic of MySQL connector
...
The MySQL connector uses several threads, so previously upon connector shutdown these threads were simply cancelled. This is fine for the binlog reader (which can stop at any moment), but is a poor approach for the snapshot as we didn’t always properly release the database resources and also didn’t complete the writing of the DDL history.
With this change, the snapshot reader stops in a very controlled manner, basically by having the 10-step snapshot procedure frequently check whether the reader is to continue working, and to completely avoid thread interruption altogether. And, the snapshot procedure will always clean up its database resources (locks, transactions, etc.), even if the procedure is stopped before completion.
This change also refactors how the snapshot and binlog reader are managed. This is no longer done in the MySqlConnectorTask class (which is busy enough), but rather the logic has been encapsulated in a new `ChainedReader` that makes use of a new `Reader` interface. This makes testing of `ChainedReader` easier, and ensure that `ChainedReader` relies only upon the primary methods of `Reader` rather than upon `AbstractReader`. `ChainedReader` handles multiple readers generically, and ensures that when stopped the readers are all handled correctly and completely process all records, yet avoid accidentally starting a subsequent reader(s) when stopping the previous reader.
2016-12-15 10:55:18 -06:00