Jenkins user
f1ae79ff73
[maven-release-plugin] prepare release v0.9.0.CR1
2019-01-28 11:07:04 +00:00
Gunnar Morling
ccff79e07d
DBZ-1105 Removing TableImpl#columnNames field
2019-01-28 10:23:56 +01:00
Gunnar Morling
c9c84afc9d
DBZ-1099 Don't enque further records after request to stop
2019-01-25 10:19:19 +01:00
Gunnar Morling
95a13b5dd7
DBZ-1062 Fix MySQL trustStore password setting
2019-01-24 17:38:42 +01:00
Moira Tagle
01c779fb62
DBZ-175 Setting completed GTIDs in the final binlog reader context in setupUnifiedReader()
2019-01-24 10:43:44 +01:00
Moira Tagle
0e10c4af48
DBZ-175 Modifying BinlogReaderMetrics to take in the BinlogReader name
2019-01-24 10:43:44 +01:00
Gunnar Morling
884f2f40ea
DBZ-175 Adding toString() for readers
2019-01-24 10:43:44 +01:00
Gunnar Morling
206cd06c24
DBZ-175 Using passed filters instead of rebuilding them in MySqlSchema
2019-01-24 10:43:44 +01:00
Gunnar Morling
ed4787bc8a
DBZ-175 Misc. clean-up;
...
* Removing unused code (constructor, method and return value)
* Making ParallelSnapshotReader logger static
* Making RecordMakers#restartOffset final; avoiding unchecked cast
2019-01-24 10:43:44 +01:00
Gunnar Morling
965b789dab
DBZ-175 Misc. adjustments;
...
* Dedicated functional interface for halting predicates
* Using semantic temporal types
* Typo fixes
* Exposing SnapshotNewTables through connector config
* Static loggers
* Adding Moira to COPYRIGHT.txt
2019-01-24 10:43:44 +01:00
Moira Tagle
a6d791cd8c
DBZ-175 Initial implementation for support to whitelist/blacklist changes;
...
* The filter config is stored in offsets now, allowing to detect changes
to the config after a connector restart
* In that case, roughly the following steps are done:
- set up a binlog reader for the "old" tables
- set up a snapshot reader for the "new" tables
- if the snapshot is done, set up a binlog reader for the new tables
- if both binlog readers are in proximity of the logs head, stop them
and set up a single binlog reader for all tables
* That behavior is disabled by default for the purposes of testing and
can be enabled via new connector option "snapshot.new.tables"
* To facilitate connector restarts while whitelist/blacklist changes are
processed, separate restart offsets are added to the offsets
2019-01-24 10:43:44 +01:00
Jiri Pechanec
a638d5599b
DBZ-1064 Use geometry instead of point for empty
2019-01-16 13:01:36 +01:00
Jiri Pechanec
ceb6c62721
DBZ-1064 Remove lazy fallback values
2019-01-16 13:01:36 +01:00
Jiri Pechanec
536e517c08
DBZ-1064 Geometry default value changes; fixes for default suppliers
2019-01-16 13:01:36 +01:00
Jiri Pechanec
bff7eea734
DBZ-1064 Use convert value template
2019-01-16 13:01:36 +01:00
Jiri Pechanec
920e0cbd3f
DBZ-1064 Set default value only for non-nullable fields
2019-01-16 13:01:36 +01:00
Amit Sela
b0b7d942a3
DBZ-1073 Adding scale as schema parameter, if present
2019-01-11 15:40:30 +01:00
Grzegorz Kołakowski
91c72738c0
DBZ-1067 Add column blacklist field to RelationalDatabaseConnectorConfig
2019-01-09 18:13:26 +01:00
Gagan Agrawal
aff45c7dc6
DBZ-1063 Fixed schema parsing bug for handling unsigned max default value
2019-01-09 17:50:18 +01:00
Jiri Pechanec
03570a1a3a
DBZ-1059 Support qualified table name in rename
2019-01-04 12:57:55 +01:00
Jenkins user
5275f73424
[maven-release-plugin] prepare for next development iteration
2018-12-19 13:06:12 +00:00
Jenkins user
b6569c18ae
[maven-release-plugin] prepare release v0.9.0.Beta2
2018-12-19 13:06:12 +00:00
Jiri Pechanec
9fcc25e4c5
DBZ-688 Uncommitted TX with temp tables are not stored in binlog in MySQL 8
2018-12-18 15:42:15 +01:00
LiuHanlin
c58031582d
DBZ-1033 Mysql binlog reader lost data if restart task when last binlog event is QUERY event.
...
https://issues.jboss.org/browse/DBZ-1033
Reset restartEventsToSkip each time updating the restartBinlogPosition.
2018-12-17 13:42:01 +01:00
Gunnar Morling
1258beae55
DBZ-688 Moving differences related solely to geo-spatial test into the test class itself
2018-12-14 10:39:05 +01:00
Jiri Pechanec
c8f09f110b
DBZ-688 Centralize MySQL differences in a single file
2018-12-14 10:39:05 +01:00
Jiri Pechanec
3c1da7d5d4
DBZ-688 Robust identification of geom collection type
2018-12-14 10:39:05 +01:00
Jiri Pechanec
12083452a5
DBZ-688 CURRNT_TIMESTAMP handled differently by MySQL 8
2018-12-14 10:39:05 +01:00
Jiri Pechanec
994f1b6e03
DBZ-688 Update geometry handling
2018-12-14 10:39:05 +01:00
Jiri Pechanec
8b83df82b5
DBZ-688 Ignore differences in generated metadata
2018-12-14 10:39:05 +01:00
Jiri Pechanec
974731b0ae
DBZ-688 Support for test execution on MySQL 8 server
2018-12-14 10:39:05 +01:00
Jiri Pechanec
edec1c3090
DBZ-978 Added last event and captured tables metrics
2018-12-13 20:52:11 +01:00
Grzegorz Kołakowski
2a1934eeb1
DBZ-966 Make snapshot.delay.ms global property
2018-12-13 14:29:54 +01:00
Gunnar Morling
0075bbb17e
DBZ-1028, DBZ-992 Expanding test
2018-12-12 23:05:29 +01:00
Jiri Pechanec
946d5c113d
DBZ-1028 Literal "CSV" is not a special token
2018-12-12 23:05:07 +01:00
Jiri Pechanec
f3a87a039f
DBZ-1008 Process first GTID after failover
2018-12-12 16:52:53 +01:00
Jiri Pechanec
edb116677d
DBZ-960 Handle TX closing safely
2018-12-11 14:15:53 +01:00
Jiri Pechanec
2b2ff0754d
DBZ-960 Run SnapshotReader in a single transaction
2018-12-11 14:15:53 +01:00
Gunnar Morling
68b6bc2efe
DBZ-977 Adding log statement
2018-12-11 14:12:23 +01:00
Jiri Pechanec
6a88309f0e
DBZ-977 ALTER TABLE filters works even for stored DDL
2018-12-11 14:12:23 +01:00
Jiri Pechanec
c5ecff4be6
DBZ-1005 Detect GTID by database setting not GTID set
2018-11-30 11:58:07 +01:00
Jiri Pechanec
415830003a
DBZ-1016 Fix expected records count
2018-11-30 11:18:47 +01:00
Jiri Pechanec
2c3d2ee083
DBZ-1010 Handle error deserialization also in deserializers
2018-11-30 11:09:30 +01:00
Jiri Pechanec
9c81e54b9d
DBZ-1006 Guard against NPE when start() fails
2018-11-23 16:13:50 +01:00
Gunnar Morling
62fe7d1ee7
DBZ-604 Making sure connector cleanly shuts down also if records are produced after polling loop has stopped
2018-11-23 14:36:05 +01:00
Gunnar Morling
2fd16a6bc0
DBZ-923 Misc. clean-up;
...
* Exposing new option via MySqlConnectorConfig
* Formatting
* Using GtidSet type in signatures
* Adding new option to configDef() and ALL_FIELDS
2018-11-22 10:54:10 +01:00
Eero Koplimets
972c08fcda
fix checkstyle errors
2018-11-22 09:50:46 +01:00
Eero Koplimets
3e1eaef318
fix GTID_NEW_CHANNEL_POSITION description
2018-11-22 09:50:46 +01:00
Eero Koplimets
268101acd5
Change config from boolean to enum
2018-11-22 09:50:46 +01:00
Eero Koplimets
8844442459
DBZ-923 config to use earliest available gtid set from mysql server when connecting
2018-11-22 09:50:46 +01:00
Jenkins user
db1d3a7fb8
[maven-release-plugin] prepare for next development iteration
2018-11-20 16:15:14 +00:00
Jenkins user
f83db82cea
[maven-release-plugin] prepare release v0.9.0.Beta1
2018-11-20 16:15:14 +00:00
Jiri Pechanec
b7e7bfa218
DBZ-990 Engine names can be quoted
2018-11-19 14:30:21 +01:00
Jiri Pechanec
530e2190a8
DBZ-757 Use antlr parser by default
2018-11-16 14:17:47 +01:00
Gunnar Morling
6bd4024618
DBZ-776 Misc. improvements;
...
* Renaming getTimeSinceLastEvent() to getMilliSecondsSinceLastEvent()
* Further unifying metrics implementation across connectors
* Emitting event in EventDispatcher also if event is filtered out
* Typo fixes
2018-11-16 08:52:07 +01:00
Gunnar Morling
818d5e4696
DBZ-776 Misc. clean-up
2018-11-16 08:52:07 +01:00
Jiri Pechanec
4510e2bbe6
DBZ-776 Cleanup after review
2018-11-16 08:52:07 +01:00
Gunnar Morling
360714a5a1
DBZ-776 Formatting and minor clean-up
2018-11-16 08:52:07 +01:00
Jiri Pechanec
b44c9fd04f
DBZ-776 A first set of metrics implemented in new framework
2018-11-16 08:52:07 +01:00
Jiri Pechanec
4cdc3e669b
DBZ-776 Extract metrics to a common code
2018-11-16 08:52:07 +01:00
Gunnar Morling
9994172860
DBZ-953 Pulling decimal mode handling up to RelationalDatabaseConnectorConfig;
...
Also using it in MySQL connector.
2018-11-08 09:23:28 +01:00
Jiri Pechanec
a5ff561deb
DBZ-959 ADD PARTITION needs parenthesis
2018-11-02 07:14:42 +01:00
s0r01ed
fdab3386d4
DBZ-780 | Adding RENAME column semantics to rename a column
2018-10-24 20:20:04 +02:00
Deepak Barr
bc0712e240
DBZ-935 Error while parsing json column type for MySQL
2018-10-11 11:40:18 +02:00
Jenkins user
d952f5dfb0
[maven-release-plugin] prepare for next development iteration
2018-10-04 11:59:14 +00:00
Jenkins user
ff9b70278b
[maven-release-plugin] prepare release v0.9.0.Alpha2
2018-10-04 11:59:13 +00:00
Deepak Barr
4aaed1fd2b
DBZ-927 Gracefully handling invalid default values for timestamp fields
2018-10-04 10:30:56 +02:00
jchipmunk
3ec0b3be81
DBZ-918 Moving connector identifier from SourceInfo to Module class
2018-10-04 09:00:13 +02:00
jchipmunk
05ba99bb0a
DBZ-918 Adding Debezium connector field to source info
...
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
2018-10-04 09:00:13 +02:00
Jiri Pechanec
3b51818864
DBZ-910 Verify also anonymous CONSTRAINT CHECK
2018-09-18 15:51:46 +02:00
Jiri Pechanec
3c5a67856b
DBZ-910 Improve support for CHECK constraint
2018-09-18 15:51:46 +02:00
Jiri Pechanec
6fe48b7764
DBZ-769 Try to release table locks during exception
2018-09-18 10:49:48 +02:00
jchipmunk
b0feaf496a
DBZ-633 Adding "field.blacklist" property for MongoDB connector
...
The "field.blacklist" configuration property is an optional comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form "databaseName.collectionName.fieldName.nestedFieldName", where "databaseName" and "collectionName" may contain the wildcard (*) which matches any characters.
Although the "field.blacklist" configuration property allows you to remove fields from the event values, the "_id" field is always included in the event’s key.
2018-09-14 12:54:11 +02:00
Jiri Pechanec
e6b28eeea4
DBZ-903 Support function named DATABASE()
2018-09-13 16:17:50 +02:00
Jiri Pechanec
8dd8327826
DBZ-901 Handle default date/time values without micros
2018-09-13 15:47:00 +02:00
Gunnar Morling
e1bbbd7992
DBZ-865 DBZ-878 Replacing invalid topic name characters with "_";
2018-09-05 12:56:53 +02:00
Cliff Wheadon
685f72f988
DBZ-880 MySQLConnectorTask no longer always reports finding the binlog
2018-08-28 08:56:52 +02:00
Jiri Pechanec
ce0b3cc940
DBZ-872 Use converters in parser tests
2018-08-24 08:12:54 +02:00
Jiri Pechanec
2cbc35dac7
DBZ-877 Process newly added columns
2018-08-24 08:12:54 +02:00
Gunnar Morling
985999b4af
DBZ-872 Making MySqlAntlrDdlParser#tableFilter final
2018-08-24 08:12:54 +02:00
Jiri Pechanec
63ee65bb7d
DBZ-872 Honor filters in ALTER TABLE parsing
2018-08-24 08:12:54 +02:00
Jiri Pechanec
29af79057b
DBZ-859 Test for MySQL heartbeat
2018-08-23 11:44:53 +02:00
Jiri Pechanec
985e03e7b5
DBZ-859 Force heartbeat after snapshot completion
2018-08-23 11:44:53 +02:00
Rao
49214491a7
DBZ-763 Correcting the JDBC driver class
2018-08-23 11:31:37 +02:00
Gunnar Morling
414ebe5f13
DBZ-870 Adding test
2018-08-23 11:19:27 +02:00
Jiri Pechanec
99ab131910
DBZ-870 Accept zero as valid date/time value
2018-08-23 11:19:27 +02:00
Jiri Pechanec
26b82c88bd
DBZ-860 DBZ-869 Enable default value tests for antlr parser
2018-08-23 11:19:27 +02:00
Jiri Pechanec
e9f9e4eb8f
DBZ-860 Fix for primary keys with default value
2018-08-23 11:19:27 +02:00
Jiri Pechanec
7e0639f733
DBZ-864 Add support for CHARSET database option
2018-08-22 13:31:34 +02:00
Gunnar Morling
207f1896ab
DBZ-763 Cleaning up tests
2018-08-22 11:54:55 +02:00
Rao
d13ea932f7
DBZ-763 Upgrading Mysql driver to version 8.0.12
2018-08-22 11:54:55 +02:00
Gunnar Morling
3862083756
DBZ-860 Expanding test
2018-08-20 19:01:55 +02:00
Jiri Pechanec
c2f014750e
DBZ-860 Handle implicitly NOT NULL PKs
2018-08-20 19:01:32 +02:00
PengLv
3d978d717b
fix gitd bug
...
method isGtidModeEnabled when to check gtid mode ,it'll always be true
2018-08-14 09:46:16 +02:00
artiship
bb4ecc79e0
DBZ-849 Refact read mysql system variables methods
2018-08-14 09:43:49 +02:00
artiship
cdb6d7301c
DBZ-855 Make BinlogReader.informAboutUnknownTableIfRequired log with tableId
2018-08-14 08:41:30 +02:00
Jenkins user
e00dad127f
[maven-release-plugin] prepare for next development iteration
2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700
[maven-release-plugin] prepare release v0.9.0.Alpha1
2018-07-26 08:00:12 +00:00
Gunnar Morling
3061c1dc0b
DBZ-815 Passing offset to Heartbeat#heartbeat();
...
That's simpler to grasp than the approach of passing a supplier lambda to the constructor.
Also it allows to pass on the offset via local variables instead of instance fields in some cases.
2018-07-25 14:49:52 +02:00
Jiri Pechanec
0e35439019
DBZ-40 Snapshotting and connector restart
2018-07-18 14:39:11 +02:00
Gunnar Morling
20db9299c5
DBZ-627 Unifying TopicSelector implementations
2018-07-18 12:36:07 +02:00
Gunnar Morling
1523f230ca
DBZ-793 Using TableFilter consistently;
...
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8
[maven-release-plugin] prepare for next development iteration
2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f
[maven-release-plugin] prepare release v0.8.0.Final
2018-07-11 07:36:29 +00:00
Peter Goransson
448330fd07
DBZ-789 Expose more granular snapshot metrics via JMX
2018-07-09 12:30:41 +02:00
Jenkins user
033db6659d
[maven-release-plugin] prepare for next development iteration
2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0
[maven-release-plugin] prepare release v0.8.0.CR1
2018-07-04 07:07:44 +00:00
Jiri Pechanec
d4c9d24b22
DBZ-779 Detect mongo replicator init completion
2018-07-03 12:24:06 +02:00
Gunnar Morling
9d52614b79
DBZ-771 Adding test for changing column type and default value
2018-07-03 11:55:20 +02:00
Gunnar Morling
99048365c9
DBZ-771 Unsetting default value when changing a column's type
2018-07-03 11:55:20 +02:00
Gunnar Morling
dcd6b01134
DBZ-644 Simplifying RelationalDatabaseSchema constructor;
...
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
2018-07-03 07:03:59 +02:00
Gunnar Morling
068aa85bd6
DBZ-644 Misc. clean-up and tests;
...
* Using parameter keys under "__debezium" namespace
* More expressive names
* Adding tests
2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a
DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema
2018-07-03 07:03:59 +02:00
orr.ganani
747184c572
DBZ-644 add JDBC data type as additional metadata to DBZ events, if configuration is opted in
2018-07-03 07:03:59 +02:00
Gunnar Morling
c4c8cbc3ab
DBZ-773 Moving management of Tables to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab
DBZ-773 Moving management of table schemas to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
90a453b0ee
DBZ-768 Checking boolean flag just once
2018-06-28 10:14:01 +02:00
Jiri Pechanec
b4b1f6976b
DBZ-768 Fix formatting. Behaviour not dependent on a timezone
2018-06-28 07:24:09 +02:00
Jiri Pechanec
741debe18e
DBZ-768 Default values with variable second fractions
2018-06-28 06:17:41 +02:00
Gunnar Morling
a3f42610ff
DBZ-751 Adding tests for NUMERIC columns
2018-06-27 14:48:59 +02:00
Gunnar Morling
0600ccf213
DBZ-751 Incorporating review remarks;
...
* Expanding test
* Extracting constant for parameter key in tests
2018-06-27 14:48:59 +02:00
Gunnar Morling
a292b05a96
DBZ-751 Propagating DECIMAL column precision to Avro schemas
2018-06-27 14:48:59 +02:00
Jiri Pechanec
d61db8930d
DBZ-770 Typename comparison should not be case dependent
2018-06-27 11:37:58 +02:00
Jiri Pechanec
ca6b1c9168
DBZ-743 Empty password for PostgreSQL and MySQL
2018-06-27 11:18:19 +02:00
Gunnar Morling
9d05fc2dc5
DBZ-767 Typo fix
2018-06-27 10:34:00 +02:00
Gunnar Morling
20f622dc40
DBZ-767 Adjusting test to more closely match original report
2018-06-27 10:33:41 +02:00
Jiri Pechanec
84f8bfb070
DBZ-767 Column should not be renamed when name is not changed
2018-06-27 09:14:59 +02:00
Jiri Pechanec
c14693c4ec
DBZ-765 Reduce the amount of test prints
2018-06-25 10:45:46 +02:00
Gunnar Morling
bf7a5018ca
Setting POM version back to 0.8.0-SNAPSHOT
2018-06-22 12:21:03 +02:00
Jenkins user
db42e4657a
[maven-release-plugin] prepare for next development iteration
2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99
[maven-release-plugin] prepare release v0.8.0.Beta1
2018-06-21 14:07:45 +00:00
Gunnar Morling
f5629d9f0b
DBZ-578 Interpret DATETIME columns using UTC;
...
This partly reverts the work previously done for DBZ-578,
where DATETIME columns (logical date + time values without TZ)
were shifted using the database's (or connection's) time zone.
Instead, the DATETIME values are interpreted using UTC again
with this change.
2018-06-20 15:58:19 +02:00
Gunnar Morling
42e313284a
DBZ-20 Adjustment after rebase
2018-06-20 13:05:37 +02:00
Gunnar Morling
b83f1493a2
DBZ-20 Avoiding Optional as method parameter
2018-06-20 13:05:37 +02:00
Gunnar Morling
dce1aa29f1
DBZ-20 Oracle precision supports negative numbers
2018-06-20 13:05:37 +02:00
Gunnar Morling
603f02e70d
DBZ-20 Excluding generated sources from CheckStyle
2018-06-20 13:05:37 +02:00
Gunnar Morling
d7e196a18e
DBZ-20 Initial import of Oracle connector based on XStream
2018-06-20 13:05:37 +02:00
Gunnar Morling
5b1807b286
DBZ-734 Avoiding repetition of option name and default value
2018-06-18 09:51:01 +02:00
Jiri Pechanec
ec03b431c9
DBZ-734 Maven profile for testing old/new DDL parser
2018-06-18 06:02:29 +02:00
Gunnar Morling
9eb4b90ec9
DBZ-191 Adding record validations
2018-06-15 20:23:17 +02:00
Gunnar Morling
84dd36df46
DBZ-252 Misc typo fixes and clean-up
2018-06-15 16:43:30 +02:00
Gunnar Morling
b56d6efdf1
DBZ-252 Making DataTypeResolver immutable
2018-06-15 16:43:29 +02:00
Gunnar Morling
54ca30624d
DBZ-252 Misc. improvements;
...
* Dedicated getter for DDL mode
* Using Objects#equal()
* Typo fixes
2018-06-15 16:43:29 +02:00
Gunnar Morling
6a6a8c9f99
DBZ-727 Initializing defaults for NUMERIC in Antlr parser
2018-06-15 16:43:29 +02:00
Gunnar Morling
759940b879
DBZ-252 Using existing parser implementation by default;
...
* Avoiding reflection for parser instantiation
* Making legacy parser instantiatable again
2018-06-15 16:43:25 +02:00
Jiri Pechanec
9580c8c290
DBZ-252 Rebase to master
2018-06-15 11:42:24 +02:00
rkuchar
5022933581
DBZ-252 add javadoc
2018-06-15 11:42:24 +02:00
rkuchar
c2e13b3b5f
DBZ-252 minor refactor
2018-06-15 11:42:24 +02:00
rkuchar
98384822eb
DBZ-252 refactor - move mysql antlr ddl parsser implementation into mysql-connector module
2018-06-15 11:42:24 +02:00
rkuchar
b8191963f7
DBZ-252 resolve todos
2018-06-15 11:42:24 +02:00
rkuchar
6812b4f7f0
DBZ-252 remove unused import
2018-06-15 11:42:24 +02:00
rkuchar
5c9e718436
DBZ-252 uncomment getting ddl parser instance from configuration + improve/refactor dataType resolver for ANTLR
2018-06-15 11:42:24 +02:00
rkuchar
99eac9a8e2
DBZ-252 some test fixes + separate ddl test for antlr parser
2018-06-15 11:42:24 +02:00
rkuchar
a4105adaaa
DBZ-252 refactor skipping nodes in tree walk
2018-06-15 11:42:24 +02:00
rkuchar
1ae7859acd
DBZ-252 parse create view
2018-06-15 11:42:23 +02:00
rkuchar
07212043b5
DBZ-252 test fixes + grammar fixes + some bug repairs in antlr parser
2018-06-15 11:42:23 +02:00
rkuchar
b4480c55b7
DBZ-252 parse database queries (create, alter, delete), parse name tokens without quotes, add new Mysql connector configuration field for choosing ddl parsing mode
2018-06-15 11:42:23 +02:00
rkuchar
32d181480d
DBZ-252 fix troubles with generalized system variables + add new data types that was missing in grammar
2018-06-15 11:42:23 +02:00
rkuchar
5851d1f07d
DBZ-252 fix initialization of system variables object + add default scope for variales in case that scope will no be set
2018-06-15 11:42:23 +02:00
rkuchar
9311c82bc5
DBZ-252 reformat code to not have anything after closing "}"
2018-06-15 11:42:23 +02:00
rkuchar
782ab75160
DBZ-252 move listener support from base to only legacy implementation, refactor system variables for possible use with other DBMSs + introduce data type resolver for antlr mysql parser
2018-06-15 11:42:23 +02:00
rkuchar
6a21702ac9
DBZ-252 finished alter table query listeners + new listener for rename table and create index
2018-06-15 11:42:23 +02:00
rkuchar
909668e14a
DBZ-252 Mysql alter table by add column/s parser listener
2018-06-15 11:42:23 +02:00
rkuchar
1df0b2033c
DBZ-252 Create new debezium-ddl-parser module
2018-06-15 11:42:23 +02:00
rkuchar
36e2f33bf6
DBZ-252 - extract creation of DdlParser interface
2018-06-15 11:42:23 +02:00
rkuchar
30cb9c3059
DBZ-252 - remove UNDERSCORE fragment from lexer
2018-06-15 11:42:23 +02:00
rkuchar
4a70d9bd59
DBZ-252 - case insensitive lexer
2018-06-15 11:42:23 +02:00
rkuchar
fb53cde565
DBZ-252 = add ANTLR gramatic for MySQLm setup build for it and exclude generated files from checkstyle
2018-06-15 11:42:23 +02:00
Jiri Pechanec
f9c84030b3
DBZ-727 Initializing defaults for NUMERIC
2018-06-14 16:39:48 +02:00
Jiri Pechanec
eadf65cd27
DBZ-578 Build failing on Jenkins CI
2018-06-13 14:43:29 +02:00
Jiri Pechanec
29f8891b4f
DBZ-578 Process timestamps without timezones as in server timezone
2018-06-11 15:27:08 +02:00
luobo
3f7186b3e5
DBZ-191 Represent all-zero time value with null if column allows null
2018-06-11 10:01:22 +02:00
Peter Goransson
3bf3e2e989
DBZ-601 Exposing new config
2018-06-07 10:42:20 +02:00
Gunnar Morling
c5b907873b
DBZ-601 Minor clean-up
2018-06-06 08:40:05 +02:00
Peter Goransson
f387f72474
DBZ-601 Snapshot delay implementation
2018-06-06 08:33:10 +02:00
luobo
5b33d81f00
DBZ-191 support default value of column in mysql
2018-06-04 10:43:53 +02:00
Gunnar Morling
61f576bb90
DBZ-706 Misc. clean-up
2018-06-01 11:23:30 +02:00
Stephen Powis
40d872f48c
DBZ-706 General cleanup
2018-06-01 11:23:30 +02:00
Stephen Powis
867202e0ca
DBZ-706 Slight refactoring / shuffling of code to validate source query field in snapshot tests
2018-06-01 11:23:12 +02:00
Stephen Powis
cc33fed16a
DBZ-706 Adding option for showing source query in sourceInfo element
2018-06-01 11:23:12 +02:00
Gunnar Morling
8d7806e71a
DBZ-693 Expanding test; updating connector option description
2018-05-28 12:35:47 +02:00
Jiri Pechanec
45bc7b1536
DBZ-693 On connect statements supported for PostgreSQL
2018-05-28 12:35:47 +02:00
Jiri Pechanec
f74a9fe059
DBZ-693 Initial SQL statements
2018-05-28 12:35:47 +02:00
Jiri Pechanec
fc107972f2
DBZ-598 Metrics expose the list of captured tables
2018-05-28 09:40:33 +02:00
Rao
15a90548fb
DBZ-666 Supporting ordered snapshot using tables.whitelist config
2018-05-17 06:23:31 +02:00
vucina.ivan
cd78b873fc
DBZ-670 Adding connect.keep.alive.interval.ms configuration option
2018-03-28 09:19:13 +02:00
Rao
1e1946d77e
DBZ-639 Removing restriction for none snapshot locking mode so that it can be used for all snapshot modes
2018-03-22 10:36:01 +01:00
Jiri Pechanec
a84b7ca682
DBZ-667 Test for scientific value sent to binlog
2018-03-20 14:42:55 +01:00
Jiri Pechanec
2b5370de1a
DBZ-667 Sceintific numbers parsing
2018-03-20 14:42:55 +01:00
Jenkins user
f4e151b23a
[maven-release-plugin] prepare for next development iteration
2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332
[maven-release-plugin] prepare release v0.7.5
2018-03-20 08:14:19 +00:00
Jiri Pechanec
40a527b6ea
DBZ-592 Make snapshot/init-sync false instead of null
2018-03-19 13:19:34 +01:00
Maciej Bryński
b7f13213b2
DBZ-646 Enhancing DDL parser to support compression parameter used with
...
CREATE TABLE on TokuDB
2018-03-18 18:29:41 +01:00
Peter Goransson
21cdf77634
DBZ-640 Destroy / de-initialization for Readers
2018-03-16 10:15:08 +01:00
Jiri Pechanec
469199350f
DBZ-660 Fix for CONSTRAINT UNIQUE syntax
2018-03-16 09:44:47 +01:00
Jenkins user
daf27207be
[maven-release-plugin] prepare for next development iteration
2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928
[maven-release-plugin] prepare release v0.7.4
2018-03-07 08:31:07 +00:00
Jiri Pechanec
a51322da0c
DBZ-611 Removed superfluous conversion
2018-03-06 16:55:24 +01:00
Jiri Pechanec
937c8b82f8
DBZ-611 Removed misleading comment
2018-03-06 16:55:24 +01:00
Jiri Pechanec
424aefbbb4
DBZ-611 Decimal-> String supported for MySQL
2018-03-06 16:55:24 +01:00
Gunnar Morling
f055020613
DBZ-602 Typo fixes; adding a few test assertions
2018-03-06 15:38:59 +01:00
stephen powis
c11875994a
DBZ-602 Adding new locking mode for bypassing table locks
2018-03-06 15:08:55 +01:00
Gunnar Morling
9b88e035f5
DBZ-641 Formatting
2018-03-02 09:05:07 +01:00
Satyajit Vegesna
e79e55d07f
DBZ-641 Enhancing parser to support CREATE TABLE with PARTITION ... ENGINE=InnoDB
2018-03-02 09:00:31 +01:00
Gunnar Morling
d89f4e91c6
DBZ-632 Removing unused method parameter
2018-02-23 21:54:41 +01:00
Gunnar Morling
c35c1827ff
DBZ-630 Refactoring around MySQL task context;
...
* Making relationship from MySqlTaskContext to MySqlJdbcContext has-a i
2018-02-21 12:04:41 +01:00
Gunnar Morling
eef1ad7c2a
DBZ-630 Refactoring around MongoDB task context;
...
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8
DBZ-630 Pulling up getClock() to ConnectorTaskContext
2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6
DBZ-630 Unifying common start-up logic across connectors
2018-02-21 12:04:41 +01:00
Gunnar Morling
7fe45db831
DBZ-626 Renaming AvroValidator to SchemaNameAdjuster;
...
Making apparent that this functionality doesn't solely validate but returns a new name actually
2018-02-20 09:26:47 +01:00
Gunnar Morling
cddd5fcc8a
DBZ-626 Passing schema validator as AvroValidator instance instead of Function
2018-02-20 09:26:47 +01:00
Jiri Pechanec
a08a00d46e
DBZ-615 Init unset Decimal props to default values
2018-02-20 08:54:28 +01:00
Jiri Pechanec
a06465ae34
DBZ-629 Fixed intermittent test issues
2018-02-20 08:01:51 +01:00
Gunnar Morling
e895fff2ac
DBZ-628 Using common properties for MySQL connector
2018-02-20 06:34:42 +01:00
Gunnar Morling
ab11f48843
DBZ-620 Creating envelope schemas only once per table type
2018-02-19 12:18:37 +01:00
Jenkins user
6d0cd88e12
[maven-release-plugin] prepare for next development iteration
2018-02-15 04:15:34 +00:00
Jenkins user
7d1e1a989e
[maven-release-plugin] prepare release v0.7.3
2018-02-15 04:15:34 +00:00
Jiri Pechanec
2e4be4c5ee
DBZ-278 Create topic only when needed
2018-02-13 16:48:42 +01:00
Jiri Pechanec
4541fc53c9
DBZ-278 Explicit history storage initialization
2018-02-13 16:48:42 +01:00
Jiri Pechanec
ecdce6529d
DBZ-220 Null Heartbeat object. Messages contains key
2018-02-13 12:26:48 +01:00
Jiri Pechanec
682cabb84a
DBZ-220 Generate heartbeat events in binlog thread
2018-02-13 12:24:51 +01:00
Jiri Pechanec
3c1ea8206b
DBZ-220 Refactor heartbeat to a separate class
2018-02-13 12:24:51 +01:00
Jiri Pechanec
4a22e17b71
DBZ-220 Disable heartbeat by default
2018-02-13 12:22:18 +01:00
Jiri Pechanec
ad682e109d
DBZ-220 Constatnts configurable, heartbeat independent of binlog
2018-02-13 12:22:18 +01:00
Jiri Pechanec
e6936616d9
DBZ-220 Send periodically messages to a heartbeat topic
2018-02-13 12:19:30 +01:00
Gunnar Morling
3e29591846
DBZ-582 Supporting TOMBSTONES_ON_DELETE option with MySQL connector
2018-02-13 10:19:18 +01:00
Gunnar Morling
e8fa7b887e
DBZ-588 Typo fix and additional method comment
2018-02-09 14:16:53 +01:00
Jiri Pechanec
2710094d11
DBZ-588 Defensively protecting CUD events themeselves
2018-02-09 11:03:16 +01:00
Jiri Pechanec
e4c9cf7191
DBZ-588 Changed message wording
2018-02-09 11:03:16 +01:00
Jiri Pechanec
16f4726a31
DBZ-588 Fail on inconsistent schema
2018-02-09 11:03:16 +01:00
Gunnar Morling
712824e43f
DBZ-593 Passing version explicitly from SourceInfos to base class instead of relying specific packaging structure
2018-02-02 13:16:43 +01:00
Jiri Pechanec
7ebee94169
DBZ-593 Add Debezium version to source in envelope
2018-02-02 13:16:43 +01:00
Gunnar Morling
3a281d1185
DBZ-595 Making sure resources are cleaned up when snapshotting fails;
...
* shutting down the snapshotting thread and the DB history producer client
if the connector is stopped while trying to write to the history topic
* reducing the time that KafkaProducer#send() will block if Kafka isn't
available; this will release the producer thread quicker in case the
connector is stopped during snapshotting
* not returning from finally block (!) in case the TX is rolled back; This
prevented the failed state to be set by the outer catch clause in execute()
2018-02-02 09:06:32 +01:00
Gunnar Morling
8821d89dfe
DBZ-587 Making sure snapshot reader thread is removed
2018-02-01 10:04:20 +01:00
Jiri Pechanec
92740a3626
DBZ-587 Fix thread leak, thread names
2018-02-01 10:04:20 +01:00
Jiri Pechanec
9b592204ac
DBZ-587 Centralize and unify thread management
2018-02-01 10:04:20 +01:00
Gunnar Morling
018a879d40
DBZ-583 Extracting a builder from ChainedReader, avoiding mutable state within its implementation
2018-01-30 08:59:41 +01:00
Jiri Pechanec
93cfb4f4cb
DBZ-572 Transaction is completed also by XID event
2018-01-29 15:40:41 +01:00
Jenkins user
04624341f5
[maven-release-plugin] prepare for next development iteration
2018-01-25 09:39:44 +00:00
Jenkins user
898f6884e1
[maven-release-plugin] prepare release v0.7.2
2018-01-25 09:39:44 +00:00
Robert Coup
f9d90a482f
DBZ-507 Expanding support for geometry types;
...
* Adding support for PostGIS geometry types
* Adding support for GEOMETRY, POLYGON and more in MySQL
* For all newly supported types, changes are represented using two new schema types Geometry and Geography, containing the WKB (binary geo data) ans srid (coord system identifier)
* The existing Point type also contains the new (optional) srid field
2018-01-24 16:51:49 +01:00
Jiri Pechanec
01577b40c3
DBZ-541 Only whitelisted tables are recorded in history
2018-01-24 14:43:57 +01:00
Peter Goransson
7d78357087
DBZ-443 Adding db-history recovery mode: schema_only_recovery
2018-01-23 09:17:39 +01:00
Gunnar Morling
7a7d43c237
DBZ-516 Returning control also from BlockingReader
2018-01-18 14:13:58 +01:00
Gunnar Morling
0c4190c493
DBZ-516 Using Duration instead of long in a few more places
2018-01-18 14:13:58 +01:00
Jiri Pechanec
24bdcaf059
DBZ-516 Return control to Connect periodically
2018-01-18 14:13:58 +01:00
Gunnar Morling
1d967a6d47
DBZ-392 Propagating value of tableIdCaseInsensitive
2018-01-17 11:21:02 +01:00
Gunnar Morling
4d1d016acb
DBZ-392 Making table map and set dedicated classes just with the needed API
2018-01-17 11:21:02 +01:00
Jiri Pechanec
9f0f0fe5d4
DBZ-392 Support for MySQL running on lowercase filesystems
2018-01-17 11:21:02 +01:00
Jiri Pechanec
446b102ad9
DBZ-534 Using correct datattype in list lookup
2018-01-12 12:30:44 +01:00
Jiri Pechanec
0b269a6e41
DBZ-538 Improve invalid DDL statement reporting
2018-01-12 12:26:30 +01:00
Jiri Pechanec
374999ee45
DBZ-411 ROLLBACK TO savepoint is parseable
2018-01-11 09:19:11 +01:00
Jiri Pechanec
3d7cc6a7e3
DBZ-530 Multiple PARTITION defintions supported
2018-01-08 11:58:09 +01:00
Jiri Pechanec
42991d260f
DBZ-524 RESTRICT in DROP COLUMN
2018-01-02 09:26:07 +01:00
Peter Goransson
5f3902d240
DBZ-522 Adding null check when encounting null TIME fields during snapshot
2018-01-02 07:00:03 +01:00
Jenkins user
6bb34b42f9
[maven-release-plugin] prepare for next development iteration
2017-12-20 07:15:12 +00:00
Jenkins user
16dcd4c980
[maven-release-plugin] prepare release v0.7.1
2017-12-20 07:15:12 +00:00
Jenkins user
5e09932cb9
[maven-release-plugin] prepare for next development iteration
2017-12-15 05:10:23 +00:00
Jenkins user
6c1d61e03b
[maven-release-plugin] prepare release v0.7.0
2017-12-15 05:10:23 +00:00
Jiri Pechanec
196f6b3571
DBZ-406 Fixing test, adding warning for disabled rollback
2017-12-13 14:12:27 +01:00
Gunnar Morling
06e72308d9
DBZ-406 Misc improvements;
...
* Making isReplayingEventsBeyondBufferCapacity() a "query method" only
* Using constant for buffer default size
* Misc. typo and doc fixes
* Removing unused variable
2017-12-13 14:08:11 +01:00
Jiri Pechanec
22dca4e498
DBZ-406 Buffering is disabled by default
2017-12-13 14:08:11 +01:00
Jiri Pechanec
5a5a85d920
DBZ-406 Support for look-ahead buffer with unlimited tx size
...
Also incorporates DBZ-405
2017-12-13 14:08:11 +01:00
Gunnar Morling
c88a521a52
DBZ-349 Avoiding need for map with overrides in config context
2017-12-13 12:34:30 +01:00
Andras Istvan Nagy
cc7459f4fb
DBZ-349 removed jackson databind dependency originally introduced for DBZ-349
2017-12-13 12:34:30 +01:00
Andras Istvan Nagy
05a232d435
DBZ-349 refactoring to use a comma-separated list of tables in the snapshot.select.statement.overrides
property and a separate config property for each override
2017-12-13 12:34:30 +01:00
Attila Szucs
c0d1c256ad
DBZ-349 fix integration test
2017-12-13 12:34:30 +01:00
Andras Istvan Nagy
631c518d8e
DBZ-349 Better support for large append-only tables by making the snapshotting process restartable
2017-12-13 12:34:30 +01:00
Jiri Pechanec
68f0a96a08
DBZ-461 Changed default bigint conversion to long
2017-12-02 21:04:29 +01:00
Gunnar Morling
db61ad2980
DBZ-491 Asserting offsets as per the given time and timezone (instead of now)
2017-12-01 10:55:43 +01:00
Gunnar Morling
79331608ae
DBZ-342 Adding regression test for existing "adaptive" time handling mode
2017-11-30 09:07:30 +01:00
rkerner
c7ac481c43
[DBZ-342] fix broken MySQL data type "TIME" handling
2017-11-29 20:34:12 +01:00
Peter Goransson
f4d6af9429
DBZ-491 Normalizing date to UTC before asserting
2017-11-29 08:56:13 +01:00
Gunnar Morling
08310f9ea5
DBZ-483 Removing two unused methods from MySqlSchema
2017-11-24 17:04:01 +01:00
Jiri Pechanec
20a2cdfdea
DBZ-476 Doubled quotes are parsed as escaped
2017-11-23 14:51:51 +01:00
Gunnar Morling
cc2afb3419
DBZ-473 Preventing unbounded polling in case connector is stopped during snapshotting
2017-11-21 11:24:05 +01:00
Jiri Pechanec
a1e0553b4d
DBZ-475 Support for RENAME USER statement
2017-11-21 08:43:44 +01:00
Gunnar Morling
fe9222a633
DBZ-474 Adding Attila Szucs to COPYRIGHT.txt
2017-11-20 17:39:51 +01:00
Attila Szucs
07da3c8718
DBZ-474 Fix failure when COLLATE comes after NOT NULL
2017-11-20 17:39:47 +01:00
Gunnar Morling
6f9086d608
DBZ-217 Typo fix
2017-11-14 18:01:26 +01:00
Gunnar Morling
951abb5c62
DBZ-217 Making behavior for dealing with event deserialization failures configurable
2017-11-14 18:01:26 +01:00
Gunnar Morling
f705ae4ac6
DBZ-217 Handling exceptions ocurring during binlog event deserialization;
...
* Catching the exception and wrapping it into a pseudo INCIDENT event
* Logging the exception and the current SourceInfo state
2017-11-14 18:01:26 +01:00
Jiri Pechanec
55a252521f
DBZ-447 Remove double initialization
2017-11-14 15:12:04 +01:00
Gunnar Morling
79a2e4a6c2
DBZ-396 Formatting and typo fix;
...
Also adding Peter to COPYRIGHT.txt
2017-11-14 11:37:28 +01:00
Peter Goransson
ba7478ffd7
DBZ-396 Adding BlockingReader to avoid high CPU utilization in
...
"snapshot_only" mode;
No binlog readers will be set up in this mode, causing the idle snapshot
reader to be polled with high frequency after it has done its work.
Therefore a blocking reader is added in that case, which prevents the
polling.
2017-11-14 11:32:27 +01:00
Jiri Pechanec
28258ece3f
DBZ-260 Timestamp correctly handles timezones
2017-11-14 10:38:32 +01:00
Gunnar Morling
c4a29f6fb7
DBZ-437 Some clean-up
2017-11-14 09:40:54 +01:00
Jiri Pechanec
4d253d2987
DBZ-437 String tokens and SQL words are handled separately in procedure parsing
2017-11-14 09:40:54 +01:00
Jiri Pechanec
1611159a85
DBZ-415 Can parse embedded IFs, can parse IF with ( as first char
2017-11-14 09:00:58 +01:00
Jiri Pechanec
6db4e9b599
DBZ-425 Fix ALTER column for changing default value
2017-11-13 22:51:59 +01:00
Gunnar Morling
2744f54e9d
DBZ-438 Renaming BufferedBlockingConsumer#flush() to close() to make clear it's onyl meant to be called once at the end
2017-11-13 14:19:39 +01:00
Jiri Pechanec
c905c7a558
DBZ-439 Support for PRECISION keyword
2017-11-13 13:58:53 +01:00
Gunnar Morling
5fbe742be8
DBZ-285 Specifying scope of dependencies in the individual POMs for the sake of comprehensibility
2017-11-10 16:48:32 +01:00
Jiri Pechanec
039566e717
DBZ-401 Log binlog reader position in case of exception
2017-11-03 09:12:09 +01:00
Jiri Pechanec
c1370556ff
DBZ-428 TEXT datatype can have an optional length
2017-11-03 08:37:00 +01:00
Jiri Pechanec
e841cbd609
DBZ-429 Fixed signed default value handling
2017-11-03 06:13:05 +01:00
Jiri Pechanec
1fe753717f
DBZ-412 Definition for column named column
can be changed
2017-11-01 16:32:44 +01:00
Jiri Pechanec
decf822c65
DBZ-419 PK constraint name is optional
2017-10-30 17:31:08 +01:00
Gunnar Morling
ce16ef5eb3
DBZ-395 Reducing log level from info to debug
2017-10-26 11:31:55 +02:00
Jiri Pechanec
f93e1e9bcd
DBZ-395 Make temp table regex stricter
2017-10-26 11:11:22 +02:00
Jiri Pechanec
1c6e652c71
DBZ-395 Filter out DROP TEMPORARY TABLE statements
2017-10-26 11:11:22 +02:00
Gunnar Morling
f86feaf9bd
DBZ-408 Allowing for columns named "column" (escaped) in ALTER TABLE ADD/DROP... without COLUMN word
2017-10-26 10:20:25 +02:00
Gunnar Morling
1d7f41af26
DBZ-408 Consuming COLUMN token only for ALTER TABLE statements, so to support columns named "column" (escaped) in CREATE TABLE statements
2017-10-26 10:20:25 +02:00
Gunnar Morling
a38c4254f1
DBZ-402 Removing redundant call to initialize()
2017-10-19 21:43:35 +02:00
Randall Hauch
c71f6d2752
DBZ-402 Reader now has ability to initialize resource prior to starting
...
Since some Reader implementations are combinations of other Readers, it is sometimes necessary for Reader implementations to initialize resources up front and not wait until they are called. Thus, a new `initialize()` method is added to Reader that will, for the combination readers, initialize all of the readers right away.
This makes it easier to correctly register JMX metrics, for example, up front once all of the readers have been configured for use, rather than to do so upon construction when it is not clear that a particular Reader will be used after it is constructed.
2017-10-19 21:43:35 +02:00
Gunnar Morling
6a8e08db5d
DBZ-363 Indentation fix; adding Ben Williams to copyright.txt
2017-10-18 10:31:00 +02:00
Ben Williams
a3b4fedd5f
DBZ-363 Add support for BIGINT UNSIGNED handling for MySQL
2017-10-18 10:20:03 +02:00
Jiri Pechanec
875c3ac87a
DBZ-336 Test for PAGE_CHECKSUM
2017-10-13 19:08:27 +02:00
Moira Tagle
4bcec0587a
Fix inaccurate comment on MySQLSchema.tables method
2017-10-03 05:23:53 +02:00
Gunnar Morling
9958faee7b
DBZ-378 Preventing NPE in MySqlConnectorTask#close()
2017-09-26 12:52:25 +02:00
Jenkins user
75937711fa
[maven-release-plugin] prepare for next development iteration
2017-09-21 04:42:02 +00:00
Jenkins user
a89b9332e4
[maven-release-plugin] prepare release v0.6.0
2017-09-21 04:42:02 +00:00
LiuHanlin
4543c001b1
DBZ-359 Add support for dec and fixed for mysql ddl parser.
...
According to Mysql document, https://dev.mysql.com/doc/refman/5.7/en/numeric-types.html
keywords DEC and FIXED are synonyms for DECIMAL.
2017-09-19 09:12:09 +02:00
Gunnar Morling
99f39038bb
DBZ-347 Formatting
2017-09-18 17:04:14 +02:00
Jiri Pechanec
38f8364647
DBZ-347 Simplified exception rethrowing
2017-09-18 17:02:21 +02:00
Jiri Pechanec
61ccefcc5c
DBZ-347 Configurable behaviour for unknown DDL, failing by default
2017-09-18 17:02:21 +02:00
Jiri Pechanec
1c5326e731
DBZ-304 MySQL test conversion to dynamic database creation
2017-09-18 16:23:55 +02:00
Jiri Pechanec
7b9c29efe6
DBZ-210 Lock tables before starting tx in snapshot
2017-09-15 12:20:35 +02:00
Jiri Pechanec
81edc3d7ee
DBZ-333 Filter database.history props out of JDBC URL
2017-09-15 11:45:53 +02:00
Jiri Pechanec
16671152f9
DBZ-346 Correctly parse MySQL merge tables
2017-09-15 10:03:44 +02:00
Jiri Pechanec
6fad4c62aa
Added log statements for error events
2017-08-31 06:37:33 +02:00
Jiri Pechanec
8cb681f106
DBZ-219 Stop BinlogReaderClient background jobs
2017-08-29 14:55:04 +02:00
Jenkins user
214696ef0c
[maven-release-plugin] prepare for next development iteration
2017-08-17 11:51:05 +00:00
Jenkins user
c867e6fea6
[maven-release-plugin] prepare release v0.5.2
2017-08-17 11:51:05 +00:00
Satyajit Vegesna
b4a0ac0c35
DBZ-324 Support for PAGE_CHECKSUM=1
2017-08-17 08:56:53 +02:00
Gunnar Morling
4d90fb5154
DBZ-284 Removing some trailing whitespace
2017-06-13 17:10:29 +02:00
Omar Al-Safi
107a3a8b56
DBZ-284 Fix NPE when inserting a NULL in a POINT column
2017-06-13 15:33:37 +02:00
Gunnar Morling
a8d1817c22
[maven-release-plugin] prepare for next development iteration
2017-06-09 16:14:31 +00:00
Gunnar Morling
3f512aace7
[maven-release-plugin] prepare release v0.5.1
2017-06-09 16:14:31 +00:00
Gunnar Morling
46bc2ab38b
DBZ-228 Avoiding some auto-boxing; extracting some additions to initialization time
2017-06-09 13:53:45 +02:00
Omar Al-Safi
730767f6c5
DBZ-228 Fix formatting issue
2017-06-09 13:53:45 +02:00
Omar Al-Safi
451bd95a75
DBZ-228 Add UNSIGNED BIGINT support
2017-06-09 13:53:45 +02:00
Omar Al-Safi
5d86ae592f
DBZ-228 Fix MySQL connector unsigned integer types
2017-06-09 11:52:14 +02:00
Randall Hauch
9581140c36
DBZ-216 MySQL connector should ignore DELETE FROM
statements
...
Parse and ignore any `DELETE` statements that might be seen in the binlog.
Theoretically, the binlog of a properly-configured MySQL server with row-level binlog enabled should never see these statements. However, users on Amazon RDS run into this quite frequently, and we should just handle and ignore them.
2017-06-01 20:04:45 +02:00
Gunnar Morling
68fc0966a2
Fixing wrong display name
2017-06-01 14:52:33 +02:00
Gunnar Morling
2a22f3d4b0
DBZ-254 Right-padding values for fixed-length BINARY columns with 0x00 (zero byte) characters for MySQL;
...
Also fixing JDBC types for binary data types for MySQL.
2017-05-30 15:04:38 +02:00
Gunnar Morling
559fb1f600
DBZ-262 Passing enum constants to configuration in one more test
2017-05-30 11:18:56 +02:00
Gunnar Morling
f17ced3b4a
DBZ-262 Letting configuration enum types implement EnumeratedValue
2017-05-23 21:26:22 +02:00
Gunnar Morling
c6674f8a4a
DBZ-253 Adding integration test
2017-05-23 21:26:01 +02:00
Gunnar Morling
1903b7dfe2
DBZ-253 Avoiding exception when receiving DDL events with table maintenance statements
2017-05-23 21:26:01 +02:00
Gunnar Morling
40cb3e2530
DBZ-215 Validating configuration to prevent that database history topic and schema change topic collide
2017-05-22 08:21:22 +02:00
Randall Hauch
02e655fa53
DBZ-198 Add another DDL parser test case
...
Added an additional test that is unable to reproduce the problem reported on April 4.
2017-05-19 15:50:03 +02:00
Gunnar Morling
03ca03c90c
DBZ-232 Removing unused constant
2017-05-19 09:28:56 +02:00
Randall Hauch
787959c4d0
DBZ-232 Removed the database and table recommenders
...
It’s not clear how valuable these recommenders actually are. First, it’s not clear about the expected semantics: can the user use values that don’t appear in the recommended values? Second, the recommenders that return large numbers of values can be slow and can result in very large REST API responses.
Debezium was using recommenders to return the database and table/collection names, but these lists can be very large for large databases. Rather than cap the number of recommended values and have the recommender return a subset of all potential values, we will instead remove the recommenders altogether.
2017-05-19 09:24:07 +02:00
Randall Hauch
8f69836c9e
DBZ-232 Changed the database and table recommenders to return no more than 100 values
2017-05-19 09:24:07 +02:00
Gunnar Morling
37529f27b1
DBZ-243 Removing superfluous default value implementation for server name as the filed is mandatory
2017-05-19 09:14:53 +02:00
Randall Hauch
c10c1204f3
Merge pull request #230 from jpechane/retries
...
DBZ-251 Prepopulate readbinlog_test database
2017-05-18 16:58:53 -05:00
Randall Hauch
5fb878137e
Merge pull request #223 from gunnarmorling/DBZ-243
...
DBZ-243 Don't mention default value for "database.server.name"
2017-05-18 12:09:39 -05:00
Jiri Pechanec
e79268ec90
Prepopulate readbinlog_test database
2017-05-12 07:20:53 +02:00
Randall Hauch
afef753d36
DBZ-242 Corrected MySQL filters to handle built-in tables
...
When the `table.ignore.built` is set to `false`, the MySQL connector included all of the ~250 system tables yet the database and table filters were never applied to these system tables and therefore all would be captured. With this change, the database filters and table filters now apply to the system tables should they not be ignored.
Note that this does change the behavior of the connector when `table.ignore.built` is set to `false`. However, it is unlikely that anyone is seriously capturing all of the system table changes, so correcting the behavior is the preferred solution.
2017-05-11 11:57:08 +02:00
Gunnar Morling
3f55398bab
DBZ-243 Don't mention default value for "database.server.name" for MySQL connector
2017-05-08 20:24:28 +02:00
Gunnar Morling
baa8119df8
Typo fix
2017-05-08 17:40:19 +02:00
Gunnar Morling
1074de4efa
DBZ-222 Some more clean-up
2017-05-04 09:25:25 +02:00
Gunnar Morling
6eea4c9717
DBZ-222 Some typo fixes
2017-05-04 08:53:51 +02:00
Omar Al-Safi
3d92011277
DBZ-222 Typo cleanup and removed unnecessary assertion
2017-05-04 08:53:34 +02:00
Gunnar Morling
5630b61be6
DBZ-222 dependency clean-up
2017-05-04 08:53:05 +02:00
Omar Al-Safi
2a17748a95
DBZ-222 Readded readbinlog_test.product and readbinlog_test.purchased to MySqlConnectorIT
2017-05-04 08:53:05 +02:00
Omar Al-Safi
791545c5f4
DBZ-222 Added support for MySQL POINT type
2017-05-04 08:53:05 +02:00
Jiri Pechanec
51a1c7cd69
DBZ-229 - check all privilege records
2017-04-26 15:25:59 +02:00
Randall Hauch
709cd8f3fe
[maven-release-plugin] prepare for next development iteration
2017-03-27 11:28:12 -05:00
Randall Hauch
2bc3d45954
[maven-release-plugin] prepare release v0.5.0
2017-03-27 11:28:11 -05:00
Randall Hauch
25adc3f642
Merge pull request #209 from rhauch/dbz-205
...
DBZ-205 Corrected MySQL connector to handle 2-digit years
2017-03-27 11:10:04 -05:00
Randall Hauch
81f62b6961
DBZ-205 Corrected MySQL connector to handle 2-digit years
...
MySQL has special handling of 2-digit years that it deems are ambiguous, such as the year value `17` that is actually treated as `2017`. Apparently the 2-digit values are stored in MySQL and the interpretation is performed when the data is extracted, so therefore the connector needs to also perform this adjustment of the year values. This commit uses the JDK’s `TemporalAdjuster` interface and passes this down to the requisite temporal-related datatype handling code. The MySQL connector then provides its own `TemporalAdjuster` implementation that adjusts the year values via the excellend JDK `Temporal` methods.
A row in one of the MySQL test databases was changed to use a 2-digit year of `16` while the test method still checks that the year is still 2016`, verifying that the year value is properly adjusted.
2017-03-27 10:58:21 -05:00
Randall Hauch
74fbb4a140
DBZ-198 Corrected MySQL DDL parser
...
The parser now handles `BEGIN…END` blocks better by properly handling `IF()` functions that are not `IF…THEN…END IF` control blocks, and `CASE … WHEN … END CASE` control blocks.
2017-03-24 13:12:46 -05:00
Sanjay Kr Singh
ac4575a13c
update mysql-dbz-198.ddl with 3 new test cases for PROCEDURE and 2 test cases for FUNCTION
...
From L49 - one DDL test case for PROCEDURE ,
From L278 - one DDL test case for FUNCTION ,
From L433 - one more DDL test case for PROCEDURE with CURSOR,
From L713 - ONE more DDL TEST FOR PROCEDURE,
From L755 - ONE more DDL TEST CASE FOR FUNCTION
2017-03-24 12:29:17 +05:30
Randall Hauch
430d756062
[maven-release-plugin] prepare for next development iteration
2017-03-17 15:41:58 -05:00
Randall Hauch
536cbf6300
[maven-release-plugin] prepare release v0.4.1
2017-03-17 15:41:57 -05:00
Randall Hauch
2086b779b1
DBZ-204 Added test case but unable replicate error
...
Added a test case that uses the MySQL DDL parser to parse similar DDL statements to those reported in the issue, but these are properly handled with the current state of the `master` branch.
2017-03-17 14:15:25 -05:00
Randall Hauch
e5ee3847dd
DBZ-195 Added tests to try to replicate a reported issue
...
Added a table and inserted rows that tries to replicate the problem reported in DBZ-195, but the test was unable to replicate the problem. In fact, this really is no different than existing tests. Changed the log messages so that if/when this happens again it will be possible to know which row was problematic.
2017-03-17 10:47:32 -05:00
Randall Hauch
ddbc1e07aa
DBZ-197 Corrected MySQL connector to handle invalid enum values
...
MySQL represents an invalid enum literal in the binlog events as an empty string or an value of `0`. Now, when the connector comes across such a value in the binlog, it will instead use an empty string for the enum literal.
2017-03-16 13:55:24 -05:00
Randall Hauch
cf5391482a
DBZ-198 Improved MySQL DDL parser to better handle blocks
...
The MySQL parser now properly handles control blocks such as `BEGIN…END`, `IF…END IF`, `REPEAT…END REPEAT`, and `LOOP…END LOOP`, even in cases where the block is preceded by and terminated by a label.
2017-03-16 13:32:21 -05:00
Randall Hauch
06d9bcb092
Merge pull request #199 from dasl-/daylightsavings
...
DBZ-202 Fix daylight savings test issues
2017-03-15 15:41:41 -05:00
dleibovic
782ccc66e5
fix daylight savings test issues
2017-03-15 15:44:03 -04:00
rich
06a25f8fd9
modify the query used to fetch the table list so that it includes tables only(no views)
2017-03-15 13:56:52 -04:00
Randall Hauch
b48ccce4b5
DBZ-200 Corrected MySQL DDL parser to better handle column definitions
...
Apparently not all reserved words must be quoted when using them as colum names, so refactored MySQL’s DDL parser to better handle a variety of unquoted colum names that are reserved words.
2017-03-08 12:12:27 -06:00
Josh Stanfield
c794416684
update to allow savepoint in mysql replication stream
2017-03-08 09:28:10 -07:00
dleibovic
9fd2afc8b4
Increase the default mysql init timeout to 60 seconds for slower computers. Also paramterize it so that users can pass a custom value via 'mvn clean install -Dmysql.init.timeout=80000' for example
2017-02-23 13:34:14 -05:00
rich
9aa49736c8
DBZ-140 when locking individual tables, use a single statement with all the table names instead of issuing a statement per table which causes a MySQL error
2017-02-16 15:45:29 -05:00
Randall Hauch
043a2d2d92
DBZ-194 Improved MySQL connector’s built-in table filtering
...
The MySQL connector’s built-in table filter now just filters out all tables within the known built-in databases, and does not check the names of the tables. Thus, the connector should no longer filter out tables in other databases that happen to have the same names as the tables in the built-in databases.
2017-02-14 09:23:39 -06:00
Randall Hauch
af94fa8759
DBZ-193 MySQL DDL parser handles FULLTEXT index
...
Corrected the MySQL DDL parser to correctly handle `FULLTEXT` indexes within a `CREATE TABLE` statement. The parser was incorrectly using `canConsume(…)` with a list of options instead of `canConsumeAnyOf(…)`.
2017-02-10 15:49:20 -06:00
Randall Hauch
9a4a177004
DBZ-188 Corrected JavaDoc
2017-02-10 15:39:22 -06:00
dleibovic
aa50bfe71a
DBZ-188: Allow a debezium mysql connector to filter production of DML events into kafka by the mysql UUID of the event
...
With GTIDs enabled, each transaction in the binlog contains a GTID event, which gives us access to the GTID of the transaction. The GTID has the following format: source_id:transaction_id, where source_id is the UUID of the mysql server the transaction was written to.
I propose to allow a debezium instance to be configured with a UUID pattern to check against before producing DML events into Kafka. Debezium would produce a DML event into kafka if and only if the UUID in the event's GTID matches the pattern with which debezium was configured.
The configuration for the UUID patterns will make use of the existing gtid.source.includes and gtid.source.excludes options. The DML event filtering will only be performed if the new option gtid.source.filter.dml.events is true.
2017-02-10 14:14:10 -05:00
Randall Hauch
d2986710a5
DBZ-188 More efficient GTID source filters for MySQL Connector
...
Changed the GTID source filters in the MySQL connector to be far more efficient when the filters specify literal UUIDs rather than regex patterns. In these cases, the predicate just checks whether a supplied value is in a hash set, and no regular expression patterns are used.
The GTID source filters can still be a combination of UUID literals and regular expressions, and the predicate will use the best implementation for each. For example, if the filters include all UUID literals, then regular expressions will never be used.
2017-02-10 11:34:24 -06:00
Randall Hauch
8c60c29883
[maven-release-plugin] prepare for next development iteration
2017-02-07 14:22:12 -06:00
Randall Hauch
20134286e9
[maven-release-plugin] prepare release v0.4.0
2017-02-07 14:22:11 -06:00
Randall Hauch
403fee1375
DBZ-185 MySQL’s database history now filters GTID sources
...
Corrects how the MySQL connector reloads database history to take into account the included and excluded GTID sources. This only affects a connector configured to capture changes from _multiple_ MySQL database servers when GTID sources are explicitly excluded or included.
2017-02-07 11:21:22 -06:00
Randall Hauch
bb0800ca3a
DBZ-140 Improved locking logic to support RDS
...
Improved the MySQL connector's logic to better handle Amazon RDS that does not allow giving user `SUPER` privileges. As before, the connector starts a transaction and attempts to get a global read lock via `FLUSH TABLES WITH READ LOCK` to prevent writes to the database so that the binlog position can be accurately read _and_ the table schemas can be read without interference from other clients. Once that is done, the connector releases the global read lock and continues in the same transaction to read all table rows. This means that our snapshot is consistent, but we maintain the global read lock for a very short period of time.
Amazon's RDS and Aurora are hosted MySQL instances that do not allow users to have the `SUPER` privilege, which means the user cannot get a global read lock. In this case, the connector detects this error, continues to read the database and table names (without any lock), and _then_ uses `FLUSH TABLES <tableName> WITH READ LOCK` on each table that satisfies the filters to prevent changes from other clients. The connector then reads the table schemas, reads _all_ table rows, commits the transaction, and _finally_ releases the table locks.
Therefore, there are two very different behaviors/requirements when the user can't obtain a global read lock because of lack of privilege, like on RDS:
# The RDS user that the connector makes use of must also have the `LOCK TABLES` privilege; without it the connector will fail during the snapshot.
# The connector must hold the table read locks _until it has completed reading all of the tables_, since release the table locks using `UNLOCK TABLES` would prematurely commit our transaction and prevent us from getting a consistent snapshot. From the [MySQL documentation](https://dev.mysql.com/doc/refman/5.7/en/flush.html ):
> `UNLOCK TABLES` implicitly commits any active transaction only if any tables currently have been locked with `LOCK TABLES`. The commit does not occur for `UNLOCK TABLES` following `FLUSH TABLES WITH READ LOCK` because the latter statement does not acquire table locks.
2017-02-06 13:56:55 -06:00
Randall Hauch
5490842449
Merge pull request #175 from rhauch/dbz-176
...
DBZ-176 Corrected MySQL DDL parser to support creating triggers with definers
2017-02-02 13:59:01 -06:00
Randall Hauch
74e5ba6448
DBZ-176 Corrected MySQL DDL parser to support creating triggers with definers
...
The MySQL DDL parser was not correclty handling `DEFINER` clauses within `CREATE TRIGGER` or `CREATE EVENT` statements. Support for `DEFINER` clauses was recently added for the various forms of `CREATE PROCEDURE`, `CREATE FUNCTION` and `CREATE VIEW` statements. These are the only kinds of statements that have the definer attribute, per the [MySQL documentation](https://dev.mysql.com/doc/refman/5.7/en/stored-programs-security.html ).
2017-02-02 12:44:28 -06:00
Randall Hauch
32a88fdc6f
DBZ-184 Added database and table name to change event metadata
2017-02-02 12:09:53 -06:00
Randall Hauch
6230cab90e
Merge pull request #173 from rhauch/dbz-113
...
DBZ-113 Added MySQL threads to the event’s source metadata
2017-02-02 12:00:19 -06:00
Randall Hauch
fe17b246af
DBZ-113 Added MySQL threads to the event’s source metadata
...
Changed the events’ `source` structure to optionally contain the identifier of the MySQL thread where appropriate. The thread is included on each `BEGIN` binlog event, so these are captured and added to all of the associated change events produced for that transaction.
2017-02-02 11:53:32 -06:00
Randall Hauch
f2a65d03df
DBZ-174 Added support for new binlog events
...
MySQL recently added additional binlog events, and this commit adds support to handle these new events by ignoring them.
2017-02-01 15:26:28 -06:00
Horia Chiorean
031c4a1552
DBZ-183 Fixes the BinlogReader's handling of TIMESTAMP columns to correctly account for timezones
2017-01-25 16:39:36 +02:00
Randall Hauch
a73f85a80f
Merge pull request #162 from rareddy/DBZ-177
...
DBZ-177: Providing an alternative way to create JDBC connection based …
2017-01-13 13:37:38 -06:00
Ramesh Reddy
a9aace3480
DBZ-177: Providing an alternative way to create JDBC connection based on the configured JDBC driver class name and supplied classloader. The loading/creating the JDBC connections is not reliable when driver libraries in a different classloader than the DriverManager.
2017-01-13 12:58:14 -06:00
Horia Chiorean
a300d3e1cf
DBZ-3 Changes the configuration of the Docker Maven plugin to only use alias naming when necessary and moves the PG connector ahead of the Mongo connector in the build
2016-12-27 14:44:33 +02:00
Horia Chiorean
23e3f59fa1
DBZ-3 Implements a connector for streaming changes from a Postgres database
...
The version of the DB server required for this to work is at least 9.4
The commit also updates the general DBZ build system for:
* custom checkstyle package exclusions - required by the Postgres driver the protobuf code for now
* adds support for debugging Surefire and Failsafe
2016-12-27 14:44:32 +02:00
Randall Hauch
e60839e76b
DBZ-164 Improved MySQL snapshot reader logic
...
Added more logic to the snapshot reader to better handle errors when reading the list of table names in each database. Now, any errors with a single database (e.g., some of the not-quite-a-database names described in the JIRA issue) will cause the snapshot reader to simply skip that database name and continue on (with proper logging).
This change also quotes all of the database and table names when used in SQL statements.
2016-12-20 22:03:46 -06:00
Randall Hauch
fd7e152852
Merge pull request #142 from rhauch/dbz-151
...
DBZ-151 Added new integration test framework
2016-12-20 17:53:16 -06:00