Commit Graph

471 Commits

Author SHA1 Message Date
Randall Hauch
e02de1a2da Merge pull request #206 from rhauch/v0_4_1_changelog
Updated changelog for 0.4.1
2017-03-17 15:34:08 -05:00
Randall Hauch
5fc3e91759 Updated changelog for 0.4.1 2017-03-17 15:33:30 -05:00
Randall Hauch
3d54f811fa Merge pull request #205 from rhauch/dbz-201
DBZ-201 Corrected handling of ObjectID within non-insert events
2017-03-17 15:12:32 -05:00
Randall Hauch
ba49756e3b Merge pull request #204 from rhauch/dbz-204
DBZ-204 Added test case but unable replicate error
2017-03-17 15:03:01 -05:00
Randall Hauch
88b5ffa113 DBZ-201 Corrected handling of ObjectID within non-insert events
Corrected how the MongoDB extracts the event key from the update event’s ObjectID, and verified the behavior with an integration test.
2017-03-17 15:02:29 -05:00
Randall Hauch
2086b779b1 DBZ-204 Added test case but unable replicate error
Added a test case that uses the MySQL DDL parser to parse similar DDL statements to those reported in the issue, but these are properly handled with the current state of the `master` branch.
2017-03-17 14:15:25 -05:00
Randall Hauch
db1aacc5f9 Merge pull request #203 from rhauch/dbz-195
DBZ-195 Added tests to try to replicate a reported issue
2017-03-17 11:02:16 -05:00
Randall Hauch
e5ee3847dd DBZ-195 Added tests to try to replicate a reported issue
Added a table and inserted rows that tries to replicate the problem reported in DBZ-195, but the test was unable to replicate the problem. In fact, this really is no different than existing tests. Changed the log messages so that if/when this happens again it will be possible to know which row was problematic.
2017-03-17 10:47:32 -05:00
Randall Hauch
a2d6d62fb0 Merge pull request #201 from rhauch/dbz-197
DBZ-197 Corrected MySQL connector to handle invalid enum values
2017-03-16 14:16:23 -05:00
Randall Hauch
ddbc1e07aa DBZ-197 Corrected MySQL connector to handle invalid enum values
MySQL represents an invalid enum literal in the binlog events as an empty string or an value of `0`. Now, when the connector comes across such a value in the binlog, it will instead use an empty string for the enum literal.
2017-03-16 13:55:24 -05:00
Randall Hauch
0c2716d470 Merge pull request #200 from rhauch/dbz-198
DBZ-198 Improved MySQL DDL parser to better handle blocks
2017-03-16 13:46:17 -05:00
Randall Hauch
cf5391482a DBZ-198 Improved MySQL DDL parser to better handle blocks
The MySQL parser now properly handles control blocks such as `BEGIN…END`, `IF…END IF`, `REPEAT…END REPEAT`, and `LOOP…END LOOP`, even in cases where the block is preceded by and terminated by a label.
2017-03-16 13:32:21 -05:00
Randall Hauch
06d9bcb092 Merge pull request #199 from dasl-/daylightsavings
DBZ-202 Fix daylight savings test issues
2017-03-15 15:41:41 -05:00
dleibovic
782ccc66e5 fix daylight savings test issues 2017-03-15 15:44:03 -04:00
Randall Hauch
59ffd7011b Merge pull request #198 from richoconnell/DBZ-140
DBZ-140: modify the query used to fetch the table list so that it includes tables only (no views)
2017-03-15 14:37:39 -05:00
rich
06a25f8fd9 modify the query used to fetch the table list so that it includes tables only(no views) 2017-03-15 13:56:52 -04:00
Randall Hauch
f5c1e823b3 Merge pull request #196 from rhauch/dbz-200
DBZ-200 Corrected MySQL DDL parser to better handle column definitions
2017-03-08 13:21:42 -06:00
Randall Hauch
b48ccce4b5 DBZ-200 Corrected MySQL DDL parser to better handle column definitions
Apparently not all reserved words must be quoted when using them as colum names, so refactored MySQL’s DDL parser to better handle a variety of unquoted colum names that are reserved words.
2017-03-08 12:12:27 -06:00
Randall Hauch
e0a631431b Merge pull request #195 from p5k6/fix_mysql_savepoint
DBZ-196 Handle SAVEPOINT statements appearing in a MySQL replication stream
2017-03-08 10:42:57 -06:00
Josh Stanfield
c794416684 update to allow savepoint in mysql replication stream 2017-03-08 09:28:10 -07:00
Randall Hauch
8f9fb7a7cd Merge pull request #191 from dasl-/mysql-init
Increase the default mysql init timeout to 60 seconds for slower comp…
2017-02-24 07:03:47 -05:00
Randall Hauch
665cf346d9 Merge pull request #192 from hchiorean/DBZ-158
DBZ-158 Changes the version of the Postgres JDBC driver to 42.0.0
2017-02-24 06:40:30 -05:00
Horia Chiorean
d2210a2a50 DBZ-158 Changes the version of the Postgres JDBC driver to 42.0.0 2017-02-24 09:09:04 +02:00
dleibovic
9fd2afc8b4 Increase the default mysql init timeout to 60 seconds for slower computers. Also paramterize it so that users can pass a custom value via 'mvn clean install -Dmysql.init.timeout=80000' for example 2017-02-23 13:34:14 -05:00
Randall Hauch
14f62dc844 Merge pull request #190 from richoconnell/DBZ-140
DBZ-140 when locking individual tables, use a single statement with …
2017-02-16 16:07:32 -06:00
rich
9aa49736c8 DBZ-140 when locking individual tables, use a single statement with all the table names instead of issuing a statement per table which causes a MySQL error 2017-02-16 15:45:29 -05:00
Randall Hauch
e86fee9834 Merge pull request #189 from rhauch/dbz-194
DBZ-194 Improved MySQL connector’s built-in table filtering
2017-02-14 09:34:32 -06:00
Randall Hauch
043a2d2d92 DBZ-194 Improved MySQL connector’s built-in table filtering
The MySQL connector’s built-in table filter now just filters out all tables within the known built-in databases, and does not check the names of the tables. Thus, the connector should no longer filter out tables in other databases that happen to have the same names as the tables in the built-in databases.
2017-02-14 09:23:39 -06:00
Randall Hauch
3fe0467be0 Merge pull request #188 from rhauch/dbz-193
DBZ-193 MySQL DDL parser handling FULLTEXT index
2017-02-10 15:57:24 -06:00
Randall Hauch
af94fa8759 DBZ-193 MySQL DDL parser handles FULLTEXT index
Corrected the MySQL DDL parser to correctly handle `FULLTEXT` indexes within a `CREATE TABLE` statement. The parser was incorrectly using `canConsume(…)` with a list of options instead of `canConsumeAnyOf(…)`.
2017-02-10 15:49:20 -06:00
Randall Hauch
9a4a177004 DBZ-188 Corrected JavaDoc 2017-02-10 15:39:22 -06:00
Randall Hauch
333cf8e010 Updated list of contributors 2017-02-10 15:37:23 -06:00
Randall Hauch
8304625d7c Merge pull request #185 from dasl-/DBZ-188
DBZ-188: Allow a debezium mysql connector to filter production of DML…
2017-02-10 15:17:09 -06:00
Randall Hauch
1ed9f66308 Merge pull request #187 from rhauch/dbz-192
DBZ-192 Corrected links to release notes to public URLs
2017-02-10 15:12:07 -06:00
Randall Hauch
01ee528526 DBZ-192 Corrected links to release notes to public URLs 2017-02-10 15:11:31 -06:00
dleibovic
aa50bfe71a DBZ-188: Allow a debezium mysql connector to filter production of DML events into kafka by the mysql UUID of the event
With GTIDs enabled, each transaction in the binlog contains a GTID event, which gives us access to the GTID of the transaction. The GTID has the following format: source_id:transaction_id, where source_id is the UUID of the mysql server the transaction was written to.

I propose to allow a debezium instance to be configured with a UUID pattern to check against before producing DML events into Kafka. Debezium would produce a DML event into kafka if and only if the UUID in the event's GTID matches the pattern with which debezium was configured.

The configuration for the UUID patterns will make use of the existing gtid.source.includes and gtid.source.excludes options. The DML event filtering will only be performed if the new option gtid.source.filter.dml.events is true.
2017-02-10 14:14:10 -05:00
Randall Hauch
3aea39397f Merge pull request #183 from rhauch/dbz-188-uuid-filters
DBZ-188 More efficient GTID source filters for MySQL Connector
2017-02-10 12:14:05 -06:00
Randall Hauch
d2986710a5 DBZ-188 More efficient GTID source filters for MySQL Connector
Changed the GTID source filters in the MySQL connector to be far more efficient when the filters specify literal UUIDs rather than regex patterns. In these cases, the predicate just checks whether a supplied value is in a hash set, and no regular expression patterns are used.

The GTID source filters can still be a combination of UUID literals and regular expressions, and the predicate will use the best implementation for each. For example, if the filters include all UUID literals, then regular expressions will never be used.
2017-02-10 11:34:24 -06:00
Randall Hauch
8c60c29883 [maven-release-plugin] prepare for next development iteration 2017-02-07 14:22:12 -06:00
Randall Hauch
20134286e9 [maven-release-plugin] prepare release v0.4.0 2017-02-07 14:22:11 -06:00
Randall Hauch
e5b42e4bc1 Updated changelog for 0.4.0 2017-02-07 13:35:30 -06:00
Randall Hauch
6da2a8a1b2 Merge pull request #178 from rhauch/dbz-185
DBZ-185 MySQL’s database history now filters GTID sources
2017-02-07 13:25:32 -06:00
Randall Hauch
e8414ad6c1 Merge pull request #181 from rhauch/dbz-187
DBZ-187 Upgrade MongoDB server and Java driver versions
2017-02-07 12:56:52 -06:00
Randall Hauch
896dd35bcb DBZ-187 Upgrade MongoDB server and Java driver versions
Upgraded the MongoDB server to 3.2.12 and the Java driver to 3.4.2.
2017-02-07 12:49:50 -06:00
Randall Hauch
7303bdfccb Merge pull request #180 from rhauch/dbz-186
DBZ-186 Upgraded MySQL binary log client library
2017-02-07 12:43:06 -06:00
Randall Hauch
9ae50b3691 DBZ-186 Upgraded MySQL binary log client library
Upgraded Shyiko’s MySQL binary log client library from 0.8.0 to 0.9.0 to get new timeout behavior when it opens sockets and fix for JSON array processing.
2017-02-07 12:34:12 -06:00
Randall Hauch
403fee1375 DBZ-185 MySQL’s database history now filters GTID sources
Corrects how the MySQL connector reloads database history to take into account the included and excluded GTID sources. This only affects a connector configured to capture changes from _multiple_ MySQL database servers when GTID sources are explicitly excluded or included.
2017-02-07 11:21:22 -06:00
Randall Hauch
6cbc78c5c4 Merge pull request #179 from rhauch/dbz-173
DBZ-173 Upgraded Confluent Platform libraries used in test cases
2017-02-07 11:20:43 -06:00
Randall Hauch
65951308f7 DBZ-173 Upgraded Confluent Platform libraries
Some of our test cases verify (de)serialization using the Avro Converter, which is included in the Confluent Platform. This commit upgrades the Confluent Platform to version 3.1.2, which matches Kafka 0.10.1.1.
2017-02-07 11:18:21 -06:00
Randall Hauch
f10970a4c9 Merge pull request #177 from rhauch/dbz-140
DBZ-140 Improved locking logic to support RDS
2017-02-06 14:33:00 -06:00