alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
1,586 Commits 3 Branches 0 Tags 51 MiB
2a80931d03
Commit Graph

464 Commits

Author SHA1 Message Date
Gunnar Morling
1523f230ca DBZ-793 Using TableFilter consistently; 
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
 2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8 [maven-release-plugin] prepare for next development iteration 2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f [maven-release-plugin] prepare release v0.8.0.Final 2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d [maven-release-plugin] prepare for next development iteration 2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0 [maven-release-plugin] prepare release v0.8.0.CR1 2018-07-04 07:07:44 +00:00
Gunnar Morling
99048365c9 DBZ-771 Unsetting default value when changing a column's type 2018-07-03 11:55:20 +02:00
Gunnar Morling
c7d5288d40 DBZ-771 Making sure causing DDL statement is logged also if default value can't be parsed 2018-07-03 11:55:20 +02:00
Gunnar Morling
dcd6b01134 DBZ-644 Simplifying RelationalDatabaseSchema constructor; 
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
 2018-07-03 07:03:59 +02:00
Gunnar Morling
b0980b994c DBZ-644 Adding test for Postgres 2018-07-03 07:03:59 +02:00
Gunnar Morling
068aa85bd6 DBZ-644 Misc. clean-up and tests; 
* Using parameter keys under "__debezium" namespace
* More expressive names
* Adding tests
 2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema 2018-07-03 07:03:59 +02:00
orr.ganani
ac515b8064 DBZ-644 change Original Data Type mapper and add a test 2018-07-03 07:03:59 +02:00
orr.ganani
747184c572 DBZ-644 add JDBC data type as additional metadata to DBZ events, if configuration is opted in 2018-07-03 07:03:59 +02:00
Gunnar Morling
c4c8cbc3ab DBZ-773 Moving management of Tables to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab DBZ-773 Moving management of table schemas to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
a292b05a96 DBZ-751 Propagating DECIMAL column precision to Avro schemas 2018-06-27 14:48:59 +02:00
Gunnar Morling
b27421ddd0 DBZ-759 Testing via SchemaUtil; 
* Making RecordWriter private
* Removing quotes around string representation of byte arrays
 2018-06-26 12:03:53 +02:00
Andreas Bergmeier
82bac78ba8 DBZ-759 Add Test to ensure correct serialization of byte arrays 
Needed to make RecordWriter public to have access in tests.
 2018-06-26 11:50:54 +02:00
Andreas Bergmeier
71abca2afb DBZ-759 Fix unusable array information in Serialization 
According to ErrorProne, (implicitly) calling toString on the array does
not give useful information.
Also applying Base64 encoding is strange here.
 2018-06-26 11:50:54 +02:00
Gunnar Morling
8d0d35762e DBZ-759 Removing superfluous (and broken) ColumnEditorImpl#compareTo() implementation 2018-06-25 17:04:45 +02:00
Gunnar Morling
bf7a5018ca Setting POM version back to 0.8.0-SNAPSHOT 2018-06-22 12:21:03 +02:00
Gunnar Morling
357102158a DBZ-759 Fixing broken Document#setArray() method 2018-06-22 11:03:00 +02:00
Gunnar Morling
ce81f37990 DBZ-759 Adding unit test 2018-06-22 09:19:45 +02:00
Andreas Bergmeier
7417baea4d DBZ-759 Fix equality comparison for BinaryValue 
If value is binary or string, it should be better to compare the content
of the actual byte arrays.
According to ErrorProne you use reference equality when calling equals on
an array.

Part of https://issues.jboss.org/browse/DBZ-759
 2018-06-22 09:19:45 +02:00
Andreas Bergmeier
333d6cb57d DBZ-759 Fix hashCode of BinaryValue 
According to ErrorProne, the hashCode of a byte array returns useless
information.

Part of https://issues.jboss.org/browse/DBZ-759
 2018-06-22 09:19:41 +02:00
Andreas Bergmeier
d374ca0464 DBZ-759 Fix ColumnImpl.scale equality check 
According to ErrorProne, checking Optional via operator == leads to
reference equality check. Probably want rather to check for contained
values.
 2018-06-22 09:08:27 +02:00
Jenkins user
db42e4657a [maven-release-plugin] prepare for next development iteration 2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99 [maven-release-plugin] prepare release v0.8.0.Beta1 2018-06-21 14:07:45 +00:00
Gunnar Morling
96c59a1568 DBZ-20 Moving debezium-connector-oracle from main to incubator repo 2018-06-20 13:05:37 +02:00
Gunnar Morling
b83f1493a2 DBZ-20 Avoiding Optional as method parameter 2018-06-20 13:05:37 +02:00
Gunnar Morling
dce1aa29f1 DBZ-20 Oracle precision supports negative numbers 2018-06-20 13:05:37 +02:00
Gunnar Morling
f4e3668ebf DBZ-20 Tune up timeouts/connector/oracle/OracleStreamingChangeEventSource.java 2018-06-20 13:05:37 +02:00
Gunnar Morling
cf6219ccda DBZ-20 Committing processed offsets in the DB, allowing log files to be released 2018-06-20 13:05:37 +02:00
Jiri Pechanec
661c34fc6b DBZ-20 Use fluent API in ColumnEditor 2018-06-20 13:05:37 +02:00
Jiri Pechanec
232a1d5573 DBZ-20 Support for day to second interval 2018-06-20 13:05:37 +02:00
Jiri Pechanec
89694ffbbf DBZ-20 Support for year to month interval 2018-06-20 13:05:37 +02:00
Jiri Pechanec
28b844bf99 DBZ-20 Added ANSI double precision, real and variable scale number datatypes 2018-06-20 13:05:37 +02:00
Jiri Pechanec
10ddb69dea DBZ-20 A subset of string, time and number datatypes 2018-06-20 13:05:37 +02:00
Gunnar Morling
603f02e70d DBZ-20 Excluding generated sources from CheckStyle 2018-06-20 13:05:37 +02:00
Gunnar Morling
e8614c7890 DBZ-20 Adding table whitelist/blacklist connector options; 
* Restructuring OracleSnapshotChangeEventSource into smaller methods
 2018-06-20 13:05:37 +02:00
Gunnar Morling
1e87fae023 DBZ-20 Recovering database history 2018-06-20 13:05:37 +02:00
Gunnar Morling
0733110696 DBZ-20 Recording schema history; 
This records the DDL for DDL events captured during streaming. For the
initial schema snapshot, a JSON-style representation of the captured
Table objects is used in a new field of HistoryRecord, as the DDL
returned by dbms_metadata.get_ddl() isn't fully parseable by our
grammar.
 2018-06-20 13:05:37 +02:00
Gunnar Morling
1a7786d254 DBZ-20 Further simplifying threading model and stopping logic 2018-06-20 13:05:37 +02:00
Gunnar Morling
d8a9865e51 DBZ-20 Some more docs 2018-06-20 13:05:37 +02:00
Gunnar Morling
225c0f05a6 DBZ-20 Extracting RelationalChangeRecordEmitter base class 2018-06-20 13:05:37 +02:00
Gunnar Morling
979c60d956 DBZ-20 Handling update events 2018-06-20 13:05:37 +02:00
Gunnar Morling
24e22f4985 DBZ-20 Simpler and more robust interruption handling in ChangeEventSourceCoordinator 2018-06-20 13:05:37 +02:00
Gunnar Morling
d7e196a18e DBZ-20 Initial import of Oracle connector based on XStream 2018-06-20 13:05:37 +02:00
Gunnar Morling
9eb4b90ec9 DBZ-191 Adding record validations 2018-06-15 20:23:17 +02:00
Gunnar Morling
84dd36df46 DBZ-252 Misc typo fixes and clean-up 2018-06-15 16:43:30 +02:00
Gunnar Morling
54ca30624d DBZ-252 Misc. improvements; 
* Dedicated getter for DDL mode
* Using Objects#equal()
* Typo fixes
 2018-06-15 16:43:29 +02:00
Jiri Pechanec
9580c8c290 DBZ-252 Rebase to master 2018-06-15 11:42:24 +02:00
rkuchar
7d7f740721 DBZ-252 refactor parser listener 2018-06-15 11:42:24 +02:00
rkuchar
5c9e718436 DBZ-252 uncomment getting ddl parser instance from configuration + improve/refactor dataType resolver for ANTLR 2018-06-15 11:42:24 +02:00
rkuchar
99eac9a8e2 DBZ-252 some test fixes + separate ddl test for antlr parser 2018-06-15 11:42:24 +02:00
rkuchar
22151250fa DBZ-252 refactor proxy listener delegation 2018-06-15 11:42:24 +02:00
rkuchar
07212043b5 DBZ-252 test fixes + grammar fixes + some bug repairs in antlr parser 2018-06-15 11:42:23 +02:00
rkuchar
1124f81d94 DBZ-252 parse set and use statements 2018-06-15 11:42:23 +02:00
rkuchar
b4480c55b7 DBZ-252 parse database queries (create, alter, delete), parse name tokens without quotes, add new Mysql connector configuration field for choosing ddl parsing mode 2018-06-15 11:42:23 +02:00
rkuchar
5d911d511f DBZ-252 signal event for truncate table into schema changes 2018-06-15 11:42:23 +02:00
rkuchar
32d181480d DBZ-252 fix troubles with generalized system variables + add new data types that was missing in grammar 2018-06-15 11:42:23 +02:00
rkuchar
85e8146d1f DBZ-252 implementation of scope synonym for local and session scope in MySQL 2018-06-15 11:42:23 +02:00
rkuchar
5851d1f07d DBZ-252 fix initialization of system variables object + add default scope for variales in case that scope will no be set 2018-06-15 11:42:23 +02:00
rkuchar
9311c82bc5 DBZ-252 reformat code to not have anything after closing "}" 2018-06-15 11:42:23 +02:00
rkuchar
782ab75160 DBZ-252 move listener support from base to only legacy implementation, refactor system variables for possible use with other DBMSs + introduce data type resolver for antlr mysql parser 2018-06-15 11:42:23 +02:00
rkuchar
b92e8ef760 DBZ-252 exclude logging of parsed comment 2018-06-15 11:42:23 +02:00
rkuchar
30e158aa71 DBZ-252 create table and drop table listeners 2018-06-15 11:42:23 +02:00
rkuchar
785c346bcb DBZ-252 Exclude reusable logic from LegacyDdlParser 2018-06-15 11:42:23 +02:00
rkuchar
36e2f33bf6 DBZ-252 - extract creation of DdlParser interface 2018-06-15 11:42:23 +02:00
Jiri Pechanec
29f8891b4f DBZ-578 Process timestamps without timezones as in server timezone 2018-06-11 15:27:08 +02:00
Jiri Pechanec
de0e1ad4b5 DBZ-702 Mask SASL configuration 2018-06-08 08:57:50 +02:00
Jiri Pechanec
176dcbbc0b DBZ-709 PostgreSQL uses scale for time precision not length 2018-06-05 15:51:35 +02:00
luobo
5b33d81f00 DBZ-191 support default value of column in mysql 2018-06-04 10:43:53 +02:00
Gunnar Morling
61f576bb90 DBZ-706 Misc. clean-up 2018-06-01 11:23:30 +02:00
Stephen Powis
40d872f48c DBZ-706 General cleanup 2018-06-01 11:23:30 +02:00
Stephen Powis
867202e0ca DBZ-706 Slight refactoring / shuffling of code to validate source query field in snapshot tests 2018-06-01 11:23:12 +02:00
Jiri Pechanec
764a1d0d58 DBZ-693 SQL parsing in separate method 2018-05-28 12:35:47 +02:00
Jiri Pechanec
45bc7b1536 DBZ-693 On connect statements supported for PostgreSQL 2018-05-28 12:35:47 +02:00
Jiri Pechanec
f74a9fe059 DBZ-693 Initial SQL statements 2018-05-28 12:35:47 +02:00
Jiri Pechanec
f844054f50 DBZ-696 Support nanoseconds in timestamp data 2018-05-22 11:33:50 +02:00
Jiri Pechanec
09b28c0913 DBZ-694 Support milliseconds for pre-epoch dates 2018-05-22 11:22:06 +02:00
Rao
15a90548fb DBZ-666 Supporting ordered snapshot using tables.whitelist config 2018-05-17 06:23:31 +02:00
Jiri Pechanec
0f0a5d4cd4 DBZ-687 Kafka 1.1.0 2018-05-14 09:37:33 +02:00
Omar Al-Safi
8ef9267808 DBZ-668 Added unit test and changed the key field name 2018-03-23 10:50:42 +01:00
Omar Al-Safi
21d2e0b8a9 DBZ-668 Changed the key schema for the heartebat messages from STRING to STRUCT 2018-03-23 10:50:42 +01:00
Jiri Pechanec
2b5370de1a DBZ-667 Sceintific numbers parsing 2018-03-20 14:42:55 +01:00
Gunnar Morling
00b63bb808 DBZ-669 Closing AdminClient instance after usage 2018-03-20 13:07:42 +01:00
Jenkins user
f4e151b23a [maven-release-plugin] prepare for next development iteration 2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332 [maven-release-plugin] prepare release v0.7.5 2018-03-20 08:14:19 +00:00
Gunnar Morling
5f61cb167b DBZ-655 Using AssertJ asserts in UnwrapFromEnvelopTest 2018-03-19 09:28:30 +01:00
Gunnar Morling
5358167446 DBZ-655 Adding test 2018-03-19 09:28:30 +01:00
Maciej Bryński
d5abb8415c DBZ-655 Use cache to avoid using regexp replace 2018-03-19 09:28:23 +01:00
Gunnar Morling
d73c27d9bd DBZ-657 Adding test 2018-03-18 18:26:27 +01:00
Jiri Pechanec
87bd3e9c2f DBZ-657 Handle JSON with control chars not according to specs 2018-03-18 18:03:56 +01:00
Jiri Pechanec
9884a5ca1d DBZ-663 disable cleaning of history topic based on size 2018-03-16 16:14:12 +01:00
Jiri Pechanec
62c7fe12f9 Do not log FLUSH RELAY LOGS statements 2018-03-16 09:51:48 +01:00
Jenkins user
daf27207be [maven-release-plugin] prepare for next development iteration 2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928 [maven-release-plugin] prepare release v0.7.4 2018-03-07 08:31:07 +00:00
Jiri Pechanec
327085164e DBZ-611 Special values supported for double 2018-03-06 16:55:24 +01:00
Gunnar Morling
1f385c21f9 DBZ-611 Typo fix and formatting 2018-03-06 16:55:24 +01:00
Jiri Pechanec
424aefbbb4 DBZ-611 Decimal-> String supported for MySQL 2018-03-06 16:55:24 +01:00
Gunnar Morling
401cbae852 DBZ-632 Removing methods from HistoryRecord which where only used in tests; 
Also adding a new test for HistoryRecord (de-)serialization
 2018-03-02 05:28:16 +01:00
Gunnar Morling
d040ec4427 DBZ-351, DBZ-606 Misc. clean-up; 
* Renaming DebeziumDecimal to SpecialValueDecimal
* Simplifying logic in PostgresValueConverter
 2018-02-27 16:36:21 +01:00
Jiri Pechanec
2534696b21 DBZ-606 Exception thrown for special values when mode handling is not string 2018-02-27 16:36:21 +01:00
Jiri Pechanec
5827a7ac1b DBZ-611 Decimals can be encoded as strings 2018-02-27 16:36:21 +01:00
Jiri Pechanec
c71707afc9 DBZ-351 wal2json usses unlimited precision too 2018-02-27 16:36:21 +01:00
Jiri Pechanec
0c0ac72568 DBZ-606 Type for handling special BigDecimals 2018-02-27 16:36:21 +01:00
Jiri Pechanec
ceaa7f8efe DBZ-351 Numeric is passed with unlimited precision 2018-02-27 16:36:21 +01:00
Gunnar Morling
d89f4e91c6 DBZ-632 Removing unused method parameter 2018-02-23 21:54:41 +01:00
Gunnar Morling
eef1ad7c2a DBZ-630 Refactoring around MongoDB task context; 
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
 2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8 DBZ-630 Pulling up getClock() to ConnectorTaskContext 2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6 DBZ-630 Unifying common start-up logic across connectors 2018-02-21 12:04:41 +01:00
Gunnar Morling
7fe45db831 DBZ-626 Renaming AvroValidator to SchemaNameAdjuster; 
Making apparent that this functionality doesn't solely validate but returns a new name actually
 2018-02-20 09:26:47 +01:00
Gunnar Morling
cddd5fcc8a DBZ-626 Passing schema validator as AvroValidator instance instead of Function 2018-02-20 09:26:47 +01:00
Jiri Pechanec
6e00924ee9 DBZ-625 Fix parsing of float without decimals 2018-02-20 08:47:43 +01:00
Gunnar Morling
b5856f37c7 DBZ-628 Unrelated typo fix 2018-02-20 06:34:42 +01:00
Gunnar Morling
8369f10d52 DBZ-628 Adding MAX_QUEUE_SIZE, MAX_BATCH_SIZE and POLL_INTERVAL_MS to CommonConnectorConfig; 
Also using these options in PG connector
 2018-02-20 06:34:42 +01:00
Gunnar Morling
5bc62dc862 DBZ-620 Removing method only used in tests 2018-02-19 12:18:37 +01:00
Gunnar Morling
ab11f48843 DBZ-620 Creating envelope schemas only once per table type 2018-02-19 12:18:37 +01:00
Jenkins user
6d0cd88e12 [maven-release-plugin] prepare for next development iteration 2018-02-15 04:15:34 +00:00
Jenkins user
7d1e1a989e [maven-release-plugin] prepare release v0.7.3 2018-02-15 04:15:34 +00:00
Gunnar Morling
b1362f9db9 DBZ-278 Extracting broker config retrieval into its own method 2018-02-13 17:12:11 +01:00
Jiri Pechanec
90e4aa0dea DBZ-278 Fail if topic is not created 2018-02-13 16:48:42 +01:00
Jiri Pechanec
4541fc53c9 DBZ-278 Explicit history storage initialization 2018-02-13 16:48:42 +01:00
Jiri Pechanec
ad7b92310e DBZ-278 Test for broker without auto-created topics 2018-02-13 16:48:42 +01:00
Jiri Pechanec
55e81c458b DBZ-278 Create database history topic 2018-02-13 16:48:42 +01:00
Jiri Pechanec
5a0c7c07b0 DBZ-609 DBZ-571 Refactor Postgres type handling 
TypeRegistry introduced for Postgres connector

JDBC column does not have a special componentType

JDBC column provide a database specific type id

OID is a primary type identifier to be used in Postgres connector code - dropping JDB/OID dichotomy
 2018-02-13 14:54:42 +01:00
Jiri Pechanec
dbac1429b2 DBZ-581 Improve error handling for replicators 2018-02-13 14:40:54 +01:00
Jiri Pechanec
ecdce6529d DBZ-220 Null Heartbeat object. Messages contains key 2018-02-13 12:26:48 +01:00
Jiri Pechanec
682cabb84a DBZ-220 Generate heartbeat events in binlog thread 2018-02-13 12:24:51 +01:00
Gunnar Morling
96b4023839 DBZ-220 Misc. improvements; 
* interpreting heartbeat interval as MS
* adding Configuration#getDuration()
 2018-02-13 12:24:51 +01:00
Jiri Pechanec
3c1ea8206b DBZ-220 Refactor heartbeat to a separate class 2018-02-13 12:24:51 +01:00
Gunnar Morling
2e0b9e5e33 DBZ-582 Moving TOMBSTONES_ON_DELETE to CommonsConnectorConfig for re-use 2018-02-13 10:19:18 +01:00
Jiri Pechanec
d0bd1560ff DBZ-576 Make DDL filter internal parameter 2018-02-12 11:22:28 +01:00
Jiri Pechanec
3740aee4e8 DBZ-576 Internal parameters 2018-02-12 11:22:28 +01:00
Gunnar Morling
a77c8d733a DBZ-580, DBZ-586 Making enqueue() and thus snapshots in Postgres interruptable; 
* ChangeEventQueue#enqueue() checks the interrupted state of the calling
thread now, raising an InterruptedException in case the interrupted flag
has been set (because the producer's thread executor has been stopped)
* RecordSnapshotProducer has been adjusted to check for the interrupted
regularly, aborting if it has been set
 2018-02-11 10:05:06 +01:00
Gunnar Morling
45a9847d42 DBZ-580 Initial implementation of dedicated change message queue: 
* To be used across connectors, unifying the enqueuing/polling logic and handling of related config options
* Using it for Postgres connector
 2018-02-11 10:05:06 +01:00
Gunnar Morling
2a724d9611 DBZ-537 Misc. adjustments: 
* Renaming ConfigurationHelper to Instantiator
* Doc improvements and typo fixes
* Bringing getInstance() methods into consistent order
* Raising exception instead of logging error if instantion fails
 2018-02-09 15:45:57 +01:00
Jiri Pechanec
5be55ae8ff DBZ-537 OffsetCommitPolicy supports Configuration constructor initialization 2018-02-09 15:19:48 +01:00
Gunnar Morling
7b6db00db7 DBZ-593 JavaDoc 2018-02-02 13:16:43 +01:00
Gunnar Morling
712824e43f DBZ-593 Passing version explicitly from SourceInfos to base class instead of relying specific packaging structure 2018-02-02 13:16:43 +01:00
Jiri Pechanec
7ebee94169 DBZ-593 Add Debezium version to source in envelope 2018-02-02 13:16:43 +01:00
Gunnar Morling
3a281d1185 DBZ-595 Making sure resources are cleaned up when snapshotting fails; 
* shutting down the snapshotting thread and the DB history producer client
  if the connector is stopped while trying to write to the history topic
* reducing the time that KafkaProducer#send() will block if Kafka isn't
  available; this will release the producer thread quicker in case the
  connector is stopped during snapshotting
* not returning from finally block (!) in case the TX is rolled back; This
  prevented the failed state to be set by the outer catch clause in execute()
 2018-02-02 09:06:32 +01:00
Jiri Pechanec
92740a3626 DBZ-587 Fix thread leak, thread names 2018-02-01 10:04:20 +01:00
Jiri Pechanec
9b592204ac DBZ-587 Centralize and unify thread management 2018-02-01 10:04:20 +01:00
Jenkins user
04624341f5 [maven-release-plugin] prepare for next development iteration 2018-01-25 09:39:44 +00:00
Jenkins user
898f6884e1 [maven-release-plugin] prepare release v0.7.2 2018-01-25 09:39:44 +00:00
Gunnar Morling
8edcf9f3d8 DBZ-507 Making "wkb" optional in Point schema to keep compatability with previous version 2018-01-24 16:51:54 +01:00
Robert Coup
f9d90a482f DBZ-507 Expanding support for geometry types; 
* Adding support for PostGIS geometry types
* Adding support for GEOMETRY, POLYGON and more in MySQL
* For all newly supported types, changes are represented using two new schema types Geometry and Geography, containing the WKB (binary geo data) ans srid (coord system identifier)
* The existing Point type also contains the new (optional) srid field
 2018-01-24 16:51:49 +01:00
Jiri Pechanec
01577b40c3 DBZ-541 Only whitelisted tables are recorded in history 2018-01-24 14:43:57 +01:00