alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
1,700 Commits 3 Branches 0 Tags 51 MiB
7524b1ec89
Commit Graph

510 Commits

Author SHA1 Message Date
Gunnar Morling
39a47fabfa DBZ-899 Using custom statement to obtain SQL type info in order to avoid N+1 issues with PgDatabaseMetaData#getTypeInfo() 2018-09-19 09:51:38 +02:00
Gunnar Morling
3331d4c1a9 DBZ-899 Removing unused field; 
Also inlining method only used in one place.
 2018-09-19 09:51:38 +02:00
Jiri Pechanec
2253eaa6a6 DBZ-819 JdbcConnection is not thread safe 2018-09-18 09:05:36 +02:00
Jiri Pechanec
05ce2e3063 DBZ-819 Cache prepared statements in connection 2018-09-18 09:05:36 +02:00
jchipmunk
b0feaf496a DBZ-633 Adding "field.blacklist" property for MongoDB connector 
The "field.blacklist" configuration property is an optional comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form "databaseName.collectionName.fieldName.nestedFieldName", where "databaseName" and "collectionName" may contain the wildcard (*) which matches any characters.

Although the "field.blacklist" configuration property allows you to remove fields from the event values, the "_id" field is always included in the event’s key.
 2018-09-14 12:54:11 +02:00
Gunnar Morling
5878e5105f
DBZ-878 Using LRU eviction strategy 
[As recommended](4961277637 (commitcomment-30412629)) by Ben Manes, the LIRS implementation seems to provide no advantage and isn't well tested.
 2018-09-06 11:21:31 +02:00
Gunnar Morling
4961277637 DBZ-878 Making topic name cache size bound; 
* Imported BoundedConcurrentHashMap from Hibernate ORM as a thread-safe concurrent cache.
 2018-09-05 12:56:53 +02:00
Gunnar Morling
e1bbbd7992 DBZ-865 DBZ-878 Replacing invalid topic name characters with "_"; 2018-09-05 12:56:53 +02:00
Gunnar Morling
eba134817f DBZ-894 Making TableIdToStringMapper dedicated functional interface with specifically named method 2018-09-04 12:43:50 +02:00
Jiri Pechanec
1373ac8475 DBZ-894 Introduce explicit mapping functional type 2018-09-04 12:39:10 +02:00
Jiri Pechanec
3e0648a42c DBZ-894 Customize TableId comparison per-connector in filters 2018-09-04 09:18:15 +02:00
Gunnar Morling
985999b4af DBZ-872 Making MySqlAntlrDdlParser#tableFilter final 2018-08-24 08:12:54 +02:00
Jiri Pechanec
63ee65bb7d DBZ-872 Honor filters in ALTER TABLE parsing 2018-08-24 08:12:54 +02:00
Jiri Pechanec
82a93cdac5 DBZ-859 Heartbeat is sent after pipeline snapshot 2018-08-23 11:44:53 +02:00
Jiri Pechanec
985e03e7b5 DBZ-859 Force heartbeat after snapshot completion 2018-08-23 11:44:53 +02:00
Maciej Bryński
947abf9815 DBZ-857 Ability to rewrites deleted records 2018-08-14 10:37:39 +02:00
artiship
57fa31e1bc DBZ-854 Correct param name for excludeColumns(String fullyQualifiedTableNames) 2018-08-14 08:39:28 +02:00
LiuHanlin
41e16f4a4f DBZ-853 Fix kafka database history storage miscount attemp number even if there are more records to consume 2018-08-13 20:26:48 +02:00
Jenkins user
e00dad127f [maven-release-plugin] prepare for next development iteration 2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700 [maven-release-plugin] prepare release v0.9.0.Alpha1 2018-07-26 08:00:12 +00:00
Gunnar Morling
3061c1dc0b DBZ-815 Passing offset to Heartbeat#heartbeat(); 
That's simpler to grasp than the approach of passing a supplier lambda to the constructor.
Also it allows to pass on the offset via local variables instead of instance fields in some cases.
 2018-07-25 14:49:52 +02:00
Gunnar Morling
0a659c0159 DBZ-815 Ensuring latest offset is propagated also if change events relate to non-whitelisted tables 2018-07-25 14:49:52 +02:00
Gunnar Morling
fa68c1a158 DBZ-816 Preparing code for database history set-up for re-use 2018-07-25 11:54:15 +02:00
Jiri Pechanec
e86ddc0ab2 DBZ-40 Add blocking version of MultiResultSetConsumer 2018-07-25 11:51:09 +02:00
Gunnar Morling
bd4f3d60fa DBZ-804 Simplifying construction of VariableScaleDecimal 2018-07-24 20:48:52 +02:00
Gunnar Morling
98c329aabe DBZ-40 Adding SnapshottingTask#toString() 2018-07-24 18:53:32 +02:00
Gunnar Morling
0362b333bf DBZ-804 Better schema comparison in tests; preparing byte conversion 2018-07-24 05:18:52 +02:00
Gunnar Morling
9814a5c82c DBZ-803 Finalizing snapshot in offsets also for schema-only snapshots 2018-07-23 06:44:04 +02:00
Gunnar Morling
2905cc498b DBZ-800 Don't expect that partition is a Map<String, String> 2018-07-20 13:37:54 +02:00
Gunnar Morling
fe53360f30 DBZ-800 Avoiding "mysql" in commonly used heartbeat key schema 2018-07-20 13:37:54 +02:00
Gunnar Morling
86a4731587 DBZ-800 Emitting heartbeat events in generic event dispatcher 2018-07-20 13:37:54 +02:00
Gunnar Morling
26106cdd8e DBZ-800 Removing unused method from Heartbeat 2018-07-20 13:37:54 +02:00
Jiri Pechanec
0e35439019 DBZ-40 Snapshotting and connector restart 2018-07-18 14:39:11 +02:00
Gunnar Morling
a5b5e7d9eb DBZ-627 Using unfified TopicSelector for MongoDB, too 2018-07-18 12:36:07 +02:00
Gunnar Morling
20db9299c5 DBZ-627 Unifying TopicSelector implementations 2018-07-18 12:36:07 +02:00
Gunnar Morling
610da70cc5 DBZ-801 Parameterizing ChangeEventQueue<DataChangeEvent> 2018-07-18 10:30:22 +02:00
Gunnar Morling
80085567ae DBZ-720 Proper check for previous unfinished snapshot 2018-07-18 10:07:50 +02:00
Gunnar Morling
33170d4c3d DBZ-40 Pulling up some common code to HistorizedRelationalDatabaseSchema 2018-07-18 10:07:50 +02:00
Gunnar Morling
422a303f87 DBZ-720 Signaling cancellation of snapshot through InterruptedException 2018-07-17 13:11:02 +02:00
Jiri Pechanec
f118e37da0 DBZ-40 Changes to support SQL Server connector 2018-07-13 22:23:00 +02:00
Gunnar Morling
c5e2d5f9e7 DBZ-720 Making scan of single table interruptable; 
Also better logging.
 2018-07-13 19:42:18 +02:00
Gunnar Morling
648d695c08 DBZ-720 Preparing HistorizedRelationalSnapshotChangeEventSource to support data snapshotting 2018-07-13 19:42:18 +02:00
Gunnar Morling
aa1272cc21 DBZ-720 Re-using single instance of streaming event receiver in EventDispatcher 2018-07-13 09:51:15 +02:00
Gunnar Morling
657650c581 DBZ-720 Passing change event emitter directly to dispatch methods; 
The indirection of going through a supplier wasn't really necessary
 2018-07-13 09:51:15 +02:00
Gunnar Morling
a565361e56 DBZ-720 Making SnapshotContext auto-closeable 2018-07-13 09:51:15 +02:00
Gunnar Morling
986df43649 DBZ-720 Extracting HistorizedRelationalSnapshotChangeEventSource from Oracle connector 2018-07-13 09:51:15 +02:00
Gunnar Morling
1523f230ca DBZ-793 Using TableFilter consistently; 
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
 2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8 [maven-release-plugin] prepare for next development iteration 2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f [maven-release-plugin] prepare release v0.8.0.Final 2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d [maven-release-plugin] prepare for next development iteration 2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0 [maven-release-plugin] prepare release v0.8.0.CR1 2018-07-04 07:07:44 +00:00
Gunnar Morling
99048365c9 DBZ-771 Unsetting default value when changing a column's type 2018-07-03 11:55:20 +02:00
Gunnar Morling
c7d5288d40 DBZ-771 Making sure causing DDL statement is logged also if default value can't be parsed 2018-07-03 11:55:20 +02:00
Gunnar Morling
dcd6b01134 DBZ-644 Simplifying RelationalDatabaseSchema constructor; 
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
 2018-07-03 07:03:59 +02:00
Gunnar Morling
b0980b994c DBZ-644 Adding test for Postgres 2018-07-03 07:03:59 +02:00
Gunnar Morling
068aa85bd6 DBZ-644 Misc. clean-up and tests; 
* Using parameter keys under "__debezium" namespace
* More expressive names
* Adding tests
 2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema 2018-07-03 07:03:59 +02:00
orr.ganani
ac515b8064 DBZ-644 change Original Data Type mapper and add a test 2018-07-03 07:03:59 +02:00
orr.ganani
747184c572 DBZ-644 add JDBC data type as additional metadata to DBZ events, if configuration is opted in 2018-07-03 07:03:59 +02:00
Gunnar Morling
c4c8cbc3ab DBZ-773 Moving management of Tables to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab DBZ-773 Moving management of table schemas to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
a292b05a96 DBZ-751 Propagating DECIMAL column precision to Avro schemas 2018-06-27 14:48:59 +02:00
Gunnar Morling
b27421ddd0 DBZ-759 Testing via SchemaUtil; 
* Making RecordWriter private
* Removing quotes around string representation of byte arrays
 2018-06-26 12:03:53 +02:00
Andreas Bergmeier
82bac78ba8 DBZ-759 Add Test to ensure correct serialization of byte arrays 
Needed to make RecordWriter public to have access in tests.
 2018-06-26 11:50:54 +02:00
Andreas Bergmeier
71abca2afb DBZ-759 Fix unusable array information in Serialization 
According to ErrorProne, (implicitly) calling toString on the array does
not give useful information.
Also applying Base64 encoding is strange here.
 2018-06-26 11:50:54 +02:00
Gunnar Morling
8d0d35762e DBZ-759 Removing superfluous (and broken) ColumnEditorImpl#compareTo() implementation 2018-06-25 17:04:45 +02:00
Gunnar Morling
bf7a5018ca Setting POM version back to 0.8.0-SNAPSHOT 2018-06-22 12:21:03 +02:00
Gunnar Morling
357102158a DBZ-759 Fixing broken Document#setArray() method 2018-06-22 11:03:00 +02:00
Gunnar Morling
ce81f37990 DBZ-759 Adding unit test 2018-06-22 09:19:45 +02:00
Andreas Bergmeier
7417baea4d DBZ-759 Fix equality comparison for BinaryValue 
If value is binary or string, it should be better to compare the content
of the actual byte arrays.
According to ErrorProne you use reference equality when calling equals on
an array.

Part of https://issues.jboss.org/browse/DBZ-759
 2018-06-22 09:19:45 +02:00
Andreas Bergmeier
333d6cb57d DBZ-759 Fix hashCode of BinaryValue 
According to ErrorProne, the hashCode of a byte array returns useless
information.

Part of https://issues.jboss.org/browse/DBZ-759
 2018-06-22 09:19:41 +02:00
Andreas Bergmeier
d374ca0464 DBZ-759 Fix ColumnImpl.scale equality check 
According to ErrorProne, checking Optional via operator == leads to
reference equality check. Probably want rather to check for contained
values.
 2018-06-22 09:08:27 +02:00
Jenkins user
db42e4657a [maven-release-plugin] prepare for next development iteration 2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99 [maven-release-plugin] prepare release v0.8.0.Beta1 2018-06-21 14:07:45 +00:00
Gunnar Morling
96c59a1568 DBZ-20 Moving debezium-connector-oracle from main to incubator repo 2018-06-20 13:05:37 +02:00
Gunnar Morling
b83f1493a2 DBZ-20 Avoiding Optional as method parameter 2018-06-20 13:05:37 +02:00
Gunnar Morling
dce1aa29f1 DBZ-20 Oracle precision supports negative numbers 2018-06-20 13:05:37 +02:00
Gunnar Morling
f4e3668ebf DBZ-20 Tune up timeouts/connector/oracle/OracleStreamingChangeEventSource.java 2018-06-20 13:05:37 +02:00
Gunnar Morling
cf6219ccda DBZ-20 Committing processed offsets in the DB, allowing log files to be released 2018-06-20 13:05:37 +02:00
Jiri Pechanec
661c34fc6b DBZ-20 Use fluent API in ColumnEditor 2018-06-20 13:05:37 +02:00
Jiri Pechanec
232a1d5573 DBZ-20 Support for day to second interval 2018-06-20 13:05:37 +02:00
Jiri Pechanec
89694ffbbf DBZ-20 Support for year to month interval 2018-06-20 13:05:37 +02:00
Jiri Pechanec
28b844bf99 DBZ-20 Added ANSI double precision, real and variable scale number datatypes 2018-06-20 13:05:37 +02:00
Jiri Pechanec
10ddb69dea DBZ-20 A subset of string, time and number datatypes 2018-06-20 13:05:37 +02:00
Gunnar Morling
603f02e70d DBZ-20 Excluding generated sources from CheckStyle 2018-06-20 13:05:37 +02:00
Gunnar Morling
e8614c7890 DBZ-20 Adding table whitelist/blacklist connector options; 
* Restructuring OracleSnapshotChangeEventSource into smaller methods
 2018-06-20 13:05:37 +02:00
Gunnar Morling
1e87fae023 DBZ-20 Recovering database history 2018-06-20 13:05:37 +02:00
Gunnar Morling
0733110696 DBZ-20 Recording schema history; 
This records the DDL for DDL events captured during streaming. For the
initial schema snapshot, a JSON-style representation of the captured
Table objects is used in a new field of HistoryRecord, as the DDL
returned by dbms_metadata.get_ddl() isn't fully parseable by our
grammar.
 2018-06-20 13:05:37 +02:00
Gunnar Morling
1a7786d254 DBZ-20 Further simplifying threading model and stopping logic 2018-06-20 13:05:37 +02:00
Gunnar Morling
d8a9865e51 DBZ-20 Some more docs 2018-06-20 13:05:37 +02:00
Gunnar Morling
225c0f05a6 DBZ-20 Extracting RelationalChangeRecordEmitter base class 2018-06-20 13:05:37 +02:00
Gunnar Morling
979c60d956 DBZ-20 Handling update events 2018-06-20 13:05:37 +02:00
Gunnar Morling
24e22f4985 DBZ-20 Simpler and more robust interruption handling in ChangeEventSourceCoordinator 2018-06-20 13:05:37 +02:00
Gunnar Morling
d7e196a18e DBZ-20 Initial import of Oracle connector based on XStream 2018-06-20 13:05:37 +02:00
Gunnar Morling
9eb4b90ec9 DBZ-191 Adding record validations 2018-06-15 20:23:17 +02:00
Gunnar Morling
84dd36df46 DBZ-252 Misc typo fixes and clean-up 2018-06-15 16:43:30 +02:00
Gunnar Morling
54ca30624d DBZ-252 Misc. improvements; 
* Dedicated getter for DDL mode
* Using Objects#equal()
* Typo fixes
 2018-06-15 16:43:29 +02:00
Jiri Pechanec
9580c8c290 DBZ-252 Rebase to master 2018-06-15 11:42:24 +02:00
rkuchar
7d7f740721 DBZ-252 refactor parser listener 2018-06-15 11:42:24 +02:00
rkuchar
5c9e718436 DBZ-252 uncomment getting ddl parser instance from configuration + improve/refactor dataType resolver for ANTLR 2018-06-15 11:42:24 +02:00
rkuchar
99eac9a8e2 DBZ-252 some test fixes + separate ddl test for antlr parser 2018-06-15 11:42:24 +02:00
rkuchar
22151250fa DBZ-252 refactor proxy listener delegation 2018-06-15 11:42:24 +02:00
rkuchar
07212043b5 DBZ-252 test fixes + grammar fixes + some bug repairs in antlr parser 2018-06-15 11:42:23 +02:00
rkuchar
1124f81d94 DBZ-252 parse set and use statements 2018-06-15 11:42:23 +02:00
rkuchar
b4480c55b7 DBZ-252 parse database queries (create, alter, delete), parse name tokens without quotes, add new Mysql connector configuration field for choosing ddl parsing mode 2018-06-15 11:42:23 +02:00
rkuchar
5d911d511f DBZ-252 signal event for truncate table into schema changes 2018-06-15 11:42:23 +02:00
rkuchar
32d181480d DBZ-252 fix troubles with generalized system variables + add new data types that was missing in grammar 2018-06-15 11:42:23 +02:00
rkuchar
85e8146d1f DBZ-252 implementation of scope synonym for local and session scope in MySQL 2018-06-15 11:42:23 +02:00
rkuchar
5851d1f07d DBZ-252 fix initialization of system variables object + add default scope for variales in case that scope will no be set 2018-06-15 11:42:23 +02:00
rkuchar
9311c82bc5 DBZ-252 reformat code to not have anything after closing "}" 2018-06-15 11:42:23 +02:00
rkuchar
782ab75160 DBZ-252 move listener support from base to only legacy implementation, refactor system variables for possible use with other DBMSs + introduce data type resolver for antlr mysql parser 2018-06-15 11:42:23 +02:00
rkuchar
b92e8ef760 DBZ-252 exclude logging of parsed comment 2018-06-15 11:42:23 +02:00
rkuchar
30e158aa71 DBZ-252 create table and drop table listeners 2018-06-15 11:42:23 +02:00
rkuchar
785c346bcb DBZ-252 Exclude reusable logic from LegacyDdlParser 2018-06-15 11:42:23 +02:00
rkuchar
36e2f33bf6 DBZ-252 - extract creation of DdlParser interface 2018-06-15 11:42:23 +02:00
Jiri Pechanec
29f8891b4f DBZ-578 Process timestamps without timezones as in server timezone 2018-06-11 15:27:08 +02:00
Jiri Pechanec
de0e1ad4b5 DBZ-702 Mask SASL configuration 2018-06-08 08:57:50 +02:00
Jiri Pechanec
176dcbbc0b DBZ-709 PostgreSQL uses scale for time precision not length 2018-06-05 15:51:35 +02:00
luobo
5b33d81f00 DBZ-191 support default value of column in mysql 2018-06-04 10:43:53 +02:00
Gunnar Morling
61f576bb90 DBZ-706 Misc. clean-up 2018-06-01 11:23:30 +02:00
Stephen Powis
40d872f48c DBZ-706 General cleanup 2018-06-01 11:23:30 +02:00
Stephen Powis
867202e0ca DBZ-706 Slight refactoring / shuffling of code to validate source query field in snapshot tests 2018-06-01 11:23:12 +02:00
Jiri Pechanec
764a1d0d58 DBZ-693 SQL parsing in separate method 2018-05-28 12:35:47 +02:00
Jiri Pechanec
45bc7b1536 DBZ-693 On connect statements supported for PostgreSQL 2018-05-28 12:35:47 +02:00
Jiri Pechanec
f74a9fe059 DBZ-693 Initial SQL statements 2018-05-28 12:35:47 +02:00
Jiri Pechanec
f844054f50 DBZ-696 Support nanoseconds in timestamp data 2018-05-22 11:33:50 +02:00
Jiri Pechanec
09b28c0913 DBZ-694 Support milliseconds for pre-epoch dates 2018-05-22 11:22:06 +02:00
Rao
15a90548fb DBZ-666 Supporting ordered snapshot using tables.whitelist config 2018-05-17 06:23:31 +02:00
Jiri Pechanec
0f0a5d4cd4 DBZ-687 Kafka 1.1.0 2018-05-14 09:37:33 +02:00
Omar Al-Safi
8ef9267808 DBZ-668 Added unit test and changed the key field name 2018-03-23 10:50:42 +01:00
Omar Al-Safi
21d2e0b8a9 DBZ-668 Changed the key schema for the heartebat messages from STRING to STRUCT 2018-03-23 10:50:42 +01:00
Jiri Pechanec
2b5370de1a DBZ-667 Sceintific numbers parsing 2018-03-20 14:42:55 +01:00
Gunnar Morling
00b63bb808 DBZ-669 Closing AdminClient instance after usage 2018-03-20 13:07:42 +01:00
Jenkins user
f4e151b23a [maven-release-plugin] prepare for next development iteration 2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332 [maven-release-plugin] prepare release v0.7.5 2018-03-20 08:14:19 +00:00
Gunnar Morling
5f61cb167b DBZ-655 Using AssertJ asserts in UnwrapFromEnvelopTest 2018-03-19 09:28:30 +01:00
Gunnar Morling
5358167446 DBZ-655 Adding test 2018-03-19 09:28:30 +01:00
Maciej Bryński
d5abb8415c DBZ-655 Use cache to avoid using regexp replace 2018-03-19 09:28:23 +01:00
Gunnar Morling
d73c27d9bd DBZ-657 Adding test 2018-03-18 18:26:27 +01:00
Jiri Pechanec
87bd3e9c2f DBZ-657 Handle JSON with control chars not according to specs 2018-03-18 18:03:56 +01:00
Jiri Pechanec
9884a5ca1d DBZ-663 disable cleaning of history topic based on size 2018-03-16 16:14:12 +01:00
Jiri Pechanec
62c7fe12f9 Do not log FLUSH RELAY LOGS statements 2018-03-16 09:51:48 +01:00
Jenkins user
daf27207be [maven-release-plugin] prepare for next development iteration 2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928 [maven-release-plugin] prepare release v0.7.4 2018-03-07 08:31:07 +00:00
Jiri Pechanec
327085164e DBZ-611 Special values supported for double 2018-03-06 16:55:24 +01:00
Gunnar Morling
1f385c21f9 DBZ-611 Typo fix and formatting 2018-03-06 16:55:24 +01:00
Jiri Pechanec
424aefbbb4 DBZ-611 Decimal-> String supported for MySQL 2018-03-06 16:55:24 +01:00
Gunnar Morling
401cbae852 DBZ-632 Removing methods from HistoryRecord which where only used in tests; 
Also adding a new test for HistoryRecord (de-)serialization
 2018-03-02 05:28:16 +01:00
Gunnar Morling
d040ec4427 DBZ-351, DBZ-606 Misc. clean-up; 
* Renaming DebeziumDecimal to SpecialValueDecimal
* Simplifying logic in PostgresValueConverter
 2018-02-27 16:36:21 +01:00
Jiri Pechanec
2534696b21 DBZ-606 Exception thrown for special values when mode handling is not string 2018-02-27 16:36:21 +01:00