alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
8,463 Commits 3 Branches 0 Tags 51 MiB
7ff18cb22c
Commit Graph

1316 Commits

Author SHA1 Message Date
Jiri Pechanec
29f6d9dd09 DBZ-1368 Support toasted placeholder in wal2json 2019-09-25 17:24:21 +02:00
Jiri Pechanec
9dd5a16160 DBZ-1368 Support toasted placeholder in protobuf 2019-09-25 17:24:21 +02:00
Gunnar Morling
ae0bf61bb4 DBZ-1367 Handling "datum_missing" marker 2019-09-25 17:24:21 +02:00
Jiri Pechanec
ab31147593 DBZ-1368 Support toasted placeholder in pgoutput 2019-09-25 17:24:21 +02:00
Gunnar Morling
442408be02 DBZ-1497 Using int64 for INTERVAL microseconds mapping 2019-09-25 11:21:40 +03:00
Jiri Pechanec
9b2fcdc13a DBZ-1438 Reduce test runtime 2019-09-24 12:51:48 +02:00
Jiri Pechanec
8f308746d3 DBZ-1438 Interrupt only after interrupt affected operations 2019-09-24 12:51:48 +02:00
Gunnar Morling
c2b66ef2d7 DBZ-1436 Referencing default publication name via constant 2019-09-23 20:36:40 +02:00
Chris Cranford
f446ce6ffd DBZ-1436 Fix postgres connector not honoring configured publication.name setting 2019-09-23 20:33:48 +02:00
Gunnar Morling
008ac7a7e4 DBZ-1426 Misc clean-up; 
* Removing unused method parameters
* Formatting
* Using logger methods with argument placeholders
 2019-09-23 11:34:18 +02:00
trolison
e5f77b3a32 DBZ-1426, DBZ-1427 Introducing retry if replication slot is already in use; 
- Added PG connector config parameters for retry count and delay
- Added basic test for connection retry and delay logic
- Added a more clear error message when a PG connection cant be made and it has nothing to do with the schema registry (DBZ-1427)
- Updated documentation for the new connection retry parameters
 2019-09-23 11:23:38 +02:00
Chris Cranford
130210b5ad DBZ-1044 Add config option sanitize.field.names and FieldNameSelector 2019-09-20 07:40:23 +03:00
Chris Cranford
b569c184da DBZ-1383 Fix PostgreSQL tests that fail intermittently 2019-09-17 11:19:17 +03:00
Chris Cranford
06c7621217 DBZ-1383 Fix PostgreSQL tests that fail intermittently 
* Test dropping slot & publication before each test
 2019-09-17 11:19:17 +03:00
Chris Cranford
278bbad8ae DBZ-1383 Revert commit 8e636685: Fix PostgreSQL tests that fail intermittently 2019-09-17 11:19:17 +03:00
Chris Cranford
fd6450442b DBZ-1383 Fix PostgreSQL tests that fail intermittently 
* Introduced wait steps to OutboxEventRouterIT to check if the problem is timing related
 2019-09-17 11:19:17 +03:00
Chris Cranford
37191ab713 DBZ-1383 Fix PostgreSQL tests that fail intermittently 2019-09-17 11:19:17 +03:00
Chris Cranford
022d0d07c8 DBZ-1383 Fix PostgreSQL tests that fail intermittently 
* Only execute `DROP PUBLICATION` when using the PGOUTPUT logical decoder
 2019-09-17 11:19:17 +03:00
Chris Cranford
99727dea76 DBZ-1383 Fix PostgreSQL tests that fail intermittently 2019-09-17 11:19:17 +03:00
Debezium Builder
9b7e09fdda [maven-release-plugin] prepare for next development iteration 2019-09-10 09:12:12 -04:00
Debezium Builder
a1c2861458 [maven-release-plugin] prepare release v0.10.0.CR1 2019-09-10 09:12:11 -04:00
Renato Mefi
c8c4266e34 DBZ-1457 Fix typo 2019-09-03 22:39:37 +02:00
Gunnar Morling
fe9fc1c0dc DBZ-1255 Adjustments after rebase 2019-08-21 07:50:40 +02:00
Gunnar Morling
08ef1418d9 DBZ-DBZ-1255 Preventing value overflows for TIMESTAMP; 
* Generally avoiding conversion to nano-seconds to accomodate for larger values
* Generally avoiding the usage of long to convey temporal values internally:
 * Instant is used for TIMESTAMP
 * OffsetDateTime for TIMESTAMPTZ
 * Duration for TIME (LocalTime cannot be used as it doesn't support 24:00:00 as possible with Postgres)
 * OffsetTime for TIMETZ
* Avoiding usage of methods under test for calculating expected values in tests
 2019-08-21 07:50:40 +02:00
Gunnar Morling
94eb75257b DBZ-1255 Using Instant for representing timestamps 2019-08-21 07:50:40 +02:00
Ivan Luzyanin
fd5841ab6d fix import 2019-08-21 07:50:40 +02:00
Ivan Luzyanin
fa10cc502d [DBZ-1255,DBZ-1205] Fix for parsing dates with year larger than 9999. Avoid using nonoseconds and parse the time directly to OffsetDateTime. 2019-08-21 07:50:40 +02:00
Chris Cranford
ef30b8a8cf DBZ-1419 Fix timing issue waiting for streaming to start 2019-08-20 18:43:03 +02:00
Chris Cranford
c6f6020da5 DBZ-777 Fix more timing issues with heartbeats 2019-08-20 18:43:03 +02:00
Jiri Pechanec
0e39965c0e DBZ-1419 Override time precision for MySQL 2019-08-20 18:43:03 +02:00
Jiri Pechanec
51c12d930d DBZ-1419 Support for Connect temporal types in SQL Server 2019-08-20 18:43:03 +02:00
Chris Cranford
8504f6aafd DBZ-1437 Fix InitialOnlySnapshotter to properly determine when to run a snapshot. 2019-08-19 22:01:32 +02:00
Chris Cranford
68f0bba7b3 DBZ-1437 Fix ExportedSnapshotter to properly determine when to run a snapshot. 2019-08-19 22:01:32 +02:00
Cyril Scetbon
e7dc1dfb62 DBZ-1440 Use ServerVersion values instead of integers 2019-08-19 14:25:53 -04:00
Cyril Scetbon
135e66dd0d DBZ-1440 Temporary replication slots on PG10+ only 2019-08-19 14:25:53 -04:00
Cyril Scetbon
4b5475391c DBZ-1440 Supports exported snapshots for Pg 9.4+ 2019-08-19 14:25:53 -04:00
Debezium Builder
4ae105bdce [maven-release-plugin] prepare for next development iteration 2019-08-16 03:42:55 -04:00
Debezium Builder
945ccf68cb [maven-release-plugin] prepare release v0.10.0.Beta4 2019-08-16 03:42:54 -04:00
Jiri Pechanec
9c25a192b7 DBZ-777 Fix timing issue with heartbeats 2019-08-16 08:05:48 +02:00
Jiri Pechanec
540021cf68 DBZ-777 Minor fixes 2019-08-15 20:43:31 +02:00
Jiri Pechanec
4d18c70834 DBZ-777 Make sure the empty events are generated while streaming 2019-08-15 20:43:31 +02:00
Gunnar Morling
5ffe8123f1 DBZ-777 Further fixes 2019-08-15 20:43:31 +02:00
Gunnar Morling
63250d0c13 DBZ-777 Misc. adjustments 
* Avoiding repeated parsing of table id
* Making 'inconsistentSchemaHandler' field final in EventDispatcher
* Not exposing synchronizeTableSchema() on SPI level
 2019-08-15 20:43:31 +02:00
Jiri Pechanec
780545d0d2 DBZ-777 Fix wal2json decoder 2019-08-15 20:43:31 +02:00
Jiri Pechanec
5307a5822d DBZ-777 Fix pgoutput decoder 2019-08-15 20:43:31 +02:00
Jiri Pechanec
de2345f87b DBZ-777 Fix wal2json decoder 2019-08-15 20:43:31 +02:00
Jiri Pechanec
037d581844 DBZ-777 Fix heartbeats 2019-08-15 20:43:31 +02:00
Jiri Pechanec
ffb7e6d7d8 DBZ-777 Assertions against internal schema 2019-08-15 20:43:31 +02:00
Jiri Pechanec
857d22bfda DBZ-777 Fix toasted cols test 2019-08-15 20:43:31 +02:00
Jiri Pechanec
8f08f2d505 DBZ-777 Remove unused code 2019-08-15 20:43:31 +02:00
Jiri Pechanec
d72afa8e9b DBZ-777 Skip empty messages 2019-08-15 20:43:31 +02:00
Jiri Pechanec
f4246df6e4 DBZ-777 PostgreSQL connector rewrite to the standard framework 2019-08-15 20:43:31 +02:00
Chris Cranford
89ae4b2151 DBZ-1436 Fix postgres connector not honoring publication.name configuration 2019-08-15 12:00:29 -04:00
Jiri Pechanec
8adb5e48ca DBZ-1385 Elevate payload for only headers setup 2019-08-12 10:36:52 +02:00
Jiri Pechanec
0991c70ac9 DBZ-1385 eventType can be configured as add field 2019-08-12 10:36:52 +02:00
Jiri Pechanec
c07c994434 DBZ-1381 Introduce permanent metadata connection 2019-08-09 11:53:45 +02:00
Chris Cranford
ae58b0b44c DBZ-1035 Checkstyle fixup 2019-07-29 15:49:18 +02:00
Chris Cranford
e852f082a2 DBZ-1035 Support PostgreSQL lock-free snapshot using exported snapshots 2019-07-29 15:49:18 +02:00
Jiri Pechanec
0742e5775f Revert "[maven-release-plugin] prepare release v0.10.0.Beta4" 
This reverts commit c7340e5152.
 2019-07-23 14:54:55 +02:00
Jiri Pechanec
f7d20635ce Revert "[maven-release-plugin] prepare for next development iteration" 
This reverts commit 22879777e4.
 2019-07-23 14:54:49 +02:00
Jiri Pechanec
22879777e4 [maven-release-plugin] prepare for next development iteration 2019-07-23 14:10:03 +02:00
Jiri Pechanec
c7340e5152 [maven-release-plugin] prepare release v0.10.0.Beta4 2019-07-23 14:10:02 +02:00
Debezium Builder
71333fda98 [maven-release-plugin] prepare for next development iteration 2019-07-23 02:55:08 -04:00
Debezium Builder
20d006fac7 [maven-release-plugin] prepare release v0.10.0.Beta3 2019-07-23 02:55:08 -04:00
Jiri Pechanec
81ba1a3adb DBZ-766 Remove autoboxing 2019-07-23 06:22:41 +02:00
Chris Cranford
2f92553d8f DBZ-766 Implement PgOutput logical decoding for Postgres 10+ 2019-07-23 06:22:41 +02:00
Jiri Pechanec
6a13b0e0b5 DBZ-1400 Record with same flush LSN should not be skipped for the first record 2019-07-22 14:17:59 -04:00
Gunnar Morling
0383f9178c DBZ-1238 Removing incorrect comment 2019-07-09 21:14:28 +02:00
Gunnar Morling
6aaa8b183c DBZ-1238 Simplifying logic 2019-07-09 21:14:28 +02:00
Gunnar Morling
bb8d7683e3 DBZ-1238 Avoiding explicit check for slot state; 
This causes issues with PG 10 temporary slots which are always active.
The check isn't really needed anyways, because the server will raise an
exception itself when trying to connect to an already active slot.
 2019-07-09 21:14:28 +02:00
Gunnar Morling
a2e0c7cd2a DBZ-1238 Misc. clean-up; 
* Formatting, typo fixes
* Combining createReplicationSlot() and getSlotCreationResult()
* Removing unused field
 2019-07-09 21:14:28 +02:00
Addison Higham
62328bf137 DBZ-1238 Expose more options to the PG Snapshot SPI 
This commit adds some more options to the Postgres Snapshottter SPI
interface, specifically, the ability to create the query for the start
of a snapshot, as well as the ability to generate a statement to lock
the tables.

Additionally, some rework is done to allow for the usage of an exported
snapshot in conjuction with the SPI taking metadata about a newly
created slot. This also changes how slots are initialized to allow for
the exported snapshot to be referenced, which requires adding a more
granular setup of a ReplicationConnection

Together, these options allow for a lot more flexibility, such as not
taking locks when snapshotting large databases as well as potentially to
use exported snapshots.

A future version should perhaps make some options possible without
implementing the SPI (such as using the exported snapshot by default)
but this commit is just trying to get all the groundwork in place to
allow for the Snapshotter SPI to have this functionality.
 2019-07-09 21:14:28 +02:00
Gunnar Morling
369f74d875 DBZ-1320 Addressing review remarks 2019-07-09 21:07:18 +02:00
Brandon Brown
efff4f0809 DBZ-1320 Adding support for deletion events via new configurable property route.tombstone.on.empty.payload 2019-07-09 21:07:18 +02:00
Gunnar Morling
50e3a02d4d DBZ-1371 README updates 2019-07-09 10:13:21 +02:00
Gunnar Morling
8c5d5021f4 DBZ-1371 Restoring original timeouts 2019-07-09 10:13:21 +02:00
Chris Cranford
faa31b079d DBZ-1371 Added link to RDS.md from README.md 2019-07-09 10:13:21 +02:00
Chris Cranford
bf248e6fb0 DBZ-1371 Update to PostgreSQL connector RDS document 2019-07-09 10:13:21 +02:00
Chris Cranford
af513c2732 DBZ-1371 Derive timeouts from configurable timeout 2019-07-09 10:13:21 +02:00
Chris Cranford
f88970f2e6 DBZ-1371 Fix formatting and typos in RDS documentation 2019-07-09 10:13:21 +02:00
Chris Cranford
051d5a478d DBZ-1371 Added RDS setup documentation for PostgreSQL 2019-07-09 10:13:21 +02:00
Chris Cranford
a50b9847ac DBZ-1371 Fix tests to run more reliably on Amazon RDS 2019-07-09 10:13:21 +02:00
Renato Mefi
c990256fd1 DBZ-1379 Fix PostgreSQL false empty password Warn 2019-07-08 12:51:49 +02:00
Gunnar Morling
e2d8736a79 DBZ-1380 Misc. clean-up; 
* reducing visibility of some methods
* resetting interrupted state
 2019-07-05 21:08:35 +02:00
Jiri Pechanec
5dc4812ffa DBZ-1380 Guard against partially initilized replication slot 2019-07-05 21:08:35 +02:00
Jiri Pechanec
13d638fd43 DBZ-1356 Metrics for schema history (#947) 
* DBZ-1356 Metrics for schema history
 2019-07-02 13:26:17 +02:00
Jiri Pechanec
b1c1c05d1f DBZ-1363 Generate heartbeat periodically not upon event 2019-07-02 12:02:33 +02:00
Gunnar Morling
08c5d36789
Merge pull request #942 from jpechane/stab-lsn-flush-test 
DBZ-1362 Stabilize test executions on CI
 2019-07-01 09:34:24 +02:00
Gunnar Morling
a02daae3c1 DBZ-1362 Comment 2019-07-01 09:34:01 +02:00
Chris Cranford
61f5e15889 DBZ-1366 Avoid creating unnecessary TypeMetadataImpl instances; only init once. 2019-06-28 13:44:53 -04:00
Jiri Pechanec
b546dc886e DBZ-1362 Fix Snapshot/streaming switch race condition 2019-06-28 09:09:18 +02:00
Jiri Pechanec
e0b188f80a DBZ-1362 Drop replication slot after test execution 2019-06-28 05:41:07 +02:00
Jiri Pechanec
046064d653 DBZ-1362 Make toast test compatible with PostgreSQL 11 2019-06-28 05:32:11 +02:00
Jiri Pechanec
412ce20638 DBZ-1362 Switch to Awaitility implementation 2019-06-28 05:19:01 +02:00
Jiri Pechanec
5a5d969e99 DBZ-1362 Wait for real LSN change during test 2019-06-28 05:19:01 +02:00
Jenkins user
e95d656bca [maven-release-plugin] prepare for next development iteration 2019-06-27 07:36:13 +00:00
Jenkins user
d95fd87311 [maven-release-plugin] prepare release v0.10.0.Beta2 2019-06-27 07:36:12 +00:00
Guillaume Rosauro
dc81d3c8ac DBZ-1352 Removing the unused topic selection strategy configuration from Postgres connector 2019-06-26 16:24:38 +02:00
Jiri Pechanec
355c8a94ac DBZ-1347 JavaDoc update 2019-06-25 14:52:58 +02:00
Gunnar Morling
62fc03a44d DBZ-1347 Commenting 2019-06-25 14:52:58 +02:00
Jiri Pechanec
e72d7edd2f DBZ-1347 Switch to non-blocking stream read 2019-06-25 14:52:58 +02:00
Gunnar Morling
e05ef934a0 DBZ-1345 Adding @FixFor 2019-06-25 12:44:00 +02:00
Chris Cranford
d518fa59fe DBZ-1345 Should not snapshot materialized views 2019-06-25 12:44:00 +02:00
Jiri Pechanec
2dfdf5cb12 DBZ-1342 Partitioned table is included in snapshot 2019-06-21 12:26:40 +02:00
Gunnar Morling
21176ff70f Adding link to replication API 2019-06-21 11:14:35 +02:00
Gunnar Morling
deaa5b7f4d DBZ-1337 Adding support for HSTORE arrays 2019-06-19 17:01:13 +02:00
Jiri Pechanec
badae96e1d DBZ-1335 Process unknown datatypes for snapshotted arrays 2019-06-18 16:17:49 +02:00
Jenkins user
5fdb0ca81b [maven-release-plugin] prepare for next development iteration 2019-06-11 11:20:46 +00:00
Jenkins user
61729b9e00 [maven-release-plugin] prepare release v0.10.0.Beta1 2019-06-11 11:20:46 +00:00
Jiri Pechanec
6a3c87e2f1 DBZ-1323 Initialize commitTime for out-of-order chunk 2019-06-11 10:27:56 +02:00
Gunnar Morling
5d01f409f0 DBZ-1317 Separating update and query methods for MongoDB's SourceInfo; 
* Avoids access to collection id after events which don't have one
* Pulling up struct() and schema() methods to common base class
 2019-06-11 10:07:36 +02:00
Chris Cranford
0aadaa10dc DBZ-1242 Fixed typo and added negative test checks 2019-06-05 13:31:25 -04:00
Chris Cranford
b3ca33287a DBZ-1242 Issue warning when filters produce an empty schema 2019-06-05 13:31:25 -04:00
Jenkins user
056730d06a [maven-release-plugin] prepare for next development iteration 2019-06-03 10:24:08 +00:00
Jenkins user
7aeaec236e [maven-release-plugin] prepare release v0.10.0.Alpha2 2019-06-03 10:24:08 +00:00
Gunnar Morling
246c1be305 DBZ-1283 Enforcing presence of database.server.name globally 2019-06-03 10:58:14 +02:00
Chris Cranford
5f0b44b4ec DBZ-1283 Only validate history topic name for affected connectors 2019-06-03 10:58:14 +02:00
Chris Cranford
b4736a7e03 DBZ-1283 Only validate history topic name for affected connectors 2019-06-03 10:58:14 +02:00
Gunnar Morling
73590173b5 DBZ-1310 Removing methods only needed for testing 2019-05-31 18:09:41 -04:00
Jiri Pechanec
4b64c8e447 DBZ-1295 3state snapshot for MySQL 2019-05-31 10:56:05 +02:00
Jiri Pechanec
348b2415d5 DBZ-1295 3state snapshot for PostgreSQL 2019-05-31 10:56:05 +02:00
Jiri Pechanec
946019982e DBZ-1303 Assembly without database drivers 2019-05-30 14:32:47 +02:00
Gunnar Morling
7759333fe3 DBZ-1092 Using ColumnNameFilter instead of Predicate<Column> 2019-05-29 15:49:49 +02:00
Jenkins user
ac5c76e431 [maven-release-plugin] prepare for next development iteration 2019-05-28 14:38:32 +00:00
Jenkins user
35f59ba460 [maven-release-plugin] prepare release v0.10.0.Alpha1 2019-05-28 14:38:32 +00:00
Gunnar Morling
8ecfbee77d DBZ-1235 Adding TODO marker 2019-05-28 09:35:15 +02:00
Gunnar Morling
b8eede2f41 DBZ-1235 Misc. clean-up; 
* Putting collection name after replica set name for MongoDB
* Removing redundant method for converting schema to string
* Removing redundant keys from PG SourceInfo
 2019-05-28 09:35:15 +02:00
Jiri Pechanec
526e780513 DBZ-1235 Misc fixes 2019-05-28 09:35:15 +02:00
Gunnar Morling
eef37cb567 DBZ-1235 Misc adjustments; 
* Removing superfluous parameter from getSourceInfoStructMaker()
* JavaDoc
* Renaming field to "source.struct.version"
 2019-05-28 09:35:15 +02:00
Gunnar Morling
8f28f323e7 DBZ-1235 Using Instant to represent timestamps internally; 
Only "rendered" as epoch milli-seconds at the system boundaries.
 2019-05-28 09:35:15 +02:00
Jiri Pechanec
6f6c8ddf06 DBZ-1235 Refactored to unify the code 2019-05-28 09:35:15 +02:00
Jiri Pechanec
c1bf1752fe DBZ-1235 Pluggable source maker for MongoDB 2019-05-28 09:35:15 +02:00
Jiri Pechanec
df16ed84cd DBZ-1235 Pluggable source maker for PostgreSQL 2019-05-28 09:35:15 +02:00
Jiri Pechanec
f5757dbd9f DBZ-1235 Extract serverName to common source 2019-05-28 09:35:15 +02:00
Gunnar Morling
4ed8b4636c DBZ-1224 Completing implementation; 
* Further unification of override handling;
* Undoing change related to SnapshotStatementFactory as it seems not needed
* Adding test
 2019-05-27 08:44:42 +02:00
mokkara
b3de0250a9 DBZ-1224 Snapshot SELECT support for SQL Server 2019-05-27 08:44:42 +02:00
Gunnar Morling
a0e30b7c90 DBZ-1289 Making outbox router options more consistent 2019-05-24 13:18:55 +02:00
Chris Cranford
e2c74111ad DBZ-1234 Removed deprecated configuration options 2019-05-22 10:19:05 +02:00
Gunnar Morling
4751b266f3 DBZ-1164 Misc. improvements; 
* Moving stringToDuration() for the sake of re-use and testability
* Making pattern and durations constants
 2019-05-21 09:38:23 +02:00
GraySmog
9b52ac86e8 DBZ-1164 Supporting TIME columns with value of "24:00:00" 2019-05-21 09:38:09 +02:00
Jenkins user
7748be6056 [maven-release-plugin] prepare for next development iteration 2019-05-02 16:04:56 +00:00
Jenkins user
5af083b86e [maven-release-plugin] prepare release v0.9.5.Final 2019-05-02 16:04:55 +00:00
Gunnar Morling
d12926c404 DBZ-1214 Applying default status update interval 2019-05-02 11:30:16 +02:00
Gunnar Morling
6960c880ca DBZ-1214 Misc. clean-up: 
* Using parker instead of Thread#sleep()
* Using flag for executor cancelation
* Using unique context name in thread name
 2019-05-02 11:30:16 +02:00
Addison Higham
e1570dd330 [DBZ-1214] Ensure PG connection keep alive 
This adds a small change to the postgres connector which
works around a problem when large databases take more
than a minute to refresh the schema

This works by creating a simple background thread that ensures we send
statusUpdate messages periodically

I considered if we could refresh the schema before we create the
replication connection but I believe we choose our offset in the log
and then get the schema in order to ensure we don't get out of sync with
any potential schema changes

This also makes some minor refactors to use a Duration instead of an
integer for the statusUpdateInterval
 2019-05-02 11:30:16 +02:00
Jiri Pechanec
3b78e0bebb DBZ-1245 Fix empty heartbeat message test 2019-05-02 09:16:21 +02:00
Jiri Pechanec
b53f456912 DBZ-1245 Heartbeat only when offsets ready 2019-05-02 08:22:28 +02:00
Gunnar Morling
2f7cab6eb3 DBZ-1260 Removing some dead code 2019-05-02 07:28:23 +02:00
Chris Cranford
dc61e52bc5 DBZ-1260 Fix timetz microsecond precision support 2019-05-02 07:28:23 +02:00
Chris Cranford
7602136cf6 DBZ-1236 Removed millisecond truncation. 2019-05-01 14:16:17 +02:00
Chris Cranford
16840d7d59 DBZ-1236 Handle timez precision consistently. 2019-05-01 14:16:17 +02:00
Jiri Pechanec
3ff81348b8 DBZ-1236 Support full timestamptz precision 2019-05-01 14:16:17 +02:00
Gunnar Morling
a84230447e DBZ-1247 Making defaultSnapshotFetchSize() abstract 2019-04-30 09:57:11 +02:00
jchipmunk
72b06366fe Removing extra initialisation of "config" variable in PG config 2019-04-30 09:57:11 +02:00
jchipmunk
9fe033ce24 DBZ-1247 Specifying default snapshot fetch size for each connector config 2019-04-30 09:57:11 +02:00
jchipmunk
a5cc83c0d1 DBZ-1247 Adding property to specify snapshot fetch size 2019-04-30 09:57:11 +02:00
jorkzijlstra
d0e2a72cd1 DBZ-1241 Allowing null values again for commitTime 2019-04-30 09:45:10 +02:00
shubham
f55bec86d3 Added support for all range types 2019-04-24 07:40:03 +02:00
krizhan
70c564b4c2 DBZ-212 Moved SERVER_NAME to RelationalDatabaseConnectorConfig 2019-04-23 12:29:44 +02:00
krizhan
38f3856c31 DBZ-212 Remove PostgreSQL getLogicalName function 2019-04-23 12:29:44 +02:00
Dennis Campagna
173e8f5854 DBZ-1034 Support range column types on Postgres 
Signed-off-by: sciamp <denniscampagna@gmail.com>

DBZ-1034 Support range column types on Postgres

Signed-off-by: sciamp <denniscampagna@gmail.com>
 2019-04-12 09:44:17 +02:00
GraySmog
726a2afcd6 Add comments stating the reason for skipping tests 2019-04-12 09:27:45 +02:00
GraySmog
08ca02f8f4 DBZ-1193 Add support for network type MACADDR and MACADDR8 in PostgreSQL connnector 2019-04-12 09:27:45 +02:00
Jenkins user
cefc5ccfb9 [maven-release-plugin] prepare for next development iteration 2019-04-11 12:07:16 +00:00
Jenkins user
fc3026adc3 [maven-release-plugin] prepare release v0.9.4.Final 2019-04-11 12:07:15 +00:00
Gunnar Morling
2250c47b7f DBZ-1163 Minor improvements 2019-04-10 10:53:22 +02:00
preethi
02873e8169 DBZ-1163 Enable postgres snapshot for tables without primary key 2019-04-10 10:48:11 +02:00
ShubhamRwt
762d908eef DBZ-1193 Added support for cidr address type 2019-04-08 15:00:03 +02:00
jchipmunk
7c0ae3ee20 DBZ-1212 SLF4J usage issues 2019-04-04 21:32:12 +02:00
Jiri Pechanec
ff52f3713e DBZ-1144 Test for geometry in public schema 2019-04-02 15:57:57 +02:00
Alexander Kovryga
4616df189e DBZ-1189 Add support for inet data type. 2019-03-28 09:57:32 +01:00
ShubhamRwt
540a951211 DBZ- 362 Adding WhitespaceAfter check to Checkstyle 2019-03-28 09:24:11 +01:00
Jenkins user
8b0edd1871 [maven-release-plugin] prepare for next development iteration 2019-03-25 12:16:31 +00:00
Jenkins user
56f98a48b2 [maven-release-plugin] prepare release v0.9.3.Final 2019-03-25 12:16:30 +00:00
Renato Mefi
6300f2972f DBZ-1169 Integration tests for Outbox EventRouter 
Set the Snapshot Mode to NEVER provides more consistent tests
 2019-03-25 11:27:44 +01:00
Renato Mefi
8b689d7f03 DBZ-1169 Implement RegexRouter based on config 2019-03-25 11:27:44 +01:00
Renato Mefi
f0f13ed4a4 DBZ-1169 Implement basic behavior 2019-03-25 11:27:44 +01:00
Renato Mefi
2ee1c98298 DBZ-1169 Setup EventRouter on Postgres Test IT 2019-03-25 11:27:44 +01:00
shubham
8cca21e969 DBZ-1039 Make all ifs with braces 2019-03-25 10:23:14 +01:00
Addison Higham
8e21139ca3 DBZ-1082 Add new custom recovery mode, more metadata 
This commit does a few things:

- Refactors snapshot modes to be encapsulated by an interface and
to only use that interface in determining when to snapshot and in
determing the type of the `RecordProducer` interface to instantiate
- Refactors the configuration of existing snapshot modes to tie the
existing snapshot modes to their aligned implementation
- Adds a new snapshot.mode, custom, and a new configuration option to
specify a custom implementation that will be loaded by the class loader
- Changes the visibility of some classes to allow for custom snapshot
modes to get enough context to make an informed choice
- Adds some metadata about slots (the catalog_xmin) to give a full idea
of the state of slots which can be useful in implementing snapshot
modes (which is also configurable, as it can add some overhead)

Together, these changes allow for a much broader flexibility got end
users to implement a snapshot mode that can do more advanced snapshots,
such as partial recovery or for partial snapshots for tables where not
all records are needed.

This could also be seen as superseeding the
`snapshot.select.statement.overrides` to allow for users to dynamically
build queries based on the state of the slot and the offsets consumed.
 2019-03-24 11:56:40 +01:00
Jiri Pechanec
b11f1b8d05 DBZ-1161 Start snapshot in async thread 2019-03-21 14:32:47 +01:00
trolison
2001b178d7 - Fixed bad formatting 
- Added trace for when empty messages are processed
DBZ-1181
 2019-03-19 10:46:36 -07:00
trolison
df0e789601 - Add support for processing WAL2JSON events that have no changes 
DBZ-1181
 2019-03-18 20:32:03 -07:00
Gunnar Morling
2ea87727cf DBZ-1174 Using Instant instead of long for representing timestamps 2019-03-12 17:19:13 +01:00
Ashhar Hasan
0b07afd23e DBZ-1174 Convert commitTime from nanos to micros for wal2json 
- wal2json sends the txn commitTime using a function from PostgreSQL's C
  library. The value that Debezium recevies is in nanoseconds.
- decoderbufs sends the txn commitTime in microseconds.
- RecordsSnapshotProducer updates SourceInfo.ts_usec by converting
  System.currentTimeMillis() to microseconds.
- RecordsSnapshotProducer updates the SourceInfo's ts_usec field using
  message.getCommitTime().

This means that when using wal2json, the value of SourceInfo.ts_usec
is in microseconds since epoch during snapshot but is in nanoseconds
during streaming. To fix this, we changed
Wal2JsonReplicationMessage.getCommitTime() to return in microseconds.
 2019-03-12 17:19:13 +01:00
Josh Stanfield
e4b7b90637 DBZ-1162 fix to ensure hstore snapshots behave the same as streams 2019-03-04 13:39:50 +01:00
Jenkins user
50a7c568fa [maven-release-plugin] prepare for next development iteration 2019-02-23 09:53:58 +00:00
Jenkins user
28f3839804 [maven-release-plugin] prepare release v0.9.2.Final 2019-02-23 09:53:58 +00:00
Jiri Pechanec
78bfda09a6 DBZ-1146 Test full column diff 2019-02-22 14:16:03 +01:00
Gunnar Morling
5043d41b7b DBZ-1146 Expanding test 2019-02-22 14:16:03 +01:00
Jiri Pechanec
4dec844fa2 DBZ-1146 Fix table created in streaming test 2019-02-22 14:16:03 +01:00
Jiri Pechanec
fa1e632a90 DBZ-1146 Handle toastable columns for snapshotted tables 2019-02-22 14:16:03 +01:00
Jiri Pechanec
89feae32e7 DBZ-1141 Test for the default values 2019-02-21 15:02:51 +01:00
Gunnar Morling
6787d17d54 DBZ-1141 Misc. clean-up; 
* Removing redundant check for date mapping type
* Always using String as fallback value for temporal values where needed
* Pulling fallback temporal values up to JdbcValueConverters
 2019-02-21 15:02:51 +01:00
Jiri Pechanec
b9d326711d DBZ-1141 Incorrect money scale 2019-02-21 15:02:51 +01:00
Jiri Pechanec
992c31fa1f DBZ-1141 Add non-date types 2019-02-21 15:02:51 +01:00
Jiri Pechanec
41ba7128e3 DBZ-1141 Correct default falback values for connect date types 2019-02-21 15:02:51 +01:00
Keith Barber
95b2290df2 DBZ-1149 Ensure consistent Java types for fallback values when converting 2019-02-21 15:02:51 +01:00
taylor-rolison
17c6a7a82d Config option to pass optional logical decoder stream params (#783) 
DBZ-1130 Added config param slot.stream.params to the Postgres connector to allow passing optional params to logical decoder on stream start
 2019-02-21 14:06:51 +01:00
Jenkins user
b96e5fee0d [maven-release-plugin] prepare for next development iteration 2019-02-13 11:13:44 +00:00
Jenkins user
3de2d04bcd [maven-release-plugin] prepare release v0.9.1.Final 2019-02-13 11:13:44 +00:00
Jiri Pechanec
9049078607 DBZ-997 Handle de-sync of message and JDBC PK info 2019-02-13 10:13:12 +01:00
Gunnar Morling
2a486ecfff DBZ-1019 Removing legacy methods from Metronome 2019-02-11 18:11:34 +01:00
Ivan Lorenz
721e8112cd DBZ-1126 Localize era display value to default locale 2019-02-06 15:34:30 +01:00
Gunnar Morling
7076deba5a DBZ-1118 Adding JUnit rule to skip tests depending on Postgres version 2019-02-04 14:43:19 +01:00
Kevin Pullin
c414a8b942 Add test for DBZ-1118 2019-02-04 14:43:19 +01:00
Kevin Pullin
1330df8908 Postgres snapshot - On null records, clear prior value from currentRecord 2019-02-04 14:43:19 +01:00
Gunnar Morling
41b7fbe78e Adding note on testing Postgres connector with external DB 2019-01-30 09:25:12 +01:00
Gunnar Morling
cc0df61665 DBZ-1111 Allowing to use drop-slot-on-close with PG 10 or newer 2019-01-29 14:44:10 +01:00
Gunnar Morling
e84c6e2238 DBZ-962 Avoiding dependency to JAXB 2019-01-29 10:28:48 +01:00
Jenkins user
c5b7d21d3e [maven-release-plugin] prepare for next development iteration 2019-01-28 11:07:04 +00:00
Jenkins user
f1ae79ff73 [maven-release-plugin] prepare release v0.9.0.CR1 2019-01-28 11:07:04 +00:00
Gunnar Morling
ccff79e07d DBZ-1105 Removing TableImpl#columnNames field 2019-01-28 10:23:56 +01:00
Gunnar Morling
9dc2c525c9 DBZ-1083 Passing "include-unchanged-toast" option only for wal2json 2019-01-25 11:04:43 +01:00
Addison Higham
26f836e0f6 DBZ-1080 new API for EmbeddedEngine for batch/async 
This introduces a new API to the EmbeddedEngine, the ChangeConsumer,
which gives the user a more flexible option for consuming changes by
exposing groups of records as well as the ability to control the
comitting of those records.

This remainds completely backwards compatible with the old API as the
ChangeConsumer wraps the existing Consumer interface with a default
implementation.
 2019-01-18 14:06:50 +01:00
Jiri Pechanec
1cf257478b DBZ-781 Enable unstable tests 2019-01-17 08:59:42 +01:00
Tony Rizko
a921613877 Fix typo in PostgresConnection error message 2019-01-17 08:58:12 +01:00
Jiri Pechanec
ceb6c62721 DBZ-1064 Remove lazy fallback values 2019-01-16 13:01:36 +01:00
Jiri Pechanec
536e517c08 DBZ-1064 Geometry default value changes; fixes for default suppliers 2019-01-16 13:01:36 +01:00
Gunnar Morling
532c17f8aa DBZ-1064 Using dedicated functional interface instead of Consumer; 
* Commenting
* Moving type from "util" to "jdbc"
 2019-01-16 13:01:36 +01:00
Jiri Pechanec
bff7eea734 DBZ-1064 Use convert value template 2019-01-16 13:01:36 +01:00
Jiri Pechanec
920e0cbd3f DBZ-1064 Set default value only for non-nullable fields 2019-01-16 13:01:36 +01:00
Jiri Pechanec
484a8977bc DBZ-1083 Passing "include-unchanged-toast" option only if supported; 
The "include-unchanged-toast" option was removed in recent wal2json versions, without a transition phase. So the connector tries not to connect first giving the option, and if that fails, without it.
 2019-01-16 09:37:55 +01:00
Gunnar Morling
af01d10886 DBZ-1081 JavaDoc; adding Addison Higham to COPYRIGHT.txt 2019-01-15 18:09:08 +01:00
Addison Higham
889b307b9d DBZ-1081 Fallback to restart_lsn in pg 9.5 
In pg 9.5, confirmed_flush_lsn is not availiable. However, there is
restart_lsn, which should be safe to use with a downside of producing
some extra duplicate records.

This page: https://paquier.xyz/postgresql-2/postgres-9-6-feature-highlight-replication-slot-improvements/
indicates as such and this allows for DBZ to support PG 9.5
 2019-01-15 18:05:55 +01:00
Gunnar Morling
de356896c0 DBZ-1029 Adding one more test 2019-01-15 17:41:42 +01:00
Ilia Bogdanov
88521371cc DBZ-1029 insertion test update, code cleanup after review 2019-01-14 12:03:19 +03:00
Ilia Bogdanov
cee9bd9a5d DBZ-1029 handling messages from tables with replica identity full in postgres connector. 2019-01-14 12:03:19 +03:00
Amit Sela
b0b7d942a3 DBZ-1073 Adding scale as schema parameter, if present 2019-01-11 15:40:30 +01:00
Grzegorz Kołakowski
91c72738c0 DBZ-1067 Add column blacklist field to RelationalDatabaseConnectorConfig 2019-01-09 18:13:26 +01:00
Jenkins user
5275f73424 [maven-release-plugin] prepare for next development iteration 2018-12-19 13:06:12 +00:00
Jenkins user
b6569c18ae [maven-release-plugin] prepare release v0.9.0.Beta2 2018-12-19 13:06:12 +00:00
Gunnar Morling
92ae01ccdf DBZ-996 Adding column name to log message 2018-12-18 17:53:42 +01:00
Jiri Pechanec
01eee09f90 DBZ-996 Tolerate race condition in PK updates 2018-12-18 17:52:28 +01:00
Jiri Pechanec
edec1c3090 DBZ-978 Added last event and captured tables metrics 2018-12-13 20:52:11 +01:00
Gunnar Morling
5542c0de29 DBZ-966 Making sure Postgres connector can be stopped while awaiting initial snapshot delay; 
* Also skipping snapshotting when requested to stop during initial delay
* Showing remaining delay time in logs
* Adding missing config
 2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
e2942ab72a DBZ-966 Replace sleep with polling loop 2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
35a5acfd8b DBZ-966 Enable to delay initial snapshot in PostgreSQL connector 2018-12-13 14:29:54 +01:00
Gunnar Morling
0833b23feb DBZ-842 Expanding test to cover TOASTed NOT NULL column, too 2018-12-11 18:42:51 +01:00
Jiri Pechanec
94c0a42286 DBZ-842 Do not send TOASTed values from wal2json 2018-12-11 17:10:58 +01:00
Jiri Pechanec
2b2ff0754d DBZ-960 Run SnapshotReader in a single transaction 2018-12-11 14:15:53 +01:00
Gunnar Morling
470cd1eaba DBZ-1024 Doing some clean-up 2018-12-07 15:39:59 +01:00
Jiri Pechanec
ea5c8512b7 DBZ-1024 Return test assert 2018-12-07 15:39:59 +01:00
Jiri Pechanec
0b78f0c31a DBZ-1024 Emit correct last_snapshot_record in source 2018-12-07 15:39:59 +01:00
Jiri Pechanec
9c225c63eb DBZ-1021 Ignore events for dropped table 2018-12-06 18:54:50 +01:00
Gunnar Morling
101bd8ef6d DBZ-1000 Making Checkstyle happy 2018-12-03 15:00:43 +01:00
Jiri Pechanec
7a3cbd6cb6 DBZ-1000 Detect nullability change in schema 2018-12-03 14:54:36 +01:00
Jiri Pechanec
68766e9355 DBZ-1009 Support for unknown types during snapshot 2018-11-30 11:16:36 +01:00
Jenkins user
db1d3a7fb8 [maven-release-plugin] prepare for next development iteration 2018-11-20 16:15:14 +00:00
Jenkins user
f83db82cea [maven-release-plugin] prepare release v0.9.0.Beta1 2018-11-20 16:15:14 +00:00
Jiri Pechanec
0611d4975f DBZ-965 Reduce the test time 2018-11-13 09:16:30 +01:00
Jiri Pechanec
a4abd2a336 DBZ-965 Test of flushed LSN 2018-11-13 09:15:09 +01:00
Gunnar Morling
5a9ca6eb31 DBZ-965 Some clean-up 2018-11-13 09:15:09 +01:00
Jiri Pechanec
675f5b6479 DBZ-965 Renames for better clarification 2018-11-13 09:15:09 +01:00
Gunnar Morling
b455b7ccaf DBZ-970 Making LogicalDecoder#getPostgresPluginName() abstract 2018-11-13 09:15:09 +01:00
Jiri Pechanec
390965f4f9 DBZ-965 Track latest LSN per each change 2018-11-13 09:15:09 +01:00
Jiri Pechanec
b1707c395d DBZ-970 Correct plging name for wal2json_streaming decoder 2018-11-13 09:15:09 +01:00
Jiri Pechanec
83137d0a41 DBZ-965 Force heartbeat after end of each TX 2018-11-13 09:15:09 +01:00
Gunnar Morling
efbd46b7f6 DBZ-953 Having PG connector use decimal mode option from RelationalDatabaseConnectorConfig 2018-11-08 09:23:28 +01:00
Jiri Pechanec
3a8293a4ba DBZ-937 Disable auto-handling of unknown types in test 2018-10-11 09:55:07 +02:00
Olavi Mustanoja
0c986b4a78 DBZ-937 add wal2json replication message handling for CITEXT 2018-10-09 08:46:55 +02:00
Jenkins user
d952f5dfb0 [maven-release-plugin] prepare for next development iteration 2018-10-04 11:59:14 +00:00
Jenkins user
ff9b70278b [maven-release-plugin] prepare release v0.9.0.Alpha2 2018-10-04 11:59:13 +00:00
Gunnar Morling
1f25536e2b DBZ-934 Using API for creating replication slot if possible 2018-10-04 10:12:48 +02:00
PSanetra
dd3859d8bc DBZ-934 Use temporary replication slots if slot.drop_on_stop is enabled and Postgres >= 10 is used 2018-10-04 10:12:48 +02:00
jchipmunk
3ec0b3be81 DBZ-918 Moving connector identifier from SourceInfo to Module class 2018-10-04 09:00:13 +02:00
jchipmunk
05ba99bb0a DBZ-918 Adding Debezium connector field to source info 
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
 2018-10-04 09:00:13 +02:00
Gunnar Morling
984081dd1a DBZ-926 Adding flushed LSN to log message 2018-10-03 09:47:38 +02:00
Gunnar Morling
08ec4f48d0 DBZ-911 Style adjustments; wording fix 2018-10-01 12:49:56 +02:00
Ian Axelrod
427249887b DBZ-911 Simplifying schemaChanged() shortcircuit logic 
Break up initial conditional and comment missing column portion to
clarify the role of skipRefreshSchemaOnMissingToastableData.
 2018-10-01 12:49:48 +02:00
Ian Axelrod
9f083d0f1b DBZ-911 Simplifying getToastableColumnsForTableId() impl. 
No need to wrap ArrayList instance in unmodifiableList. Can just use
Collections.emptyList.
 2018-10-01 12:49:36 +02:00
Ian Axelrod
ad433c5747 DBZ-911 Don't refresh schema on absent TOAST cols 
Improves PostgreSQL RecordsStreamProducer performance for processing up
dates to a table with TOASTable columns, where those updates do not
affect said columns. Prior to this fix, these updates would trigger a
refresh of the in-memory table schema. In the worst case, this means a
query for every update. This puts significant load on the database
server and adds tens of milliseconds to the processing of each update
record.

The fix requires a new configuration option, called
'schema.refresh.mode'. This option has values 'columns_diff' (the
default) and 'columns_diff_exclude_unchanged_toast'. 'columns_diff'
maintains the pre-fix behavior. 'columns_diff_exclude_unchanged_toast'
activates the fix.

The fix must be toggleable because it decreases the consistency guarantees
Debezium provides, since in-memory table schemas may not stay
synchronized with their remote counterparts. With type metadata included
in the replication message, inconsistencies are limited to the unchanged
toast columns.
 2018-10-01 11:39:32 +02:00
Gunnar Morling
6f7bdbe5e1 DBZ-898 Using 'Json' semantic type when encoding HSTORE column values as string 2018-09-25 12:22:20 +02:00
Gunnar Morling
7150e387c7 DBZ-898 Misc. clean-up: 
* Using database encoding for string conversion
* Not making hstore schemas optional by default
* Using Jackson instead of GSon for JSON serialization
* Removing superfluous method and log messages
* Adjusting to naming and style conventions
 2018-09-24 13:36:28 +02:00
SyedMuhammadSufyian
623cc98cb0 Change log levels 2018-09-22 20:15:07 +02:00
SyedMuhammadSufyian
346f2b3761 Handling HStore type , hstore.handling.mode:map|json , default is map.Added support for protobuff 2018-09-22 20:07:13 +02:00
SyedMuhammadSufyian
38f4a0e65b HStore Data type support , configuration options are map|json , default is map 2018-09-22 05:54:45 +02:00
Gunnar Morling
f9ede861ba DBZ-912 Upgrading to Postgres driver 42.2.5 2018-09-19 15:26:44 +02:00
Gunnar Morling
39a47fabfa DBZ-899 Using custom statement to obtain SQL type info in order to avoid N+1 issues with PgDatabaseMetaData#getTypeInfo() 2018-09-19 09:51:38 +02:00
Gunnar Morling
3331d4c1a9 DBZ-899 Removing unused field; 
Also inlining method only used in one place.
 2018-09-19 09:51:38 +02:00
Jiri Pechanec
59fbcca725 DBZ-907 Log refreshing schema due to different column count 2018-09-12 19:07:01 +02:00
Jiri Pechanec
e20c183d5d DBZ-895 Make tcpKeepalive enabled by default 2018-09-11 09:34:21 +02:00
Gunnar Morling
e1bbbd7992 DBZ-865 DBZ-878 Replacing invalid topic name characters with "_"; 2018-09-05 12:56:53 +02:00
Gunnar Morling
bd692b8b84 DBZ-862 Extracting values for controlling retry attempts into constants 2018-09-03 14:50:54 +02:00
Jiri Pechanec
c6e20d4967 DBZ-862 Fix test interdependency problem 2018-09-03 14:40:57 +02:00
Jiri Pechanec
1ff53e35ec DBZ-862 Fix Travis timing issue 2018-09-03 14:40:57 +02:00
Jiri Pechanec
4b9141e1fd DBZ-862 Handle concurrent transaction blocking taking snapshot 2018-09-03 14:40:57 +02:00
Jiri Pechanec
e65fb26ae9 DBZ-887 Close connection even for non-started producer 2018-08-31 13:02:23 +02:00
Gunnar Morling
ae879cf475 DBZ-859 Adjustments after rebase; typo fix 2018-08-23 12:00:44 +02:00
Jiri Pechanec
cd6dc09e72 DBZ-859 Heartbeat is sent after PostgreSQL snapshot 2018-08-23 11:44:53 +02:00
oripwk
7d958a1928 DBZ-866 PR fixes 2018-08-22 17:09:26 +02:00
oripwk
d82372a6b2 DBZ-866 Add schema, database and table to Postgres connector 2018-08-22 17:09:26 +02:00
Jiri Pechanec
d4e64a61cb DBZ-705 Allow to configure postgres config file location 2018-08-13 20:21:41 +02:00
Jenkins user
e00dad127f [maven-release-plugin] prepare for next development iteration 2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700 [maven-release-plugin] prepare release v0.9.0.Alpha1 2018-07-26 08:00:12 +00:00
Gunnar Morling
3061c1dc0b DBZ-815 Passing offset to Heartbeat#heartbeat(); 
That's simpler to grasp than the approach of passing a supplier lambda to the constructor.
Also it allows to pass on the offset via local variables instead of instance fields in some cases.
 2018-07-25 14:49:52 +02:00
Gunnar Morling
8e1450bc71 DBZ-800 Adding test for receiving heartbeat events with Postgres 2018-07-20 13:37:54 +02:00
Gunnar Morling
4abc471bcb DBZ-800 Emitting heartbeat events in Postgres streaming loop 2018-07-20 13:37:54 +02:00
Gunnar Morling
20db9299c5 DBZ-627 Unifying TopicSelector implementations 2018-07-18 12:36:07 +02:00
Gunnar Morling
1523f230ca DBZ-793 Using TableFilter consistently; 
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
 2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8 [maven-release-plugin] prepare for next development iteration 2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f [maven-release-plugin] prepare release v0.8.0.Final 2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d [maven-release-plugin] prepare for next development iteration 2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0 [maven-release-plugin] prepare release v0.8.0.CR1 2018-07-04 07:07:44 +00:00
Jiri Pechanec
c2f2f686f0 DBZ-644 Disable test making problems in Travis CI 2018-07-03 07:03:59 +02:00
Jiri Pechanec
646a66672c DBZ-644 Close unused replication connection 2018-07-03 07:03:59 +02:00
Gunnar Morling
dcd6b01134 DBZ-644 Simplifying RelationalDatabaseSchema constructor; 
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
 2018-07-03 07:03:59 +02:00
Gunnar Morling
b0980b994c DBZ-644 Adding test for Postgres 2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema 2018-07-03 07:03:59 +02:00
Jiri Pechanec
0182eecd06 DBZ-778 Make the tests resilient against race condition 2018-07-02 11:52:23 +02:00
Gunnar Morling
a7be3847e3 DBZ-773 Removing some dead code and unused methods 2018-06-29 09:27:57 +02:00
Gunnar Morling
c4c8cbc3ab DBZ-773 Moving management of Tables to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab DBZ-773 Moving management of table schemas to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
0600ccf213 DBZ-751 Incorporating review remarks; 
* Expanding test
* Extracting constant for parameter key in tests
 2018-06-27 14:48:59 +02:00
Gunnar Morling
a292b05a96 DBZ-751 Propagating DECIMAL column precision to Avro schemas 2018-06-27 14:48:59 +02:00
Jiri Pechanec
ca6b1c9168 DBZ-743 Empty password for PostgreSQL and MySQL 2018-06-27 11:18:19 +02:00
Gunnar Morling
f9c1cfc134 DBZ-762 Adding support for CITEXT array columns 2018-06-26 11:40:19 +02:00
Gunnar Morling
daeeaa74cd DBZ-762 Formatting 2018-06-26 11:40:19 +02:00
Olavi Mustanoja
c401a98166 DBZ-762 Added support for CITEXT column type 2018-06-26 11:40:19 +02:00
Gunnar Morling
bf7a5018ca Setting POM version back to 0.8.0-SNAPSHOT 2018-06-22 12:21:03 +02:00
Jenkins user
db42e4657a [maven-release-plugin] prepare for next development iteration 2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99 [maven-release-plugin] prepare release v0.8.0.Beta1 2018-06-21 14:07:45 +00:00
Gunnar Morling
987e94cfbb DBZ-578 Removing unnecessary "server.zone.offset" option; 
This reverts parts of the previous work done for DBZ-578.
TIMESTAMP columns (logical date + time without time zone)
are always interpreted using UTC again.
 2018-06-20 15:58:19 +02:00
Gunnar Morling
b83f1493a2 DBZ-20 Avoiding Optional as method parameter 2018-06-20 13:05:37 +02:00
Gunnar Morling
dce1aa29f1 DBZ-20 Oracle precision supports negative numbers 2018-06-20 13:05:37 +02:00
Jiri Pechanec
89694ffbbf DBZ-20 Support for year to month interval 2018-06-20 13:05:37 +02:00
Gunnar Morling
d7e196a18e DBZ-20 Initial import of Oracle connector based on XStream 2018-06-20 13:05:37 +02:00
Gunnar Morling
9eb4b90ec9 DBZ-191 Adding record validations 2018-06-15 20:23:17 +02:00
Jiri Pechanec
1e678d9959 DBZ-679 Make streaming decoding optional 2018-06-15 08:35:50 +02:00
Jiri Pechanec
7ee1f98f57 DBZ-679 Support decoding of out-of-order fragments 2018-06-15 08:35:50 +02:00
Jiri Pechanec
eadf65cd27 DBZ-578 Build failing on Jenkins CI 2018-06-13 14:43:29 +02:00
Gunnar Morling
a9d3d0c95d DBZ-711 Using correct SSL mode names 2018-06-13 10:12:14 +02:00
Jiri Pechanec
29f8891b4f DBZ-578 Process timestamps without timezones as in server timezone 2018-06-11 15:27:08 +02:00
Jiri Pechanec
4111ddbcaf DBZ-709 Add test for timestamp with precision 2018-06-05 15:51:35 +02:00
Jiri Pechanec
176dcbbc0b DBZ-709 PostgreSQL uses scale for time precision not length 2018-06-05 15:51:35 +02:00
Jiri Pechanec
525d073fea DBZ-681 Time with precision for wal2json plug-in 2018-06-04 14:50:16 +02:00
Gunnar Morling
8d7806e71a DBZ-693 Expanding test; updating connector option description 2018-05-28 12:35:47 +02:00
Jiri Pechanec
45bc7b1536 DBZ-693 On connect statements supported for PostgreSQL 2018-05-28 12:35:47 +02:00
Jiri Pechanec
e2603d392e DBZ-699 Handle all exceptions 2018-05-24 13:50:16 +02:00
Jiri Pechanec
e533b75cbf DBZ-699 Close JDBC connection when initialization fails 2018-05-24 09:49:42 +02:00
Jiri Pechanec
fa2691ffd1 DBZ-701 Support for arrays with large OID 2018-05-23 22:13:21 +02:00
Jiri Pechanec
f844054f50 DBZ-696 Support nanoseconds in timestamp data 2018-05-22 11:33:50 +02:00
Jiri Pechanec
09b28c0913 DBZ-694 Support milliseconds for pre-epoch dates 2018-05-22 11:22:06 +02:00
Jiri Pechanec
37cfe622fc DBZ-697 Support for large type OIDs 2018-05-22 10:39:14 +02:00
Jiri Pechanec
2706858cf4 DBZ-676 Support testing against 8bit enc database 2018-04-06 18:26:45 +02:00
Gunnar Morling
9a29063ef4 DBZ-673 Converting tx id to long in one more place 2018-03-27 10:04:43 +02:00
listman
aa4bf5d367 DBZ-673 txId must be long in wal2jon decoder 2018-03-27 00:31:03 -05:00
Gunnar Morling
bdb1069b0d DBZ-638 Logging message on TRACE level instead of DEBUG 2018-03-23 16:38:05 +01:00
Jiri Pechanec
363921a7b1 DBZ-638 Chunk processing operates on byte array 2018-03-23 14:41:59 +01:00
Jiri Pechanec
4051a4a439 DBZ-638 Improved variable naming 2018-03-23 14:41:59 +01:00
Jiri Pechanec
ce5196dc3e DBZ-638 Keep non-streaming decoding as a fallback 2018-03-23 14:41:59 +01:00
Jiri Pechanec
24965dd076 DBZ-638 Process tx as stream not as batch 2018-03-23 14:41:59 +01:00
Jiri Pechanec
0743467102 DBZ-671 Date/time parsing covers more exotic formats 2018-03-23 08:55:24 +01:00
Jenkins user
f4e151b23a [maven-release-plugin] prepare for next development iteration 2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332 [maven-release-plugin] prepare release v0.7.5 2018-03-20 08:14:19 +00:00
Jiri Pechanec
40a527b6ea DBZ-592 Make snapshot/init-sync false instead of null 2018-03-19 13:19:34 +01:00
Jenkins user
daf27207be [maven-release-plugin] prepare for next development iteration 2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928 [maven-release-plugin] prepare release v0.7.4 2018-03-07 08:31:07 +00:00
Gunnar Morling
1d5d6b6f3a DBZ-635 Schema for geo-spatial array columns shouldn't be optional by default 2018-03-07 07:17:43 +01:00
Gunnar Morling
ff524695bc DBZ-635 Schema for NUMERIC columns shouldn't be optional by default 2018-03-07 07:17:43 +01:00
Gunnar Morling
6412e9e06e DBZ-605 Schema for BYTEA columns shouldn't be optional by default 2018-03-07 07:17:43 +01:00
Jiri Pechanec
327085164e DBZ-611 Special values supported for double 2018-03-06 16:55:24 +01:00
Jiri Pechanec
424aefbbb4 DBZ-611 Decimal-> String supported for MySQL 2018-03-06 16:55:24 +01:00
Gunnar Morling
6e29b82358 DBZ-351 Avoiding call to new BigDecimal() for NaN 2018-02-27 16:36:21 +01:00
Gunnar Morling
5786d44200 DBZ-351 Avoiding call to getBigDecimal() for NaN 2018-02-27 16:36:21 +01:00
Gunnar Morling
d040ec4427 DBZ-351, DBZ-606 Misc. clean-up; 
* Renaming DebeziumDecimal to SpecialValueDecimal
* Simplifying logic in PostgresValueConverter
 2018-02-27 16:36:21 +01:00
Jiri Pechanec
2534696b21 DBZ-606 Exception thrown for special values when mode handling is not string 2018-02-27 16:36:21 +01:00
Jiri Pechanec
5827a7ac1b DBZ-611 Decimals can be encoded as strings 2018-02-27 16:36:21 +01:00
Jiri Pechanec
c71707afc9 DBZ-351 wal2json usses unlimited precision too 2018-02-27 16:36:21 +01:00
Jiri Pechanec
0c0ac72568 DBZ-606 Type for handling special BigDecimals 2018-02-27 16:36:21 +01:00
Jiri Pechanec
ceaa7f8efe DBZ-351 Numeric is passed with unlimited precision 2018-02-27 16:36:21 +01:00
Jiri Pechanec
4623c1d8c4 DBZ-606 Numeric NaN represented as null 2018-02-27 16:36:21 +01:00
Gunnar Morling
c900c7fc92 DBZ-623 Formatting and indentation 2018-02-22 21:16:00 +01:00
Jiri Pechanec
01e5f8da8c DBZ-623 Extract parsing code to a separate method 2018-02-22 21:12:05 +01:00
Jiri Pechanec
80c74a9d7c DBZ-623 Fix NPE for invlid restored LSNs 2018-02-22 21:12:05 +01:00
Gunnar Morling
eef1ad7c2a DBZ-630 Refactoring around MongoDB task context; 
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
 2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8 DBZ-630 Pulling up getClock() to ConnectorTaskContext 2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6 DBZ-630 Unifying common start-up logic across connectors 2018-02-21 12:04:41 +01:00
Jiri Pechanec
4e74e0a4ac Revert default Postgres decoder plugin for testing 2018-02-20 10:29:06 +01:00
Jiri Pechanec
ad181493da DBZ-626 Make caller method naming consistent 2018-02-20 09:26:47 +01:00
Gunnar Morling
7fe45db831 DBZ-626 Renaming AvroValidator to SchemaNameAdjuster; 
Making apparent that this functionality doesn't solely validate but returns a new name actually
 2018-02-20 09:26:47 +01:00
Gunnar Morling
cddd5fcc8a DBZ-626 Passing schema validator as AvroValidator instance instead of Function 2018-02-20 09:26:47 +01:00
Jiri Pechanec
40a5422998 DBZ-605 Support for BYTEA datatype 2018-02-20 08:55:56 +01:00
Jiri Pechanec
6e00924ee9 DBZ-625 Fix parsing of float without decimals 2018-02-20 08:47:43 +01:00
Gunnar Morling
8369f10d52 DBZ-628 Adding MAX_QUEUE_SIZE, MAX_BATCH_SIZE and POLL_INTERVAL_MS to CommonConnectorConfig; 
Also using these options in PG connector
 2018-02-20 06:34:42 +01:00
Gunnar Morling
ab11f48843 DBZ-620 Creating envelope schemas only once per table type 2018-02-19 12:18:37 +01:00
Jenkins user
6d0cd88e12 [maven-release-plugin] prepare for next development iteration 2018-02-15 04:15:34 +00:00
Jenkins user
7d1e1a989e [maven-release-plugin] prepare release v0.7.3 2018-02-15 04:15:34 +00:00
Gunnar Morling
cc64a7a616 DBZ-618 Removing unused method and field 2018-02-14 16:53:52 +01:00
Gunnar Morling
f54ed51897 DBZ-618 Removing superfluous method parameter 2018-02-14 16:43:52 +01:00
Jiri Pechanec
d0198e3e26 DBZ-618 Incorrect type retrieved by stream producer for time/date columns 2018-02-14 13:06:50 +01:00
Gunnar Morling
0f726b5f4e DBZ-616 Making PostgresType#equals() and hashCode() symmetric; logging improvements 2018-02-14 10:54:39 +01:00
Jiri Pechanec
2a08a718ba DBZ-616 Get default length and scale for types 2018-02-14 10:48:13 +01:00
Gunnar Morling
bda12b59ee DBZ-571 Misc. improvements; 
* More consistent variable names
* Making type registry truly immutable
* Avoiding call to instance method from constructor
 2018-02-13 16:15:10 +01:00
Jiri Pechanec
5a0c7c07b0 DBZ-609 DBZ-571 Refactor Postgres type handling 
TypeRegistry introduced for Postgres connector

JDBC column does not have a special componentType

JDBC column provide a database specific type id

OID is a primary type identifier to be used in Postgres connector code - dropping JDB/OID dichotomy
 2018-02-13 14:54:42 +01:00
Gunnar Morling
2cd1a43db8 DBZ-220 Removing println() call 2018-02-13 12:31:36 +01:00
Jiri Pechanec
682cabb84a DBZ-220 Generate heartbeat events in binlog thread 2018-02-13 12:24:51 +01:00
Gunnar Morling
f3ea0b5683 DBZ-582 Supporting TOMBSTONES_ON_DELETE option with Postgres connector 2018-02-13 10:19:18 +01:00
Gunnar Morling
a77c8d733a DBZ-580, DBZ-586 Making enqueue() and thus snapshots in Postgres interruptable; 
* ChangeEventQueue#enqueue() checks the interrupted state of the calling
thread now, raising an InterruptedException in case the interrupted flag
has been set (because the producer's thread executor has been stopped)
* RecordSnapshotProducer has been adjusted to check for the interrupted
regularly, aborting if it has been set
 2018-02-11 10:05:06 +01:00
Gunnar Morling
45a9847d42 DBZ-580 Initial implementation of dedicated change message queue: 
* To be used across connectors, unifying the enqueuing/polling logic and handling of related config options
* Using it for Postgres connector
 2018-02-11 10:05:06 +01:00
Jiri Pechanec
6150955a47 DBZ-590 Detect non-available offset 2018-02-05 12:20:47 +01:00
Gunnar Morling
712824e43f DBZ-593 Passing version explicitly from SourceInfos to base class instead of relying specific packaging structure 2018-02-02 13:16:43 +01:00
Jiri Pechanec
7ebee94169 DBZ-593 Add Debezium version to source in envelope 2018-02-02 13:16:43 +01:00
Jiri Pechanec
9b592204ac DBZ-587 Centralize and unify thread management 2018-02-01 10:04:20 +01:00
Jiri Pechanec
2c74dd74a3 DBZ-577 Numeric arrays correctly working 2018-01-29 15:14:11 +01:00
Jenkins user
04624341f5 [maven-release-plugin] prepare for next development iteration 2018-01-25 09:39:44 +00:00
Jenkins user
898f6884e1 [maven-release-plugin] prepare release v0.7.2 2018-01-25 09:39:44 +00:00
Robert Coup
e3fbf165ed DBZ-507 Temporarily disabling geometry/geography array support in postgres until array support is improved. 
Test tidyups.
 2018-01-24 16:51:54 +01:00
Robert Coup
f9d90a482f DBZ-507 Expanding support for geometry types; 
* Adding support for PostGIS geometry types
* Adding support for GEOMETRY, POLYGON and more in MySQL
* For all newly supported types, changes are represented using two new schema types Geometry and Geography, containing the WKB (binary geo data) ans srid (coord system identifier)
* The existing Point type also contains the new (optional) srid field
 2018-01-24 16:51:49 +01:00
Jiri Pechanec
5ff2cf4dd8 DBZ-558 Return correct array OID 2018-01-23 19:34:39 +01:00
Jiri Pechanec
fe16050cdb DBZ-521 SourceRecord is now associated with metadata 2018-01-19 21:18:55 +01:00
Gunnar Morling
6f6ce33958 DBZ-521 Misc. improvements and fixes; 
* Actually assing last processed LSN to commit()
* passing LSN to commit() instead of entire SourceRecord
* Avoiding exception in case of wal2json batch with exactly one element
* Using "Lsn" instead of "LSN" in method names
* JavaDoc clarifications
 2018-01-19 21:18:55 +01:00
Jiri Pechanec
0b5348339a DBZ-521 Record LSN for the last message committed to Connect 2018-01-19 21:18:55 +01:00
Jiri Pechanec
c3edf8ee9c DBZ-504 Type changes detected for constraints 2018-01-19 14:45:08 +01:00
Gunnar Morling
0c4190c493 DBZ-516 Using Duration instead of long in a few more places 2018-01-18 14:13:58 +01:00
Jiri Pechanec
24bdcaf059 DBZ-516 Return control to Connect periodically 2018-01-18 14:13:58 +01:00
Gunnar Morling
5c88431c07 DBZ-494 Making tests more lenient towards specific List implementations; 
also fixing a few typos.
 2018-01-15 10:40:50 +01:00
Jiri Pechanec
5756f6dc40 DBZ-494 Array elements goes through value conversion 2018-01-15 10:24:54 +01:00
Jiri Pechanec
a629591f1f DBZ-506 Support for char and varchar arrays 2018-01-05 13:52:19 +01:00
Jiri Pechanec
ac5ed3bd92 DBZ-510 snapshot.select.statement.overrides for Postgres 2018-01-05 04:15:36 +01:00
jchipmunk
8df63e894d DBZ-519 NullPointerException happened for PAUSED task 
PostgresConnectorTask throws java.lang.NullPointerException during commit for PAUSED connector after the restart of Kafka Connect application
 2018-01-02 10:54:21 +01:00
Jiri Pechanec
24e305ad57 DBZ-525 Timestamp supports optional microseconds part 2018-01-02 08:52:27 +01:00
Jenkins user
6bb34b42f9 [maven-release-plugin] prepare for next development iteration 2017-12-20 07:15:12 +00:00
Jenkins user
16dcd4c980 [maven-release-plugin] prepare release v0.7.1 2017-12-20 07:15:12 +00:00
Jiri Pechanec
42d103294e DBZ-518 Remove a leftover event 2017-12-20 06:06:41 +01:00
Jiri Pechanec
ecf6a86594 DBZ-517 Manually override RDS environment 2017-12-20 06:05:59 +01:00
Jiri Pechanec
ffce17484e DBZ-512 Need to use OID with the old Postgres connector version 2017-12-20 06:04:40 +01:00
Jenkins user
5e09932cb9 [maven-release-plugin] prepare for next development iteration 2017-12-15 05:10:23 +00:00
Jenkins user
6c1d61e03b [maven-release-plugin] prepare release v0.7.0 2017-12-15 05:10:23 +00:00
Gunnar Morling
4e8cedd094 DBZ-379 Postgres connector minimizes use of JDBC metadata 2017-12-13 12:20:37 +01:00
Jiri Pechanec
d4daf639f7 DBZ-498 Unknown types can be handled as binary data 2017-12-11 13:44:25 +01:00
Jiri Pechanec
104f1bc466 DBZ-501 Avoid race condition in stop of thread executor and task submission 2017-12-06 13:21:15 +01:00
rkerner
c7ac481c43 [DBZ-342] fix broken MySQL data type "TIME" handling 2017-11-29 20:34:12 +01:00
Gunnar Morling
4309164f27 DBZ-453 Improving description of configuration option; also 
* some JavaDoc adjustments
* removing some dead code
 2017-11-29 15:14:19 +01:00
Gunnar Morling
5dae38bfab DBZ-453 Keeping existing behavior of omitting fields of unsupported column types from change messages; 
Also removing confusing log message as those fields are not propagated as binary data actually.
 2017-11-29 15:14:19 +01:00
Robert Coup
65a8113cb7 Deal with non-builtin PG types. 
Fix DBZ-454 issue with mapping to the wrong OID.
Return byte[] as the standard approach
 2017-11-29 15:14:19 +01:00
Robert Coup
71d6a1bcb6 DBZ-453 add support for newer wal2json with verbose type specifiers. 2017-11-29 15:14:19 +01:00
Jiri Pechanec
b60ff5e863 DBZ-478 Fix null array handling for wal2json 2017-11-23 14:53:46 +01:00
Gunnar Morling
833e81a3a4 DBZ-478 Avoiding NPE in case of unsupported array column types for Postgres; 
Instead, just the warning about the unsupported type will be logged.
 2017-11-22 09:43:10 +01:00
Gunnar Morling
b1cb58f7cc DBZ-478 Correctly handling null values for array-typed columns 2017-11-22 09:43:10 +01:00
Jiri Pechanec
3fae495bc7 DBZ-259 Return correct value for null Duration 2017-11-14 13:04:25 +01:00
Gunnar Morling
5fbe742be8 DBZ-285 Specifying scope of dependencies in the individual POMs for the sake of comprehensibility 2017-11-10 16:48:32 +01:00
Scofield Xu
0697ad7cd9 DBZ-424 Can't support PostgreSQL 10 2017-11-03 15:05:23 +01:00
Gunnar Morling
38bda6625a DBZ-416 Removing unneccessary configuration of build helper plug-in; 
It would have been needed for the Postres module only anyways. But it seems the generator plug-in is adding that source path automatically to the compilation already, so it's not needed at all.
 2017-10-26 15:23:02 +02:00
Jiri Pechanec
faf3e92f6c DBZ-404 Postgres container is detected as started after initialization 2017-10-20 08:45:13 +02:00
Ben Williams
a3b4fedd5f DBZ-363 Add support for BIGINT UNSIGNED handling for MySQL 2017-10-18 10:20:03 +02:00
Jiri Pechanec
cf856f1a06 DBZ-258 Added test for date/time parser 2017-10-18 09:21:22 +02:00
Gunnar Morling
9e563c8333 DBZ-258 Avoiding some data copying by directly accessing underlying message structure from column implementations 2017-10-18 09:21:22 +02:00
Gunnar Morling
73189892b3 DBZ-258 Misc. improvements while reviewing the change; 
* Removing superfluous config option
* Making loggers static
* JavaDoc fixes
* Extracting hexStringToByteArray() to helper and adding test
* Removing superfluous super() invocations
 2017-10-18 09:21:22 +02:00
Jiri Pechanec
a78ef1a1d3 DBZ-258 Fix test when messages are coming in batch 2017-10-18 09:21:22 +02:00
Jiri Pechanec
fa5090a71e DBZ-258 timestamptz can contain optional usec part 2017-10-18 09:21:22 +02:00
Jiri Pechanec
e47b4cb81c DBZ-258 Changes after first review 2017-10-18 09:21:22 +02:00
Jiri Pechanec
0bc8129961 DBZ-258 Support for wal2json plugin 2017-10-18 09:21:22 +02:00
Jiri Pechanec
271f87afda DBZ-380 Task is failed on exception in RecordsStreamProducer 2017-09-29 10:54:27 +02:00
Jenkins user
75937711fa [maven-release-plugin] prepare for next development iteration 2017-09-21 04:42:02 +00:00
Jenkins user
a89b9332e4 [maven-release-plugin] prepare release v0.6.0 2017-09-21 04:42:02 +00:00
Gunnar Morling
79fefa3cce DBZ-356 Making sure connection closing exception isn't swallowed 2017-09-19 12:08:56 +02:00
Gunnar Morling
abaf233066 DBZ-356 Closing typeResolverConnection upon RecordsStreamProducer#close() 2017-09-19 12:08:56 +02:00
Jiri Pechanec
70672525ad DBZ-356 Close leaked JDBC connection 2017-09-19 12:08:56 +02:00
Jiri Pechanec
604c2cba7f DBZ-279 Consume SQL warnings 2017-09-19 08:53:06 +02:00
Jiri Pechanec
ba3d7d762b DBZ-318 Support for a Decimale with variable scale schema 2017-09-19 08:39:07 +02:00
Jiri Pechanec
b53b4f67d2 DBZ-319 Views are now ignored in snapshots 2017-09-15 11:37:38 +02:00
Jenkins user
214696ef0c [maven-release-plugin] prepare for next development iteration 2017-08-17 11:51:05 +00:00
Jenkins user
c867e6fea6 [maven-release-plugin] prepare release v0.5.2 2017-08-17 11:51:05 +00:00
Steven Siahetiong
6c34aea9d1 DBZ-337 Added ability to treat Postgres NUMERIC as double 2017-08-17 09:55:50 +02:00
Gunnar Morling
9d2993f005 DBZ-298 Moving two methods only used in tests to test itself 2017-08-17 09:08:17 +02:00
Gunnar Morling
1665026c8a DBZ-298 Adding support for quoted identifiers for schemas, tables and 
columns; expanding test
 2017-08-17 09:08:17 +02:00
Emrul Islam
b9be392cd9 DBZ-298 Adding test case for quoted table and column names in Postgres 2017-08-17 09:03:16 +02:00
Gunnar Morling
564d90dc09 DBZ-298 Formatting 2017-08-17 09:03:16 +02:00
Emrul Islam
bb79b2f60f DBZ-297 Initial support for some Postgres array types (one-dimensional only) 2017-07-27 10:04:13 +02:00
Gunnar Morling
bddf715324 DBZ-287 Avoiding double conversion; changing test to use a larger scale 2017-07-25 14:34:21 +02:00
Matteo Capitanio
7fb8829cb1 DBZ-287 Corrected scale representation for Postgres NUMERIC/DECIMAL 2017-07-25 14:14:40 +02:00
Jiri Pechanec
1f2061c2fc DBZ-286 Connection keepAlive support 2017-07-14 15:17:10 +02:00
Jiri Pechanec
c2d6d16803 DBZ-301 Providing a hint for test extensions 2017-07-13 12:46:25 +02:00
Gunnar Morling
a8d1817c22 [maven-release-plugin] prepare for next development iteration 2017-06-09 16:14:31 +00:00
Gunnar Morling
3f512aace7 [maven-release-plugin] prepare release v0.5.1 2017-06-09 16:14:31 +00:00
Duncan Sands
19a4b0a012 DBZ-282 Correct the text describing the default values to match the actual defaults 2017-06-09 10:50:45 +02:00
Jiri Pechanec
168914c9fb DBZ-281 Fail task when connection to db is lost 2017-06-09 10:27:19 +02:00
Gunnar Morling
aa4f102ff0 DBZ-280 Expanding test 2017-06-08 22:17:29 +02:00
arosenber
616ba945bf DBZ-280 - Explicitly handle the tstzrange type to directly push the postgres string representation of the column type 2017-06-07 10:08:02 -04:00
Gunnar Morling
a88527df7d DBZ-250 Making status update interval configurable; 
Using the default of PG (10 seconds) helps to reduce CPU utilization, but it causes our tests to run much longer,
as the connector will only stop after the next status update.
So we are using the default in general now, only for the tests the default is overridden with a much smaller value.
 2017-06-01 14:52:33 +02:00
Duncan Sands
d7539e03d2 DBZ-250 Fix confusion between int with special value 0 and Integer with special value 
null, which resulted in high CPU usage due to ceaselessly polling postgres
(createReplicationConnection uses zero with the intent of disabling polling).
 2017-06-01 14:52:28 +02:00
Gunnar Morling
f17ced3b4a DBZ-262 Letting configuration enum types implement EnumeratedValue 2017-05-23 21:26:22 +02:00
Randall Hauch
787959c4d0 DBZ-232 Removed the database and table recommenders 
It’s not clear how valuable these recommenders actually are. First, it’s not clear about the expected semantics: can the user use values that don’t appear in the recommended values? Second, the recommenders that return large numbers of values can be slow and can result in very large REST API responses.

Debezium was using recommenders to return the database and table/collection names, but these lists can be very large for large databases. Rather than cap the number of recommended values and have the recommender return a subset of all potential values, we will instead remove the recommenders altogether.
 2017-05-19 09:24:07 +02:00
Randall Hauch
e365fc2af9 Merge pull request #225 from jpechane/ssl-factory-only 
[DBZ-244] Allow unvalidated SSL connections
 2017-05-10 16:19:49 -05:00
Jiri Pechanec
58c048950b Fixed import 2017-05-09 12:01:26 +02:00
Jiri Pechanec
ec915c559c [DBZ-246] Correctly handle enumeratedvale type 2017-05-09 11:24:28 +02:00
Jiri Pechanec
e8fc6d1a8c Allow unvalidated SSL connections 2017-05-09 08:02:56 +02:00
Gunnar Morling
80903b0622 DBZ-237 Using regular Postgres default port in tests 2017-05-05 22:25:17 +02:00
Gunnar Morling
64958d11db DBZ-238 Moving EnumeratedValue to its own file 2017-05-05 21:55:46 +02:00
Brendan Maguire
928c9cb5f0 DBZ-238: Fix Postgres database.sslmode = "require" bug 
* Also added EnumeratedValue interface for Configuration so getValue is used instead of toString when getting the config value
 2017-05-05 21:55:45 +02:00
Randall Hauch
709cd8f3fe [maven-release-plugin] prepare for next development iteration 2017-03-27 11:28:12 -05:00
Randall Hauch
2bc3d45954 [maven-release-plugin] prepare release v0.5.0 2017-03-27 11:28:11 -05:00
Randall Hauch
81f62b6961 DBZ-205 Corrected MySQL connector to handle 2-digit years 
MySQL has special handling of 2-digit years that it deems are ambiguous, such as the year value `17` that is actually treated as `2017`. Apparently the 2-digit values are stored in MySQL and the interpretation is performed when the data is extracted, so therefore the connector needs to also perform this adjustment of the year values. This commit uses the JDK’s `TemporalAdjuster` interface and passes this down to the requisite temporal-related datatype handling code. The MySQL connector then provides its own `TemporalAdjuster` implementation that adjusts the year values via the excellend JDK `Temporal` methods.

A row in one of the MySQL test databases was changed to use a 2-digit year of `16` while the test method still checks that the year is still 2016`, verifying that the year value is properly adjusted.
 2017-03-27 10:58:21 -05:00
Randall Hauch
430d756062 [maven-release-plugin] prepare for next development iteration 2017-03-17 15:41:58 -05:00
Randall Hauch
536cbf6300 [maven-release-plugin] prepare release v0.4.1 2017-03-17 15:41:57 -05:00
Randall Hauch
8c60c29883 [maven-release-plugin] prepare for next development iteration 2017-02-07 14:22:12 -06:00
Randall Hauch
20134286e9 [maven-release-plugin] prepare release v0.4.0 2017-02-07 14:22:11 -06:00
Randall Hauch
f0db8d1b1f DBZ-179 Corrected JavaDoc in PostgreSQL connector 
Corrected the JavaDoc and removed trailing spaces in the PostgreSQL connector code.
 2017-01-20 11:51:17 -06:00
Randall Hauch
e11f242b00 DBZ-179 Moved generated source for Protobuf 
The project requires that all JavaDoc for public methods exist and are valid (e.g., have all @param, @return and @throws to match the signature). However, the generated Java source for Protobuf contain numerous JavaDoc errors relative to these settings. This causes lots of errors inside Eclipse (and probably other IDEs), but ignoring/disabling the JavaDoc errors leads to improper JavaDoc (fixed in next commit). By moving the generated Protobuf source code to a separate directory (e.g., `generated-sources`), the IDEs will automatically discover the directory and the user can ignore any compiler and JavaDoc errors/warnings for those files while keeping the more strict JavaDoc checking enabled for the rest of the code.
 2017-01-20 11:50:08 -06:00
Horia Chiorean
c024d0789b DBZ-177 Changes the way the PostgreSQL connector loads the JDBC driver, to use the connector's classloader 2017-01-18 17:59:20 +02:00
Randall Hauch
eeb4eafacf DBZ-3 Changed the PostgreSQL connector’s use of Docker containers 
The PostgreSQL connetor was not able to build locally, since the Maven build would wait forever trying to talk to the TCP port for PostgreSQL before starting the integration tests. Even when I corrected the `wait` specification to use the localhost (rather than the direct container address), the build successfully connected to Postgres when it started the first time but before it shutdown to adjust the configuration, and thus the tests failed as the server was shutdown. The build now looks for a specific log message which is unique and output by the container after the second startup, and this seems to work great (at least locally).
 2017-01-13 16:44:30 -06:00
Horia Chiorean
c6b594e100 DBZ-3 Adds the assembly profile to the connector's POM 2017-01-04 11:41:52 +02:00
Horia Chiorean
ae85656851 DBZ-3 Fixes topic naming to include the name of the server 2016-12-30 09:42:48 +02:00
Horia Chiorean
d681c11eee DBZ-3 Adds README and NOTES documentation files for the postgres connector module 2016-12-27 14:44:34 +02:00
Horia Chiorean
6159524618 DBZ-3 Removes the sources for the Postgres JDBC driver and replaces them with a SNAPSHOT release which contains the streaming support 2016-12-27 14:44:33 +02:00
Horia Chiorean
a300d3e1cf DBZ-3 Changes the configuration of the Docker Maven plugin to only use alias naming when necessary and moves the PG connector ahead of the Mongo connector in the build 2016-12-27 14:44:33 +02:00
Horia Chiorean
737614a555 DBZ-3 Implements a connector for streaming changes from a Postgres database 
The version of the DB server required for this to work is at least 9.4. To be able to stream logical changes, the code relies on enhancements to the JDBC driver which are not yet public. Therefore, the current codebase includes the sources for the JDBC driver.
The commit also updates the general DBZ build system for:
* custom checkstyle package exclusions - required by the Postgres driver the protobuf code for now
* adds support for debugging Surefire and Failsafe
 2016-12-27 14:44:32 +02:00
Horia Chiorean
23e3f59fa1 DBZ-3 Implements a connector for streaming changes from a Postgres database 
The version of the DB server required for this to work is at least 9.4
The commit also updates the general DBZ build system for:
* custom checkstyle package exclusions - required by the Postgres driver the protobuf code for now
* adds support for debugging Surefire and Failsafe
 2016-12-27 14:44:32 +02:00
Horia Chiorean
f13208a229 DBZ-3 Adds a local copy of the PG driver which contains the logical replication support 2016-12-27 14:44:31 +02:00
Horia Chiorean
8e14f150db DBZ-3 Adds the structure for a Postgres connector which uses a Debezium Postgres docker image that has the decoderbufs plugin enabled to read WAL changes 2016-12-27 14:44:29 +02:00
Randall Hauch
fa4ae33ba2 Removed unused modules 2016-03-17 16:13:50 -05:00
Randall Hauch
046fc83850 DBZ-23 Simplified PosgreSQL Connector's use of Docker plugin 2016-02-25 10:24:52 -06:00
Randall Hauch
1d46e59048 DBZ-17 Minor changes to the POMs 2016-02-18 13:58:29 -06:00
Christian Posta
c730685a01 add option to run without integration tests 2016-02-15 16:26:32 -07:00
Randall Hauch
0e58dba9d6 DBZ-1 Renamed the connector modules and packages 2016-02-02 16:58:48 -06:00