alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
1,705 Commits 3 Branches 0 Tags 51 MiB
540a951211
Commit Graph

345 Commits

Author SHA1 Message Date
Jiri Pechanec
920e0cbd3f DBZ-1064 Set default value only for non-nullable fields 2019-01-16 13:01:36 +01:00
Jiri Pechanec
484a8977bc DBZ-1083 Passing "include-unchanged-toast" option only if supported; 
The "include-unchanged-toast" option was removed in recent wal2json versions, without a transition phase. So the connector tries not to connect first giving the option, and if that fails, without it.
 2019-01-16 09:37:55 +01:00
Gunnar Morling
af01d10886 DBZ-1081 JavaDoc; adding Addison Higham to COPYRIGHT.txt 2019-01-15 18:09:08 +01:00
Addison Higham
889b307b9d DBZ-1081 Fallback to restart_lsn in pg 9.5 
In pg 9.5, confirmed_flush_lsn is not availiable. However, there is
restart_lsn, which should be safe to use with a downside of producing
some extra duplicate records.

This page: https://paquier.xyz/postgresql-2/postgres-9-6-feature-highlight-replication-slot-improvements/
indicates as such and this allows for DBZ to support PG 9.5
 2019-01-15 18:05:55 +01:00
Gunnar Morling
de356896c0 DBZ-1029 Adding one more test 2019-01-15 17:41:42 +01:00
Ilia Bogdanov
88521371cc DBZ-1029 insertion test update, code cleanup after review 2019-01-14 12:03:19 +03:00
Ilia Bogdanov
cee9bd9a5d DBZ-1029 handling messages from tables with replica identity full in postgres connector. 2019-01-14 12:03:19 +03:00
Amit Sela
b0b7d942a3 DBZ-1073 Adding scale as schema parameter, if present 2019-01-11 15:40:30 +01:00
Grzegorz Kołakowski
91c72738c0 DBZ-1067 Add column blacklist field to RelationalDatabaseConnectorConfig 2019-01-09 18:13:26 +01:00
Jenkins user
5275f73424 [maven-release-plugin] prepare for next development iteration 2018-12-19 13:06:12 +00:00
Jenkins user
b6569c18ae [maven-release-plugin] prepare release v0.9.0.Beta2 2018-12-19 13:06:12 +00:00
Gunnar Morling
92ae01ccdf DBZ-996 Adding column name to log message 2018-12-18 17:53:42 +01:00
Jiri Pechanec
01eee09f90 DBZ-996 Tolerate race condition in PK updates 2018-12-18 17:52:28 +01:00
Jiri Pechanec
edec1c3090 DBZ-978 Added last event and captured tables metrics 2018-12-13 20:52:11 +01:00
Gunnar Morling
5542c0de29 DBZ-966 Making sure Postgres connector can be stopped while awaiting initial snapshot delay; 
* Also skipping snapshotting when requested to stop during initial delay
* Showing remaining delay time in logs
* Adding missing config
 2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
e2942ab72a DBZ-966 Replace sleep with polling loop 2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
35a5acfd8b DBZ-966 Enable to delay initial snapshot in PostgreSQL connector 2018-12-13 14:29:54 +01:00
Gunnar Morling
0833b23feb DBZ-842 Expanding test to cover TOASTed NOT NULL column, too 2018-12-11 18:42:51 +01:00
Jiri Pechanec
94c0a42286 DBZ-842 Do not send TOASTed values from wal2json 2018-12-11 17:10:58 +01:00
Jiri Pechanec
2b2ff0754d DBZ-960 Run SnapshotReader in a single transaction 2018-12-11 14:15:53 +01:00
Gunnar Morling
470cd1eaba DBZ-1024 Doing some clean-up 2018-12-07 15:39:59 +01:00
Jiri Pechanec
ea5c8512b7 DBZ-1024 Return test assert 2018-12-07 15:39:59 +01:00
Jiri Pechanec
0b78f0c31a DBZ-1024 Emit correct last_snapshot_record in source 2018-12-07 15:39:59 +01:00
Jiri Pechanec
9c225c63eb DBZ-1021 Ignore events for dropped table 2018-12-06 18:54:50 +01:00
Gunnar Morling
101bd8ef6d DBZ-1000 Making Checkstyle happy 2018-12-03 15:00:43 +01:00
Jiri Pechanec
7a3cbd6cb6 DBZ-1000 Detect nullability change in schema 2018-12-03 14:54:36 +01:00
Jiri Pechanec
68766e9355 DBZ-1009 Support for unknown types during snapshot 2018-11-30 11:16:36 +01:00
Jenkins user
db1d3a7fb8 [maven-release-plugin] prepare for next development iteration 2018-11-20 16:15:14 +00:00
Jenkins user
f83db82cea [maven-release-plugin] prepare release v0.9.0.Beta1 2018-11-20 16:15:14 +00:00
Jiri Pechanec
0611d4975f DBZ-965 Reduce the test time 2018-11-13 09:16:30 +01:00
Jiri Pechanec
a4abd2a336 DBZ-965 Test of flushed LSN 2018-11-13 09:15:09 +01:00
Gunnar Morling
5a9ca6eb31 DBZ-965 Some clean-up 2018-11-13 09:15:09 +01:00
Jiri Pechanec
675f5b6479 DBZ-965 Renames for better clarification 2018-11-13 09:15:09 +01:00
Gunnar Morling
b455b7ccaf DBZ-970 Making LogicalDecoder#getPostgresPluginName() abstract 2018-11-13 09:15:09 +01:00
Jiri Pechanec
390965f4f9 DBZ-965 Track latest LSN per each change 2018-11-13 09:15:09 +01:00
Jiri Pechanec
b1707c395d DBZ-970 Correct plging name for wal2json_streaming decoder 2018-11-13 09:15:09 +01:00
Jiri Pechanec
83137d0a41 DBZ-965 Force heartbeat after end of each TX 2018-11-13 09:15:09 +01:00
Gunnar Morling
efbd46b7f6 DBZ-953 Having PG connector use decimal mode option from RelationalDatabaseConnectorConfig 2018-11-08 09:23:28 +01:00
Jiri Pechanec
3a8293a4ba DBZ-937 Disable auto-handling of unknown types in test 2018-10-11 09:55:07 +02:00
Olavi Mustanoja
0c986b4a78 DBZ-937 add wal2json replication message handling for CITEXT 2018-10-09 08:46:55 +02:00
Jenkins user
d952f5dfb0 [maven-release-plugin] prepare for next development iteration 2018-10-04 11:59:14 +00:00
Jenkins user
ff9b70278b [maven-release-plugin] prepare release v0.9.0.Alpha2 2018-10-04 11:59:13 +00:00
Gunnar Morling
1f25536e2b DBZ-934 Using API for creating replication slot if possible 2018-10-04 10:12:48 +02:00
PSanetra
dd3859d8bc DBZ-934 Use temporary replication slots if slot.drop_on_stop is enabled and Postgres >= 10 is used 2018-10-04 10:12:48 +02:00
jchipmunk
3ec0b3be81 DBZ-918 Moving connector identifier from SourceInfo to Module class 2018-10-04 09:00:13 +02:00
jchipmunk
05ba99bb0a DBZ-918 Adding Debezium connector field to source info 
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
 2018-10-04 09:00:13 +02:00
Gunnar Morling
984081dd1a DBZ-926 Adding flushed LSN to log message 2018-10-03 09:47:38 +02:00
Gunnar Morling
08ec4f48d0 DBZ-911 Style adjustments; wording fix 2018-10-01 12:49:56 +02:00
Ian Axelrod
427249887b DBZ-911 Simplifying schemaChanged() shortcircuit logic 
Break up initial conditional and comment missing column portion to
clarify the role of skipRefreshSchemaOnMissingToastableData.
 2018-10-01 12:49:48 +02:00
Ian Axelrod
9f083d0f1b DBZ-911 Simplifying getToastableColumnsForTableId() impl. 
No need to wrap ArrayList instance in unmodifiableList. Can just use
Collections.emptyList.
 2018-10-01 12:49:36 +02:00
Ian Axelrod
ad433c5747 DBZ-911 Don't refresh schema on absent TOAST cols 
Improves PostgreSQL RecordsStreamProducer performance for processing up
dates to a table with TOASTable columns, where those updates do not
affect said columns. Prior to this fix, these updates would trigger a
refresh of the in-memory table schema. In the worst case, this means a
query for every update. This puts significant load on the database
server and adds tens of milliseconds to the processing of each update
record.

The fix requires a new configuration option, called
'schema.refresh.mode'. This option has values 'columns_diff' (the
default) and 'columns_diff_exclude_unchanged_toast'. 'columns_diff'
maintains the pre-fix behavior. 'columns_diff_exclude_unchanged_toast'
activates the fix.

The fix must be toggleable because it decreases the consistency guarantees
Debezium provides, since in-memory table schemas may not stay
synchronized with their remote counterparts. With type metadata included
in the replication message, inconsistencies are limited to the unchanged
toast columns.
 2018-10-01 11:39:32 +02:00
Gunnar Morling
6f7bdbe5e1 DBZ-898 Using 'Json' semantic type when encoding HSTORE column values as string 2018-09-25 12:22:20 +02:00
Gunnar Morling
7150e387c7 DBZ-898 Misc. clean-up: 
* Using database encoding for string conversion
* Not making hstore schemas optional by default
* Using Jackson instead of GSon for JSON serialization
* Removing superfluous method and log messages
* Adjusting to naming and style conventions
 2018-09-24 13:36:28 +02:00
SyedMuhammadSufyian
623cc98cb0 Change log levels 2018-09-22 20:15:07 +02:00
SyedMuhammadSufyian
346f2b3761 Handling HStore type , hstore.handling.mode:map|json , default is map.Added support for protobuff 2018-09-22 20:07:13 +02:00
SyedMuhammadSufyian
38f4a0e65b HStore Data type support , configuration options are map|json , default is map 2018-09-22 05:54:45 +02:00
Gunnar Morling
f9ede861ba DBZ-912 Upgrading to Postgres driver 42.2.5 2018-09-19 15:26:44 +02:00
Gunnar Morling
39a47fabfa DBZ-899 Using custom statement to obtain SQL type info in order to avoid N+1 issues with PgDatabaseMetaData#getTypeInfo() 2018-09-19 09:51:38 +02:00
Gunnar Morling
3331d4c1a9 DBZ-899 Removing unused field; 
Also inlining method only used in one place.
 2018-09-19 09:51:38 +02:00
Jiri Pechanec
59fbcca725 DBZ-907 Log refreshing schema due to different column count 2018-09-12 19:07:01 +02:00
Jiri Pechanec
e20c183d5d DBZ-895 Make tcpKeepalive enabled by default 2018-09-11 09:34:21 +02:00
Gunnar Morling
e1bbbd7992 DBZ-865 DBZ-878 Replacing invalid topic name characters with "_"; 2018-09-05 12:56:53 +02:00
Gunnar Morling
bd692b8b84 DBZ-862 Extracting values for controlling retry attempts into constants 2018-09-03 14:50:54 +02:00
Jiri Pechanec
c6e20d4967 DBZ-862 Fix test interdependency problem 2018-09-03 14:40:57 +02:00
Jiri Pechanec
1ff53e35ec DBZ-862 Fix Travis timing issue 2018-09-03 14:40:57 +02:00
Jiri Pechanec
4b9141e1fd DBZ-862 Handle concurrent transaction blocking taking snapshot 2018-09-03 14:40:57 +02:00
Jiri Pechanec
e65fb26ae9 DBZ-887 Close connection even for non-started producer 2018-08-31 13:02:23 +02:00
Gunnar Morling
ae879cf475 DBZ-859 Adjustments after rebase; typo fix 2018-08-23 12:00:44 +02:00
Jiri Pechanec
cd6dc09e72 DBZ-859 Heartbeat is sent after PostgreSQL snapshot 2018-08-23 11:44:53 +02:00
oripwk
7d958a1928 DBZ-866 PR fixes 2018-08-22 17:09:26 +02:00
oripwk
d82372a6b2 DBZ-866 Add schema, database and table to Postgres connector 2018-08-22 17:09:26 +02:00
Jiri Pechanec
d4e64a61cb DBZ-705 Allow to configure postgres config file location 2018-08-13 20:21:41 +02:00
Jenkins user
e00dad127f [maven-release-plugin] prepare for next development iteration 2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700 [maven-release-plugin] prepare release v0.9.0.Alpha1 2018-07-26 08:00:12 +00:00
Gunnar Morling
3061c1dc0b DBZ-815 Passing offset to Heartbeat#heartbeat(); 
That's simpler to grasp than the approach of passing a supplier lambda to the constructor.
Also it allows to pass on the offset via local variables instead of instance fields in some cases.
 2018-07-25 14:49:52 +02:00
Gunnar Morling
8e1450bc71 DBZ-800 Adding test for receiving heartbeat events with Postgres 2018-07-20 13:37:54 +02:00
Gunnar Morling
4abc471bcb DBZ-800 Emitting heartbeat events in Postgres streaming loop 2018-07-20 13:37:54 +02:00
Gunnar Morling
20db9299c5 DBZ-627 Unifying TopicSelector implementations 2018-07-18 12:36:07 +02:00
Gunnar Morling
1523f230ca DBZ-793 Using TableFilter consistently; 
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
 2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8 [maven-release-plugin] prepare for next development iteration 2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f [maven-release-plugin] prepare release v0.8.0.Final 2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d [maven-release-plugin] prepare for next development iteration 2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0 [maven-release-plugin] prepare release v0.8.0.CR1 2018-07-04 07:07:44 +00:00
Jiri Pechanec
c2f2f686f0 DBZ-644 Disable test making problems in Travis CI 2018-07-03 07:03:59 +02:00
Jiri Pechanec
646a66672c DBZ-644 Close unused replication connection 2018-07-03 07:03:59 +02:00
Gunnar Morling
dcd6b01134 DBZ-644 Simplifying RelationalDatabaseSchema constructor; 
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
 2018-07-03 07:03:59 +02:00
Gunnar Morling
b0980b994c DBZ-644 Adding test for Postgres 2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema 2018-07-03 07:03:59 +02:00
Jiri Pechanec
0182eecd06 DBZ-778 Make the tests resilient against race condition 2018-07-02 11:52:23 +02:00
Gunnar Morling
a7be3847e3 DBZ-773 Removing some dead code and unused methods 2018-06-29 09:27:57 +02:00
Gunnar Morling
c4c8cbc3ab DBZ-773 Moving management of Tables to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab DBZ-773 Moving management of table schemas to RelationalDatabaseSchema 2018-06-29 09:09:10 +02:00
Gunnar Morling
0600ccf213 DBZ-751 Incorporating review remarks; 
* Expanding test
* Extracting constant for parameter key in tests
 2018-06-27 14:48:59 +02:00
Gunnar Morling
a292b05a96 DBZ-751 Propagating DECIMAL column precision to Avro schemas 2018-06-27 14:48:59 +02:00
Jiri Pechanec
ca6b1c9168 DBZ-743 Empty password for PostgreSQL and MySQL 2018-06-27 11:18:19 +02:00
Gunnar Morling
f9c1cfc134 DBZ-762 Adding support for CITEXT array columns 2018-06-26 11:40:19 +02:00
Gunnar Morling
daeeaa74cd DBZ-762 Formatting 2018-06-26 11:40:19 +02:00
Olavi Mustanoja
c401a98166 DBZ-762 Added support for CITEXT column type 2018-06-26 11:40:19 +02:00
Gunnar Morling
bf7a5018ca Setting POM version back to 0.8.0-SNAPSHOT 2018-06-22 12:21:03 +02:00
Jenkins user
db42e4657a [maven-release-plugin] prepare for next development iteration 2018-06-21 14:07:45 +00:00