alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
1,903 Commits 3 Branches 0 Tags 51 MiB
21176ff70f
Commit Graph

412 Commits

Author SHA1 Message Date
jchipmunk
9fe033ce24 DBZ-1247 Specifying default snapshot fetch size for each connector config 2019-04-30 09:57:11 +02:00
jchipmunk
a5cc83c0d1 DBZ-1247 Adding property to specify snapshot fetch size 2019-04-30 09:57:11 +02:00
jorkzijlstra
d0e2a72cd1 DBZ-1241 Allowing null values again for commitTime 2019-04-30 09:45:10 +02:00
shubham
f55bec86d3 Added support for all range types 2019-04-24 07:40:03 +02:00
krizhan
70c564b4c2 DBZ-212 Moved SERVER_NAME to RelationalDatabaseConnectorConfig 2019-04-23 12:29:44 +02:00
krizhan
38f3856c31 DBZ-212 Remove PostgreSQL getLogicalName function 2019-04-23 12:29:44 +02:00
Dennis Campagna
173e8f5854 DBZ-1034 Support range column types on Postgres 
Signed-off-by: sciamp <denniscampagna@gmail.com>

DBZ-1034 Support range column types on Postgres

Signed-off-by: sciamp <denniscampagna@gmail.com>
 2019-04-12 09:44:17 +02:00
GraySmog
726a2afcd6 Add comments stating the reason for skipping tests 2019-04-12 09:27:45 +02:00
GraySmog
08ca02f8f4 DBZ-1193 Add support for network type MACADDR and MACADDR8 in PostgreSQL connnector 2019-04-12 09:27:45 +02:00
Jenkins user
cefc5ccfb9 [maven-release-plugin] prepare for next development iteration 2019-04-11 12:07:16 +00:00
Jenkins user
fc3026adc3 [maven-release-plugin] prepare release v0.9.4.Final 2019-04-11 12:07:15 +00:00
Gunnar Morling
2250c47b7f DBZ-1163 Minor improvements 2019-04-10 10:53:22 +02:00
preethi
02873e8169 DBZ-1163 Enable postgres snapshot for tables without primary key 2019-04-10 10:48:11 +02:00
ShubhamRwt
762d908eef DBZ-1193 Added support for cidr address type 2019-04-08 15:00:03 +02:00
jchipmunk
7c0ae3ee20 DBZ-1212 SLF4J usage issues 2019-04-04 21:32:12 +02:00
Jiri Pechanec
ff52f3713e DBZ-1144 Test for geometry in public schema 2019-04-02 15:57:57 +02:00
Alexander Kovryga
4616df189e DBZ-1189 Add support for inet data type. 2019-03-28 09:57:32 +01:00
ShubhamRwt
540a951211 DBZ- 362 Adding WhitespaceAfter check to Checkstyle 2019-03-28 09:24:11 +01:00
Jenkins user
8b0edd1871 [maven-release-plugin] prepare for next development iteration 2019-03-25 12:16:31 +00:00
Jenkins user
56f98a48b2 [maven-release-plugin] prepare release v0.9.3.Final 2019-03-25 12:16:30 +00:00
Renato Mefi
6300f2972f DBZ-1169 Integration tests for Outbox EventRouter 
Set the Snapshot Mode to NEVER provides more consistent tests
 2019-03-25 11:27:44 +01:00
Renato Mefi
8b689d7f03 DBZ-1169 Implement RegexRouter based on config 2019-03-25 11:27:44 +01:00
Renato Mefi
f0f13ed4a4 DBZ-1169 Implement basic behavior 2019-03-25 11:27:44 +01:00
Renato Mefi
2ee1c98298 DBZ-1169 Setup EventRouter on Postgres Test IT 2019-03-25 11:27:44 +01:00
shubham
8cca21e969 DBZ-1039 Make all ifs with braces 2019-03-25 10:23:14 +01:00
Addison Higham
8e21139ca3 DBZ-1082 Add new custom recovery mode, more metadata 
This commit does a few things:

- Refactors snapshot modes to be encapsulated by an interface and
to only use that interface in determining when to snapshot and in
determing the type of the `RecordProducer` interface to instantiate
- Refactors the configuration of existing snapshot modes to tie the
existing snapshot modes to their aligned implementation
- Adds a new snapshot.mode, custom, and a new configuration option to
specify a custom implementation that will be loaded by the class loader
- Changes the visibility of some classes to allow for custom snapshot
modes to get enough context to make an informed choice
- Adds some metadata about slots (the catalog_xmin) to give a full idea
of the state of slots which can be useful in implementing snapshot
modes (which is also configurable, as it can add some overhead)

Together, these changes allow for a much broader flexibility got end
users to implement a snapshot mode that can do more advanced snapshots,
such as partial recovery or for partial snapshots for tables where not
all records are needed.

This could also be seen as superseeding the
`snapshot.select.statement.overrides` to allow for users to dynamically
build queries based on the state of the slot and the offsets consumed.
 2019-03-24 11:56:40 +01:00
Jiri Pechanec
b11f1b8d05 DBZ-1161 Start snapshot in async thread 2019-03-21 14:32:47 +01:00
trolison
2001b178d7 - Fixed bad formatting 
- Added trace for when empty messages are processed
DBZ-1181
 2019-03-19 10:46:36 -07:00
trolison
df0e789601 - Add support for processing WAL2JSON events that have no changes 
DBZ-1181
 2019-03-18 20:32:03 -07:00
Gunnar Morling
2ea87727cf DBZ-1174 Using Instant instead of long for representing timestamps 2019-03-12 17:19:13 +01:00
Ashhar Hasan
0b07afd23e DBZ-1174 Convert commitTime from nanos to micros for wal2json 
- wal2json sends the txn commitTime using a function from PostgreSQL's C
  library. The value that Debezium recevies is in nanoseconds.
- decoderbufs sends the txn commitTime in microseconds.
- RecordsSnapshotProducer updates SourceInfo.ts_usec by converting
  System.currentTimeMillis() to microseconds.
- RecordsSnapshotProducer updates the SourceInfo's ts_usec field using
  message.getCommitTime().

This means that when using wal2json, the value of SourceInfo.ts_usec
is in microseconds since epoch during snapshot but is in nanoseconds
during streaming. To fix this, we changed
Wal2JsonReplicationMessage.getCommitTime() to return in microseconds.
 2019-03-12 17:19:13 +01:00
Josh Stanfield
e4b7b90637 DBZ-1162 fix to ensure hstore snapshots behave the same as streams 2019-03-04 13:39:50 +01:00
Jenkins user
50a7c568fa [maven-release-plugin] prepare for next development iteration 2019-02-23 09:53:58 +00:00
Jenkins user
28f3839804 [maven-release-plugin] prepare release v0.9.2.Final 2019-02-23 09:53:58 +00:00
Jiri Pechanec
78bfda09a6 DBZ-1146 Test full column diff 2019-02-22 14:16:03 +01:00
Gunnar Morling
5043d41b7b DBZ-1146 Expanding test 2019-02-22 14:16:03 +01:00
Jiri Pechanec
4dec844fa2 DBZ-1146 Fix table created in streaming test 2019-02-22 14:16:03 +01:00
Jiri Pechanec
fa1e632a90 DBZ-1146 Handle toastable columns for snapshotted tables 2019-02-22 14:16:03 +01:00
Jiri Pechanec
89feae32e7 DBZ-1141 Test for the default values 2019-02-21 15:02:51 +01:00
Gunnar Morling
6787d17d54 DBZ-1141 Misc. clean-up; 
* Removing redundant check for date mapping type
* Always using String as fallback value for temporal values where needed
* Pulling fallback temporal values up to JdbcValueConverters
 2019-02-21 15:02:51 +01:00
Jiri Pechanec
b9d326711d DBZ-1141 Incorrect money scale 2019-02-21 15:02:51 +01:00
Jiri Pechanec
992c31fa1f DBZ-1141 Add non-date types 2019-02-21 15:02:51 +01:00
Jiri Pechanec
41ba7128e3 DBZ-1141 Correct default falback values for connect date types 2019-02-21 15:02:51 +01:00
Keith Barber
95b2290df2 DBZ-1149 Ensure consistent Java types for fallback values when converting 2019-02-21 15:02:51 +01:00
taylor-rolison
17c6a7a82d Config option to pass optional logical decoder stream params (#783) 
DBZ-1130 Added config param slot.stream.params to the Postgres connector to allow passing optional params to logical decoder on stream start
 2019-02-21 14:06:51 +01:00
Jenkins user
b96e5fee0d [maven-release-plugin] prepare for next development iteration 2019-02-13 11:13:44 +00:00
Jenkins user
3de2d04bcd [maven-release-plugin] prepare release v0.9.1.Final 2019-02-13 11:13:44 +00:00
Jiri Pechanec
9049078607 DBZ-997 Handle de-sync of message and JDBC PK info 2019-02-13 10:13:12 +01:00
Gunnar Morling
2a486ecfff DBZ-1019 Removing legacy methods from Metronome 2019-02-11 18:11:34 +01:00
Ivan Lorenz
721e8112cd DBZ-1126 Localize era display value to default locale 2019-02-06 15:34:30 +01:00
Gunnar Morling
7076deba5a DBZ-1118 Adding JUnit rule to skip tests depending on Postgres version 2019-02-04 14:43:19 +01:00
Kevin Pullin
c414a8b942 Add test for DBZ-1118 2019-02-04 14:43:19 +01:00
Kevin Pullin
1330df8908 Postgres snapshot - On null records, clear prior value from currentRecord 2019-02-04 14:43:19 +01:00
Gunnar Morling
41b7fbe78e Adding note on testing Postgres connector with external DB 2019-01-30 09:25:12 +01:00
Gunnar Morling
cc0df61665 DBZ-1111 Allowing to use drop-slot-on-close with PG 10 or newer 2019-01-29 14:44:10 +01:00
Gunnar Morling
e84c6e2238 DBZ-962 Avoiding dependency to JAXB 2019-01-29 10:28:48 +01:00
Jenkins user
c5b7d21d3e [maven-release-plugin] prepare for next development iteration 2019-01-28 11:07:04 +00:00
Jenkins user
f1ae79ff73 [maven-release-plugin] prepare release v0.9.0.CR1 2019-01-28 11:07:04 +00:00
Gunnar Morling
ccff79e07d DBZ-1105 Removing TableImpl#columnNames field 2019-01-28 10:23:56 +01:00
Gunnar Morling
9dc2c525c9 DBZ-1083 Passing "include-unchanged-toast" option only for wal2json 2019-01-25 11:04:43 +01:00
Addison Higham
26f836e0f6 DBZ-1080 new API for EmbeddedEngine for batch/async 
This introduces a new API to the EmbeddedEngine, the ChangeConsumer,
which gives the user a more flexible option for consuming changes by
exposing groups of records as well as the ability to control the
comitting of those records.

This remainds completely backwards compatible with the old API as the
ChangeConsumer wraps the existing Consumer interface with a default
implementation.
 2019-01-18 14:06:50 +01:00
Jiri Pechanec
1cf257478b DBZ-781 Enable unstable tests 2019-01-17 08:59:42 +01:00
Tony Rizko
a921613877 Fix typo in PostgresConnection error message 2019-01-17 08:58:12 +01:00
Jiri Pechanec
ceb6c62721 DBZ-1064 Remove lazy fallback values 2019-01-16 13:01:36 +01:00
Jiri Pechanec
536e517c08 DBZ-1064 Geometry default value changes; fixes for default suppliers 2019-01-16 13:01:36 +01:00
Gunnar Morling
532c17f8aa DBZ-1064 Using dedicated functional interface instead of Consumer; 
* Commenting
* Moving type from "util" to "jdbc"
 2019-01-16 13:01:36 +01:00
Jiri Pechanec
bff7eea734 DBZ-1064 Use convert value template 2019-01-16 13:01:36 +01:00
Jiri Pechanec
920e0cbd3f DBZ-1064 Set default value only for non-nullable fields 2019-01-16 13:01:36 +01:00
Jiri Pechanec
484a8977bc DBZ-1083 Passing "include-unchanged-toast" option only if supported; 
The "include-unchanged-toast" option was removed in recent wal2json versions, without a transition phase. So the connector tries not to connect first giving the option, and if that fails, without it.
 2019-01-16 09:37:55 +01:00
Gunnar Morling
af01d10886 DBZ-1081 JavaDoc; adding Addison Higham to COPYRIGHT.txt 2019-01-15 18:09:08 +01:00
Addison Higham
889b307b9d DBZ-1081 Fallback to restart_lsn in pg 9.5 
In pg 9.5, confirmed_flush_lsn is not availiable. However, there is
restart_lsn, which should be safe to use with a downside of producing
some extra duplicate records.

This page: https://paquier.xyz/postgresql-2/postgres-9-6-feature-highlight-replication-slot-improvements/
indicates as such and this allows for DBZ to support PG 9.5
 2019-01-15 18:05:55 +01:00
Gunnar Morling
de356896c0 DBZ-1029 Adding one more test 2019-01-15 17:41:42 +01:00
Ilia Bogdanov
88521371cc DBZ-1029 insertion test update, code cleanup after review 2019-01-14 12:03:19 +03:00
Ilia Bogdanov
cee9bd9a5d DBZ-1029 handling messages from tables with replica identity full in postgres connector. 2019-01-14 12:03:19 +03:00
Amit Sela
b0b7d942a3 DBZ-1073 Adding scale as schema parameter, if present 2019-01-11 15:40:30 +01:00
Grzegorz Kołakowski
91c72738c0 DBZ-1067 Add column blacklist field to RelationalDatabaseConnectorConfig 2019-01-09 18:13:26 +01:00
Jenkins user
5275f73424 [maven-release-plugin] prepare for next development iteration 2018-12-19 13:06:12 +00:00
Jenkins user
b6569c18ae [maven-release-plugin] prepare release v0.9.0.Beta2 2018-12-19 13:06:12 +00:00
Gunnar Morling
92ae01ccdf DBZ-996 Adding column name to log message 2018-12-18 17:53:42 +01:00
Jiri Pechanec
01eee09f90 DBZ-996 Tolerate race condition in PK updates 2018-12-18 17:52:28 +01:00
Jiri Pechanec
edec1c3090 DBZ-978 Added last event and captured tables metrics 2018-12-13 20:52:11 +01:00
Gunnar Morling
5542c0de29 DBZ-966 Making sure Postgres connector can be stopped while awaiting initial snapshot delay; 
* Also skipping snapshotting when requested to stop during initial delay
* Showing remaining delay time in logs
* Adding missing config
 2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
e2942ab72a DBZ-966 Replace sleep with polling loop 2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
35a5acfd8b DBZ-966 Enable to delay initial snapshot in PostgreSQL connector 2018-12-13 14:29:54 +01:00
Gunnar Morling
0833b23feb DBZ-842 Expanding test to cover TOASTed NOT NULL column, too 2018-12-11 18:42:51 +01:00
Jiri Pechanec
94c0a42286 DBZ-842 Do not send TOASTed values from wal2json 2018-12-11 17:10:58 +01:00
Jiri Pechanec
2b2ff0754d DBZ-960 Run SnapshotReader in a single transaction 2018-12-11 14:15:53 +01:00
Gunnar Morling
470cd1eaba DBZ-1024 Doing some clean-up 2018-12-07 15:39:59 +01:00
Jiri Pechanec
ea5c8512b7 DBZ-1024 Return test assert 2018-12-07 15:39:59 +01:00
Jiri Pechanec
0b78f0c31a DBZ-1024 Emit correct last_snapshot_record in source 2018-12-07 15:39:59 +01:00
Jiri Pechanec
9c225c63eb DBZ-1021 Ignore events for dropped table 2018-12-06 18:54:50 +01:00
Gunnar Morling
101bd8ef6d DBZ-1000 Making Checkstyle happy 2018-12-03 15:00:43 +01:00
Jiri Pechanec
7a3cbd6cb6 DBZ-1000 Detect nullability change in schema 2018-12-03 14:54:36 +01:00
Jiri Pechanec
68766e9355 DBZ-1009 Support for unknown types during snapshot 2018-11-30 11:16:36 +01:00
Jenkins user
db1d3a7fb8 [maven-release-plugin] prepare for next development iteration 2018-11-20 16:15:14 +00:00
Jenkins user
f83db82cea [maven-release-plugin] prepare release v0.9.0.Beta1 2018-11-20 16:15:14 +00:00
Jiri Pechanec
0611d4975f DBZ-965 Reduce the test time 2018-11-13 09:16:30 +01:00
Jiri Pechanec
a4abd2a336 DBZ-965 Test of flushed LSN 2018-11-13 09:15:09 +01:00
Gunnar Morling
5a9ca6eb31 DBZ-965 Some clean-up 2018-11-13 09:15:09 +01:00
Jiri Pechanec
675f5b6479 DBZ-965 Renames for better clarification 2018-11-13 09:15:09 +01:00