Jiri Pechanec
b546dc886e
DBZ-1362 Fix Snapshot/streaming switch race condition
2019-06-28 09:09:18 +02:00
Jiri Pechanec
e0b188f80a
DBZ-1362 Drop replication slot after test execution
2019-06-28 05:41:07 +02:00
Jiri Pechanec
046064d653
DBZ-1362 Make toast test compatible with PostgreSQL 11
2019-06-28 05:32:11 +02:00
Jiri Pechanec
412ce20638
DBZ-1362 Switch to Awaitility implementation
2019-06-28 05:19:01 +02:00
Jiri Pechanec
5a5d969e99
DBZ-1362 Wait for real LSN change during test
2019-06-28 05:19:01 +02:00
Jenkins user
e95d656bca
[maven-release-plugin] prepare for next development iteration
2019-06-27 07:36:13 +00:00
Jenkins user
d95fd87311
[maven-release-plugin] prepare release v0.10.0.Beta2
2019-06-27 07:36:12 +00:00
Guillaume Rosauro
dc81d3c8ac
DBZ-1352 Removing the unused topic selection strategy configuration from Postgres connector
2019-06-26 16:24:38 +02:00
Jiri Pechanec
355c8a94ac
DBZ-1347 JavaDoc update
2019-06-25 14:52:58 +02:00
Gunnar Morling
62fc03a44d
DBZ-1347 Commenting
2019-06-25 14:52:58 +02:00
Jiri Pechanec
e72d7edd2f
DBZ-1347 Switch to non-blocking stream read
2019-06-25 14:52:58 +02:00
Gunnar Morling
e05ef934a0
DBZ-1345 Adding @FixFor
2019-06-25 12:44:00 +02:00
Chris Cranford
d518fa59fe
DBZ-1345 Should not snapshot materialized views
2019-06-25 12:44:00 +02:00
Jiri Pechanec
2dfdf5cb12
DBZ-1342 Partitioned table is included in snapshot
2019-06-21 12:26:40 +02:00
Gunnar Morling
21176ff70f
Adding link to replication API
2019-06-21 11:14:35 +02:00
Gunnar Morling
deaa5b7f4d
DBZ-1337 Adding support for HSTORE arrays
2019-06-19 17:01:13 +02:00
Jiri Pechanec
badae96e1d
DBZ-1335 Process unknown datatypes for snapshotted arrays
2019-06-18 16:17:49 +02:00
Jenkins user
5fdb0ca81b
[maven-release-plugin] prepare for next development iteration
2019-06-11 11:20:46 +00:00
Jenkins user
61729b9e00
[maven-release-plugin] prepare release v0.10.0.Beta1
2019-06-11 11:20:46 +00:00
Jiri Pechanec
6a3c87e2f1
DBZ-1323 Initialize commitTime for out-of-order chunk
2019-06-11 10:27:56 +02:00
Gunnar Morling
5d01f409f0
DBZ-1317 Separating update and query methods for MongoDB's SourceInfo;
...
* Avoids access to collection id after events which don't have one
* Pulling up struct() and schema() methods to common base class
2019-06-11 10:07:36 +02:00
Chris Cranford
0aadaa10dc
DBZ-1242 Fixed typo and added negative test checks
2019-06-05 13:31:25 -04:00
Chris Cranford
b3ca33287a
DBZ-1242 Issue warning when filters produce an empty schema
2019-06-05 13:31:25 -04:00
Jenkins user
056730d06a
[maven-release-plugin] prepare for next development iteration
2019-06-03 10:24:08 +00:00
Jenkins user
7aeaec236e
[maven-release-plugin] prepare release v0.10.0.Alpha2
2019-06-03 10:24:08 +00:00
Gunnar Morling
246c1be305
DBZ-1283 Enforcing presence of database.server.name globally
2019-06-03 10:58:14 +02:00
Chris Cranford
5f0b44b4ec
DBZ-1283 Only validate history topic name for affected connectors
2019-06-03 10:58:14 +02:00
Chris Cranford
b4736a7e03
DBZ-1283 Only validate history topic name for affected connectors
2019-06-03 10:58:14 +02:00
Gunnar Morling
73590173b5
DBZ-1310 Removing methods only needed for testing
2019-05-31 18:09:41 -04:00
Jiri Pechanec
4b64c8e447
DBZ-1295 3state snapshot for MySQL
2019-05-31 10:56:05 +02:00
Jiri Pechanec
348b2415d5
DBZ-1295 3state snapshot for PostgreSQL
2019-05-31 10:56:05 +02:00
Jiri Pechanec
946019982e
DBZ-1303 Assembly without database drivers
2019-05-30 14:32:47 +02:00
Gunnar Morling
7759333fe3
DBZ-1092 Using ColumnNameFilter instead of Predicate<Column>
2019-05-29 15:49:49 +02:00
Jenkins user
ac5c76e431
[maven-release-plugin] prepare for next development iteration
2019-05-28 14:38:32 +00:00
Jenkins user
35f59ba460
[maven-release-plugin] prepare release v0.10.0.Alpha1
2019-05-28 14:38:32 +00:00
Gunnar Morling
8ecfbee77d
DBZ-1235 Adding TODO marker
2019-05-28 09:35:15 +02:00
Gunnar Morling
b8eede2f41
DBZ-1235 Misc. clean-up;
...
* Putting collection name after replica set name for MongoDB
* Removing redundant method for converting schema to string
* Removing redundant keys from PG SourceInfo
2019-05-28 09:35:15 +02:00
Jiri Pechanec
526e780513
DBZ-1235 Misc fixes
2019-05-28 09:35:15 +02:00
Gunnar Morling
eef37cb567
DBZ-1235 Misc adjustments;
...
* Removing superfluous parameter from getSourceInfoStructMaker()
* JavaDoc
* Renaming field to "source.struct.version"
2019-05-28 09:35:15 +02:00
Gunnar Morling
8f28f323e7
DBZ-1235 Using Instant to represent timestamps internally;
...
Only "rendered" as epoch milli-seconds at the system boundaries.
2019-05-28 09:35:15 +02:00
Jiri Pechanec
6f6c8ddf06
DBZ-1235 Refactored to unify the code
2019-05-28 09:35:15 +02:00
Jiri Pechanec
c1bf1752fe
DBZ-1235 Pluggable source maker for MongoDB
2019-05-28 09:35:15 +02:00
Jiri Pechanec
df16ed84cd
DBZ-1235 Pluggable source maker for PostgreSQL
2019-05-28 09:35:15 +02:00
Jiri Pechanec
f5757dbd9f
DBZ-1235 Extract serverName to common source
2019-05-28 09:35:15 +02:00
Gunnar Morling
4ed8b4636c
DBZ-1224 Completing implementation;
...
* Further unification of override handling;
* Undoing change related to SnapshotStatementFactory as it seems not needed
* Adding test
2019-05-27 08:44:42 +02:00
mokkara
b3de0250a9
DBZ-1224 Snapshot SELECT support for SQL Server
2019-05-27 08:44:42 +02:00
Gunnar Morling
a0e30b7c90
DBZ-1289 Making outbox router options more consistent
2019-05-24 13:18:55 +02:00
Chris Cranford
e2c74111ad
DBZ-1234 Removed deprecated configuration options
2019-05-22 10:19:05 +02:00
Gunnar Morling
4751b266f3
DBZ-1164 Misc. improvements;
...
* Moving stringToDuration() for the sake of re-use and testability
* Making pattern and durations constants
2019-05-21 09:38:23 +02:00
GraySmog
9b52ac86e8
DBZ-1164 Supporting TIME columns with value of "24:00:00"
2019-05-21 09:38:09 +02:00
Jenkins user
7748be6056
[maven-release-plugin] prepare for next development iteration
2019-05-02 16:04:56 +00:00
Jenkins user
5af083b86e
[maven-release-plugin] prepare release v0.9.5.Final
2019-05-02 16:04:55 +00:00
Gunnar Morling
d12926c404
DBZ-1214 Applying default status update interval
2019-05-02 11:30:16 +02:00
Gunnar Morling
6960c880ca
DBZ-1214 Misc. clean-up:
...
* Using parker instead of Thread#sleep()
* Using flag for executor cancelation
* Using unique context name in thread name
2019-05-02 11:30:16 +02:00
Addison Higham
e1570dd330
[DBZ-1214] Ensure PG connection keep alive
...
This adds a small change to the postgres connector which
works around a problem when large databases take more
than a minute to refresh the schema
This works by creating a simple background thread that ensures we send
statusUpdate messages periodically
I considered if we could refresh the schema before we create the
replication connection but I believe we choose our offset in the log
and then get the schema in order to ensure we don't get out of sync with
any potential schema changes
This also makes some minor refactors to use a Duration instead of an
integer for the statusUpdateInterval
2019-05-02 11:30:16 +02:00
Jiri Pechanec
3b78e0bebb
DBZ-1245 Fix empty heartbeat message test
2019-05-02 09:16:21 +02:00
Jiri Pechanec
b53f456912
DBZ-1245 Heartbeat only when offsets ready
2019-05-02 08:22:28 +02:00
Gunnar Morling
2f7cab6eb3
DBZ-1260 Removing some dead code
2019-05-02 07:28:23 +02:00
Chris Cranford
dc61e52bc5
DBZ-1260 Fix timetz microsecond precision support
2019-05-02 07:28:23 +02:00
Chris Cranford
7602136cf6
DBZ-1236 Removed millisecond truncation.
2019-05-01 14:16:17 +02:00
Chris Cranford
16840d7d59
DBZ-1236 Handle timez precision consistently.
2019-05-01 14:16:17 +02:00
Jiri Pechanec
3ff81348b8
DBZ-1236 Support full timestamptz precision
2019-05-01 14:16:17 +02:00
Gunnar Morling
a84230447e
DBZ-1247 Making defaultSnapshotFetchSize() abstract
2019-04-30 09:57:11 +02:00
jchipmunk
72b06366fe
Removing extra initialisation of "config" variable in PG config
2019-04-30 09:57:11 +02:00
jchipmunk
9fe033ce24
DBZ-1247 Specifying default snapshot fetch size for each connector config
2019-04-30 09:57:11 +02:00
jchipmunk
a5cc83c0d1
DBZ-1247 Adding property to specify snapshot fetch size
2019-04-30 09:57:11 +02:00
jorkzijlstra
d0e2a72cd1
DBZ-1241 Allowing null values again for commitTime
2019-04-30 09:45:10 +02:00
shubham
f55bec86d3
Added support for all range types
2019-04-24 07:40:03 +02:00
krizhan
70c564b4c2
DBZ-212 Moved SERVER_NAME to RelationalDatabaseConnectorConfig
2019-04-23 12:29:44 +02:00
krizhan
38f3856c31
DBZ-212 Remove PostgreSQL getLogicalName function
2019-04-23 12:29:44 +02:00
Dennis Campagna
173e8f5854
DBZ-1034 Support range column types on Postgres
...
Signed-off-by: sciamp <denniscampagna@gmail.com>
DBZ-1034 Support range column types on Postgres
Signed-off-by: sciamp <denniscampagna@gmail.com>
2019-04-12 09:44:17 +02:00
GraySmog
726a2afcd6
Add comments stating the reason for skipping tests
2019-04-12 09:27:45 +02:00
GraySmog
08ca02f8f4
DBZ-1193 Add support for network type MACADDR and MACADDR8 in PostgreSQL connnector
2019-04-12 09:27:45 +02:00
Jenkins user
cefc5ccfb9
[maven-release-plugin] prepare for next development iteration
2019-04-11 12:07:16 +00:00
Jenkins user
fc3026adc3
[maven-release-plugin] prepare release v0.9.4.Final
2019-04-11 12:07:15 +00:00
Gunnar Morling
2250c47b7f
DBZ-1163 Minor improvements
2019-04-10 10:53:22 +02:00
preethi
02873e8169
DBZ-1163 Enable postgres snapshot for tables without primary key
2019-04-10 10:48:11 +02:00
ShubhamRwt
762d908eef
DBZ-1193 Added support for cidr address type
2019-04-08 15:00:03 +02:00
jchipmunk
7c0ae3ee20
DBZ-1212 SLF4J usage issues
2019-04-04 21:32:12 +02:00
Jiri Pechanec
ff52f3713e
DBZ-1144 Test for geometry in public schema
2019-04-02 15:57:57 +02:00
Alexander Kovryga
4616df189e
DBZ-1189 Add support for inet data type.
2019-03-28 09:57:32 +01:00
ShubhamRwt
540a951211
DBZ- 362 Adding WhitespaceAfter check to Checkstyle
2019-03-28 09:24:11 +01:00
Jenkins user
8b0edd1871
[maven-release-plugin] prepare for next development iteration
2019-03-25 12:16:31 +00:00
Jenkins user
56f98a48b2
[maven-release-plugin] prepare release v0.9.3.Final
2019-03-25 12:16:30 +00:00
Renato Mefi
6300f2972f
DBZ-1169 Integration tests for Outbox EventRouter
...
Set the Snapshot Mode to NEVER provides more consistent tests
2019-03-25 11:27:44 +01:00
Renato Mefi
8b689d7f03
DBZ-1169 Implement RegexRouter based on config
2019-03-25 11:27:44 +01:00
Renato Mefi
f0f13ed4a4
DBZ-1169 Implement basic behavior
2019-03-25 11:27:44 +01:00
Renato Mefi
2ee1c98298
DBZ-1169 Setup EventRouter on Postgres Test IT
2019-03-25 11:27:44 +01:00
shubham
8cca21e969
DBZ-1039 Make all ifs with braces
2019-03-25 10:23:14 +01:00
Addison Higham
8e21139ca3
DBZ-1082 Add new custom recovery mode, more metadata
...
This commit does a few things:
- Refactors snapshot modes to be encapsulated by an interface and
to only use that interface in determining when to snapshot and in
determing the type of the `RecordProducer` interface to instantiate
- Refactors the configuration of existing snapshot modes to tie the
existing snapshot modes to their aligned implementation
- Adds a new snapshot.mode, custom, and a new configuration option to
specify a custom implementation that will be loaded by the class loader
- Changes the visibility of some classes to allow for custom snapshot
modes to get enough context to make an informed choice
- Adds some metadata about slots (the catalog_xmin) to give a full idea
of the state of slots which can be useful in implementing snapshot
modes (which is also configurable, as it can add some overhead)
Together, these changes allow for a much broader flexibility got end
users to implement a snapshot mode that can do more advanced snapshots,
such as partial recovery or for partial snapshots for tables where not
all records are needed.
This could also be seen as superseeding the
`snapshot.select.statement.overrides` to allow for users to dynamically
build queries based on the state of the slot and the offsets consumed.
2019-03-24 11:56:40 +01:00
Jiri Pechanec
b11f1b8d05
DBZ-1161 Start snapshot in async thread
2019-03-21 14:32:47 +01:00
trolison
2001b178d7
- Fixed bad formatting
...
- Added trace for when empty messages are processed
DBZ-1181
2019-03-19 10:46:36 -07:00
trolison
df0e789601
- Add support for processing WAL2JSON events that have no changes
...
DBZ-1181
2019-03-18 20:32:03 -07:00
Gunnar Morling
2ea87727cf
DBZ-1174 Using Instant instead of long for representing timestamps
2019-03-12 17:19:13 +01:00
Ashhar Hasan
0b07afd23e
DBZ-1174 Convert commitTime from nanos to micros for wal2json
...
- wal2json sends the txn commitTime using a function from PostgreSQL's C
library. The value that Debezium recevies is in nanoseconds.
- decoderbufs sends the txn commitTime in microseconds.
- RecordsSnapshotProducer updates SourceInfo.ts_usec by converting
System.currentTimeMillis() to microseconds.
- RecordsSnapshotProducer updates the SourceInfo's ts_usec field using
message.getCommitTime().
This means that when using wal2json, the value of SourceInfo.ts_usec
is in microseconds since epoch during snapshot but is in nanoseconds
during streaming. To fix this, we changed
Wal2JsonReplicationMessage.getCommitTime() to return in microseconds.
2019-03-12 17:19:13 +01:00
Josh Stanfield
e4b7b90637
DBZ-1162 fix to ensure hstore snapshots behave the same as streams
2019-03-04 13:39:50 +01:00
Jenkins user
50a7c568fa
[maven-release-plugin] prepare for next development iteration
2019-02-23 09:53:58 +00:00
Jenkins user
28f3839804
[maven-release-plugin] prepare release v0.9.2.Final
2019-02-23 09:53:58 +00:00
Jiri Pechanec
78bfda09a6
DBZ-1146 Test full column diff
2019-02-22 14:16:03 +01:00
Gunnar Morling
5043d41b7b
DBZ-1146 Expanding test
2019-02-22 14:16:03 +01:00
Jiri Pechanec
4dec844fa2
DBZ-1146 Fix table created in streaming test
2019-02-22 14:16:03 +01:00
Jiri Pechanec
fa1e632a90
DBZ-1146 Handle toastable columns for snapshotted tables
2019-02-22 14:16:03 +01:00
Jiri Pechanec
89feae32e7
DBZ-1141 Test for the default values
2019-02-21 15:02:51 +01:00
Gunnar Morling
6787d17d54
DBZ-1141 Misc. clean-up;
...
* Removing redundant check for date mapping type
* Always using String as fallback value for temporal values where needed
* Pulling fallback temporal values up to JdbcValueConverters
2019-02-21 15:02:51 +01:00
Jiri Pechanec
b9d326711d
DBZ-1141 Incorrect money scale
2019-02-21 15:02:51 +01:00
Jiri Pechanec
992c31fa1f
DBZ-1141 Add non-date types
2019-02-21 15:02:51 +01:00
Jiri Pechanec
41ba7128e3
DBZ-1141 Correct default falback values for connect date types
2019-02-21 15:02:51 +01:00
Keith Barber
95b2290df2
DBZ-1149 Ensure consistent Java types for fallback values when converting
2019-02-21 15:02:51 +01:00
taylor-rolison
17c6a7a82d
Config option to pass optional logical decoder stream params ( #783 )
...
DBZ-1130 Added config param slot.stream.params to the Postgres connector to allow passing optional params to logical decoder on stream start
2019-02-21 14:06:51 +01:00
Jenkins user
b96e5fee0d
[maven-release-plugin] prepare for next development iteration
2019-02-13 11:13:44 +00:00
Jenkins user
3de2d04bcd
[maven-release-plugin] prepare release v0.9.1.Final
2019-02-13 11:13:44 +00:00
Jiri Pechanec
9049078607
DBZ-997 Handle de-sync of message and JDBC PK info
2019-02-13 10:13:12 +01:00
Gunnar Morling
2a486ecfff
DBZ-1019 Removing legacy methods from Metronome
2019-02-11 18:11:34 +01:00
Ivan Lorenz
721e8112cd
DBZ-1126 Localize era display value to default locale
2019-02-06 15:34:30 +01:00
Gunnar Morling
7076deba5a
DBZ-1118 Adding JUnit rule to skip tests depending on Postgres version
2019-02-04 14:43:19 +01:00
Kevin Pullin
c414a8b942
Add test for DBZ-1118
2019-02-04 14:43:19 +01:00
Kevin Pullin
1330df8908
Postgres snapshot - On null records, clear prior value from currentRecord
2019-02-04 14:43:19 +01:00
Gunnar Morling
41b7fbe78e
Adding note on testing Postgres connector with external DB
2019-01-30 09:25:12 +01:00
Gunnar Morling
cc0df61665
DBZ-1111 Allowing to use drop-slot-on-close with PG 10 or newer
2019-01-29 14:44:10 +01:00
Gunnar Morling
e84c6e2238
DBZ-962 Avoiding dependency to JAXB
2019-01-29 10:28:48 +01:00
Jenkins user
c5b7d21d3e
[maven-release-plugin] prepare for next development iteration
2019-01-28 11:07:04 +00:00
Jenkins user
f1ae79ff73
[maven-release-plugin] prepare release v0.9.0.CR1
2019-01-28 11:07:04 +00:00
Gunnar Morling
ccff79e07d
DBZ-1105 Removing TableImpl#columnNames field
2019-01-28 10:23:56 +01:00
Gunnar Morling
9dc2c525c9
DBZ-1083 Passing "include-unchanged-toast" option only for wal2json
2019-01-25 11:04:43 +01:00
Addison Higham
26f836e0f6
DBZ-1080 new API for EmbeddedEngine for batch/async
...
This introduces a new API to the EmbeddedEngine, the ChangeConsumer,
which gives the user a more flexible option for consuming changes by
exposing groups of records as well as the ability to control the
comitting of those records.
This remainds completely backwards compatible with the old API as the
ChangeConsumer wraps the existing Consumer interface with a default
implementation.
2019-01-18 14:06:50 +01:00
Jiri Pechanec
1cf257478b
DBZ-781 Enable unstable tests
2019-01-17 08:59:42 +01:00
Tony Rizko
a921613877
Fix typo in PostgresConnection error message
2019-01-17 08:58:12 +01:00
Jiri Pechanec
ceb6c62721
DBZ-1064 Remove lazy fallback values
2019-01-16 13:01:36 +01:00
Jiri Pechanec
536e517c08
DBZ-1064 Geometry default value changes; fixes for default suppliers
2019-01-16 13:01:36 +01:00
Gunnar Morling
532c17f8aa
DBZ-1064 Using dedicated functional interface instead of Consumer;
...
* Commenting
* Moving type from "util" to "jdbc"
2019-01-16 13:01:36 +01:00
Jiri Pechanec
bff7eea734
DBZ-1064 Use convert value template
2019-01-16 13:01:36 +01:00
Jiri Pechanec
920e0cbd3f
DBZ-1064 Set default value only for non-nullable fields
2019-01-16 13:01:36 +01:00
Jiri Pechanec
484a8977bc
DBZ-1083 Passing "include-unchanged-toast" option only if supported;
...
The "include-unchanged-toast" option was removed in recent wal2json versions, without a transition phase. So the connector tries not to connect first giving the option, and if that fails, without it.
2019-01-16 09:37:55 +01:00
Gunnar Morling
af01d10886
DBZ-1081 JavaDoc; adding Addison Higham to COPYRIGHT.txt
2019-01-15 18:09:08 +01:00
Addison Higham
889b307b9d
DBZ-1081 Fallback to restart_lsn in pg 9.5
...
In pg 9.5, confirmed_flush_lsn is not availiable. However, there is
restart_lsn, which should be safe to use with a downside of producing
some extra duplicate records.
This page: https://paquier.xyz/postgresql-2/postgres-9-6-feature-highlight-replication-slot-improvements/
indicates as such and this allows for DBZ to support PG 9.5
2019-01-15 18:05:55 +01:00
Gunnar Morling
de356896c0
DBZ-1029 Adding one more test
2019-01-15 17:41:42 +01:00
Ilia Bogdanov
88521371cc
DBZ-1029 insertion test update, code cleanup after review
2019-01-14 12:03:19 +03:00
Ilia Bogdanov
cee9bd9a5d
DBZ-1029 handling messages from tables with replica identity full in postgres connector.
2019-01-14 12:03:19 +03:00
Amit Sela
b0b7d942a3
DBZ-1073 Adding scale as schema parameter, if present
2019-01-11 15:40:30 +01:00
Grzegorz Kołakowski
91c72738c0
DBZ-1067 Add column blacklist field to RelationalDatabaseConnectorConfig
2019-01-09 18:13:26 +01:00
Jenkins user
5275f73424
[maven-release-plugin] prepare for next development iteration
2018-12-19 13:06:12 +00:00
Jenkins user
b6569c18ae
[maven-release-plugin] prepare release v0.9.0.Beta2
2018-12-19 13:06:12 +00:00
Gunnar Morling
92ae01ccdf
DBZ-996 Adding column name to log message
2018-12-18 17:53:42 +01:00
Jiri Pechanec
01eee09f90
DBZ-996 Tolerate race condition in PK updates
2018-12-18 17:52:28 +01:00
Jiri Pechanec
edec1c3090
DBZ-978 Added last event and captured tables metrics
2018-12-13 20:52:11 +01:00
Gunnar Morling
5542c0de29
DBZ-966 Making sure Postgres connector can be stopped while awaiting initial snapshot delay;
...
* Also skipping snapshotting when requested to stop during initial delay
* Showing remaining delay time in logs
* Adding missing config
2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
e2942ab72a
DBZ-966 Replace sleep with polling loop
2018-12-13 14:29:54 +01:00
Grzegorz Kołakowski
35a5acfd8b
DBZ-966 Enable to delay initial snapshot in PostgreSQL connector
2018-12-13 14:29:54 +01:00
Gunnar Morling
0833b23feb
DBZ-842 Expanding test to cover TOASTed NOT NULL column, too
2018-12-11 18:42:51 +01:00
Jiri Pechanec
94c0a42286
DBZ-842 Do not send TOASTed values from wal2json
2018-12-11 17:10:58 +01:00
Jiri Pechanec
2b2ff0754d
DBZ-960 Run SnapshotReader in a single transaction
2018-12-11 14:15:53 +01:00
Gunnar Morling
470cd1eaba
DBZ-1024 Doing some clean-up
2018-12-07 15:39:59 +01:00
Jiri Pechanec
ea5c8512b7
DBZ-1024 Return test assert
2018-12-07 15:39:59 +01:00
Jiri Pechanec
0b78f0c31a
DBZ-1024 Emit correct last_snapshot_record in source
2018-12-07 15:39:59 +01:00
Jiri Pechanec
9c225c63eb
DBZ-1021 Ignore events for dropped table
2018-12-06 18:54:50 +01:00
Gunnar Morling
101bd8ef6d
DBZ-1000 Making Checkstyle happy
2018-12-03 15:00:43 +01:00
Jiri Pechanec
7a3cbd6cb6
DBZ-1000 Detect nullability change in schema
2018-12-03 14:54:36 +01:00
Jiri Pechanec
68766e9355
DBZ-1009 Support for unknown types during snapshot
2018-11-30 11:16:36 +01:00
Jenkins user
db1d3a7fb8
[maven-release-plugin] prepare for next development iteration
2018-11-20 16:15:14 +00:00
Jenkins user
f83db82cea
[maven-release-plugin] prepare release v0.9.0.Beta1
2018-11-20 16:15:14 +00:00
Jiri Pechanec
0611d4975f
DBZ-965 Reduce the test time
2018-11-13 09:16:30 +01:00
Jiri Pechanec
a4abd2a336
DBZ-965 Test of flushed LSN
2018-11-13 09:15:09 +01:00
Gunnar Morling
5a9ca6eb31
DBZ-965 Some clean-up
2018-11-13 09:15:09 +01:00
Jiri Pechanec
675f5b6479
DBZ-965 Renames for better clarification
2018-11-13 09:15:09 +01:00
Gunnar Morling
b455b7ccaf
DBZ-970 Making LogicalDecoder#getPostgresPluginName() abstract
2018-11-13 09:15:09 +01:00
Jiri Pechanec
390965f4f9
DBZ-965 Track latest LSN per each change
2018-11-13 09:15:09 +01:00
Jiri Pechanec
b1707c395d
DBZ-970 Correct plging name for wal2json_streaming decoder
2018-11-13 09:15:09 +01:00
Jiri Pechanec
83137d0a41
DBZ-965 Force heartbeat after end of each TX
2018-11-13 09:15:09 +01:00
Gunnar Morling
efbd46b7f6
DBZ-953 Having PG connector use decimal mode option from RelationalDatabaseConnectorConfig
2018-11-08 09:23:28 +01:00
Jiri Pechanec
3a8293a4ba
DBZ-937 Disable auto-handling of unknown types in test
2018-10-11 09:55:07 +02:00
Olavi Mustanoja
0c986b4a78
DBZ-937 add wal2json replication message handling for CITEXT
2018-10-09 08:46:55 +02:00
Jenkins user
d952f5dfb0
[maven-release-plugin] prepare for next development iteration
2018-10-04 11:59:14 +00:00
Jenkins user
ff9b70278b
[maven-release-plugin] prepare release v0.9.0.Alpha2
2018-10-04 11:59:13 +00:00
Gunnar Morling
1f25536e2b
DBZ-934 Using API for creating replication slot if possible
2018-10-04 10:12:48 +02:00
PSanetra
dd3859d8bc
DBZ-934 Use temporary replication slots if slot.drop_on_stop is enabled and Postgres >= 10 is used
2018-10-04 10:12:48 +02:00
jchipmunk
3ec0b3be81
DBZ-918 Moving connector identifier from SourceInfo to Module class
2018-10-04 09:00:13 +02:00
jchipmunk
05ba99bb0a
DBZ-918 Adding Debezium connector field to source info
...
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
2018-10-04 09:00:13 +02:00
Gunnar Morling
984081dd1a
DBZ-926 Adding flushed LSN to log message
2018-10-03 09:47:38 +02:00
Gunnar Morling
08ec4f48d0
DBZ-911 Style adjustments; wording fix
2018-10-01 12:49:56 +02:00
Ian Axelrod
427249887b
DBZ-911 Simplifying schemaChanged() shortcircuit logic
...
Break up initial conditional and comment missing column portion to
clarify the role of skipRefreshSchemaOnMissingToastableData.
2018-10-01 12:49:48 +02:00
Ian Axelrod
9f083d0f1b
DBZ-911 Simplifying getToastableColumnsForTableId() impl.
...
No need to wrap ArrayList instance in unmodifiableList. Can just use
Collections.emptyList.
2018-10-01 12:49:36 +02:00
Ian Axelrod
ad433c5747
DBZ-911 Don't refresh schema on absent TOAST cols
...
Improves PostgreSQL RecordsStreamProducer performance for processing up
dates to a table with TOASTable columns, where those updates do not
affect said columns. Prior to this fix, these updates would trigger a
refresh of the in-memory table schema. In the worst case, this means a
query for every update. This puts significant load on the database
server and adds tens of milliseconds to the processing of each update
record.
The fix requires a new configuration option, called
'schema.refresh.mode'. This option has values 'columns_diff' (the
default) and 'columns_diff_exclude_unchanged_toast'. 'columns_diff'
maintains the pre-fix behavior. 'columns_diff_exclude_unchanged_toast'
activates the fix.
The fix must be toggleable because it decreases the consistency guarantees
Debezium provides, since in-memory table schemas may not stay
synchronized with their remote counterparts. With type metadata included
in the replication message, inconsistencies are limited to the unchanged
toast columns.
2018-10-01 11:39:32 +02:00
Gunnar Morling
6f7bdbe5e1
DBZ-898 Using 'Json' semantic type when encoding HSTORE column values as string
2018-09-25 12:22:20 +02:00
Gunnar Morling
7150e387c7
DBZ-898 Misc. clean-up:
...
* Using database encoding for string conversion
* Not making hstore schemas optional by default
* Using Jackson instead of GSon for JSON serialization
* Removing superfluous method and log messages
* Adjusting to naming and style conventions
2018-09-24 13:36:28 +02:00
SyedMuhammadSufyian
623cc98cb0
Change log levels
2018-09-22 20:15:07 +02:00
SyedMuhammadSufyian
346f2b3761
Handling HStore type , hstore.handling.mode:map|json , default is map.Added support for protobuff
2018-09-22 20:07:13 +02:00
SyedMuhammadSufyian
38f4a0e65b
HStore Data type support , configuration options are map|json , default is map
2018-09-22 05:54:45 +02:00
Gunnar Morling
f9ede861ba
DBZ-912 Upgrading to Postgres driver 42.2.5
2018-09-19 15:26:44 +02:00
Gunnar Morling
39a47fabfa
DBZ-899 Using custom statement to obtain SQL type info in order to avoid N+1 issues with PgDatabaseMetaData#getTypeInfo()
2018-09-19 09:51:38 +02:00
Gunnar Morling
3331d4c1a9
DBZ-899 Removing unused field;
...
Also inlining method only used in one place.
2018-09-19 09:51:38 +02:00
Jiri Pechanec
59fbcca725
DBZ-907 Log refreshing schema due to different column count
2018-09-12 19:07:01 +02:00
Jiri Pechanec
e20c183d5d
DBZ-895 Make tcpKeepalive enabled by default
2018-09-11 09:34:21 +02:00
Gunnar Morling
e1bbbd7992
DBZ-865 DBZ-878 Replacing invalid topic name characters with "_";
2018-09-05 12:56:53 +02:00
Gunnar Morling
bd692b8b84
DBZ-862 Extracting values for controlling retry attempts into constants
2018-09-03 14:50:54 +02:00
Jiri Pechanec
c6e20d4967
DBZ-862 Fix test interdependency problem
2018-09-03 14:40:57 +02:00
Jiri Pechanec
1ff53e35ec
DBZ-862 Fix Travis timing issue
2018-09-03 14:40:57 +02:00
Jiri Pechanec
4b9141e1fd
DBZ-862 Handle concurrent transaction blocking taking snapshot
2018-09-03 14:40:57 +02:00
Jiri Pechanec
e65fb26ae9
DBZ-887 Close connection even for non-started producer
2018-08-31 13:02:23 +02:00
Gunnar Morling
ae879cf475
DBZ-859 Adjustments after rebase; typo fix
2018-08-23 12:00:44 +02:00
Jiri Pechanec
cd6dc09e72
DBZ-859 Heartbeat is sent after PostgreSQL snapshot
2018-08-23 11:44:53 +02:00
oripwk
7d958a1928
DBZ-866 PR fixes
2018-08-22 17:09:26 +02:00
oripwk
d82372a6b2
DBZ-866 Add schema, database and table to Postgres connector
2018-08-22 17:09:26 +02:00
Jiri Pechanec
d4e64a61cb
DBZ-705 Allow to configure postgres config file location
2018-08-13 20:21:41 +02:00
Jenkins user
e00dad127f
[maven-release-plugin] prepare for next development iteration
2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700
[maven-release-plugin] prepare release v0.9.0.Alpha1
2018-07-26 08:00:12 +00:00
Gunnar Morling
3061c1dc0b
DBZ-815 Passing offset to Heartbeat#heartbeat();
...
That's simpler to grasp than the approach of passing a supplier lambda to the constructor.
Also it allows to pass on the offset via local variables instead of instance fields in some cases.
2018-07-25 14:49:52 +02:00
Gunnar Morling
8e1450bc71
DBZ-800 Adding test for receiving heartbeat events with Postgres
2018-07-20 13:37:54 +02:00
Gunnar Morling
4abc471bcb
DBZ-800 Emitting heartbeat events in Postgres streaming loop
2018-07-20 13:37:54 +02:00
Gunnar Morling
20db9299c5
DBZ-627 Unifying TopicSelector implementations
2018-07-18 12:36:07 +02:00
Gunnar Morling
1523f230ca
DBZ-793 Using TableFilter consistently;
...
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8
[maven-release-plugin] prepare for next development iteration
2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f
[maven-release-plugin] prepare release v0.8.0.Final
2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d
[maven-release-plugin] prepare for next development iteration
2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0
[maven-release-plugin] prepare release v0.8.0.CR1
2018-07-04 07:07:44 +00:00
Jiri Pechanec
c2f2f686f0
DBZ-644 Disable test making problems in Travis CI
2018-07-03 07:03:59 +02:00
Jiri Pechanec
646a66672c
DBZ-644 Close unused replication connection
2018-07-03 07:03:59 +02:00
Gunnar Morling
dcd6b01134
DBZ-644 Simplifying RelationalDatabaseSchema constructor;
...
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
2018-07-03 07:03:59 +02:00
Gunnar Morling
b0980b994c
DBZ-644 Adding test for Postgres
2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a
DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema
2018-07-03 07:03:59 +02:00
Jiri Pechanec
0182eecd06
DBZ-778 Make the tests resilient against race condition
2018-07-02 11:52:23 +02:00
Gunnar Morling
a7be3847e3
DBZ-773 Removing some dead code and unused methods
2018-06-29 09:27:57 +02:00
Gunnar Morling
c4c8cbc3ab
DBZ-773 Moving management of Tables to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab
DBZ-773 Moving management of table schemas to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
0600ccf213
DBZ-751 Incorporating review remarks;
...
* Expanding test
* Extracting constant for parameter key in tests
2018-06-27 14:48:59 +02:00
Gunnar Morling
a292b05a96
DBZ-751 Propagating DECIMAL column precision to Avro schemas
2018-06-27 14:48:59 +02:00
Jiri Pechanec
ca6b1c9168
DBZ-743 Empty password for PostgreSQL and MySQL
2018-06-27 11:18:19 +02:00
Gunnar Morling
f9c1cfc134
DBZ-762 Adding support for CITEXT array columns
2018-06-26 11:40:19 +02:00
Gunnar Morling
daeeaa74cd
DBZ-762 Formatting
2018-06-26 11:40:19 +02:00
Olavi Mustanoja
c401a98166
DBZ-762 Added support for CITEXT column type
2018-06-26 11:40:19 +02:00
Gunnar Morling
bf7a5018ca
Setting POM version back to 0.8.0-SNAPSHOT
2018-06-22 12:21:03 +02:00
Jenkins user
db42e4657a
[maven-release-plugin] prepare for next development iteration
2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99
[maven-release-plugin] prepare release v0.8.0.Beta1
2018-06-21 14:07:45 +00:00
Gunnar Morling
987e94cfbb
DBZ-578 Removing unnecessary "server.zone.offset" option;
...
This reverts parts of the previous work done for DBZ-578.
TIMESTAMP columns (logical date + time without time zone)
are always interpreted using UTC again.
2018-06-20 15:58:19 +02:00
Gunnar Morling
b83f1493a2
DBZ-20 Avoiding Optional as method parameter
2018-06-20 13:05:37 +02:00
Gunnar Morling
dce1aa29f1
DBZ-20 Oracle precision supports negative numbers
2018-06-20 13:05:37 +02:00
Jiri Pechanec
89694ffbbf
DBZ-20 Support for year to month interval
2018-06-20 13:05:37 +02:00
Gunnar Morling
d7e196a18e
DBZ-20 Initial import of Oracle connector based on XStream
2018-06-20 13:05:37 +02:00
Gunnar Morling
9eb4b90ec9
DBZ-191 Adding record validations
2018-06-15 20:23:17 +02:00
Jiri Pechanec
1e678d9959
DBZ-679 Make streaming decoding optional
2018-06-15 08:35:50 +02:00
Jiri Pechanec
7ee1f98f57
DBZ-679 Support decoding of out-of-order fragments
2018-06-15 08:35:50 +02:00
Jiri Pechanec
eadf65cd27
DBZ-578 Build failing on Jenkins CI
2018-06-13 14:43:29 +02:00
Gunnar Morling
a9d3d0c95d
DBZ-711 Using correct SSL mode names
2018-06-13 10:12:14 +02:00
Jiri Pechanec
29f8891b4f
DBZ-578 Process timestamps without timezones as in server timezone
2018-06-11 15:27:08 +02:00
Jiri Pechanec
4111ddbcaf
DBZ-709 Add test for timestamp with precision
2018-06-05 15:51:35 +02:00
Jiri Pechanec
176dcbbc0b
DBZ-709 PostgreSQL uses scale for time precision not length
2018-06-05 15:51:35 +02:00
Jiri Pechanec
525d073fea
DBZ-681 Time with precision for wal2json plug-in
2018-06-04 14:50:16 +02:00
Gunnar Morling
8d7806e71a
DBZ-693 Expanding test; updating connector option description
2018-05-28 12:35:47 +02:00
Jiri Pechanec
45bc7b1536
DBZ-693 On connect statements supported for PostgreSQL
2018-05-28 12:35:47 +02:00
Jiri Pechanec
e2603d392e
DBZ-699 Handle all exceptions
2018-05-24 13:50:16 +02:00
Jiri Pechanec
e533b75cbf
DBZ-699 Close JDBC connection when initialization fails
2018-05-24 09:49:42 +02:00
Jiri Pechanec
fa2691ffd1
DBZ-701 Support for arrays with large OID
2018-05-23 22:13:21 +02:00
Jiri Pechanec
f844054f50
DBZ-696 Support nanoseconds in timestamp data
2018-05-22 11:33:50 +02:00
Jiri Pechanec
09b28c0913
DBZ-694 Support milliseconds for pre-epoch dates
2018-05-22 11:22:06 +02:00
Jiri Pechanec
37cfe622fc
DBZ-697 Support for large type OIDs
2018-05-22 10:39:14 +02:00
Jiri Pechanec
2706858cf4
DBZ-676 Support testing against 8bit enc database
2018-04-06 18:26:45 +02:00
Gunnar Morling
9a29063ef4
DBZ-673 Converting tx id to long in one more place
2018-03-27 10:04:43 +02:00
listman
aa4bf5d367
DBZ-673 txId must be long in wal2jon decoder
2018-03-27 00:31:03 -05:00
Gunnar Morling
bdb1069b0d
DBZ-638 Logging message on TRACE level instead of DEBUG
2018-03-23 16:38:05 +01:00
Jiri Pechanec
363921a7b1
DBZ-638 Chunk processing operates on byte array
2018-03-23 14:41:59 +01:00
Jiri Pechanec
4051a4a439
DBZ-638 Improved variable naming
2018-03-23 14:41:59 +01:00
Jiri Pechanec
ce5196dc3e
DBZ-638 Keep non-streaming decoding as a fallback
2018-03-23 14:41:59 +01:00
Jiri Pechanec
24965dd076
DBZ-638 Process tx as stream not as batch
2018-03-23 14:41:59 +01:00
Jiri Pechanec
0743467102
DBZ-671 Date/time parsing covers more exotic formats
2018-03-23 08:55:24 +01:00
Jenkins user
f4e151b23a
[maven-release-plugin] prepare for next development iteration
2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332
[maven-release-plugin] prepare release v0.7.5
2018-03-20 08:14:19 +00:00
Jiri Pechanec
40a527b6ea
DBZ-592 Make snapshot/init-sync false instead of null
2018-03-19 13:19:34 +01:00
Jenkins user
daf27207be
[maven-release-plugin] prepare for next development iteration
2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928
[maven-release-plugin] prepare release v0.7.4
2018-03-07 08:31:07 +00:00
Gunnar Morling
1d5d6b6f3a
DBZ-635 Schema for geo-spatial array columns shouldn't be optional by default
2018-03-07 07:17:43 +01:00
Gunnar Morling
ff524695bc
DBZ-635 Schema for NUMERIC columns shouldn't be optional by default
2018-03-07 07:17:43 +01:00
Gunnar Morling
6412e9e06e
DBZ-605 Schema for BYTEA columns shouldn't be optional by default
2018-03-07 07:17:43 +01:00
Jiri Pechanec
327085164e
DBZ-611 Special values supported for double
2018-03-06 16:55:24 +01:00
Jiri Pechanec
424aefbbb4
DBZ-611 Decimal-> String supported for MySQL
2018-03-06 16:55:24 +01:00
Gunnar Morling
6e29b82358
DBZ-351 Avoiding call to new BigDecimal() for NaN
2018-02-27 16:36:21 +01:00
Gunnar Morling
5786d44200
DBZ-351 Avoiding call to getBigDecimal() for NaN
2018-02-27 16:36:21 +01:00
Gunnar Morling
d040ec4427
DBZ-351, DBZ-606 Misc. clean-up;
...
* Renaming DebeziumDecimal to SpecialValueDecimal
* Simplifying logic in PostgresValueConverter
2018-02-27 16:36:21 +01:00
Jiri Pechanec
2534696b21
DBZ-606 Exception thrown for special values when mode handling is not string
2018-02-27 16:36:21 +01:00
Jiri Pechanec
5827a7ac1b
DBZ-611 Decimals can be encoded as strings
2018-02-27 16:36:21 +01:00
Jiri Pechanec
c71707afc9
DBZ-351 wal2json usses unlimited precision too
2018-02-27 16:36:21 +01:00
Jiri Pechanec
0c0ac72568
DBZ-606 Type for handling special BigDecimals
2018-02-27 16:36:21 +01:00
Jiri Pechanec
ceaa7f8efe
DBZ-351 Numeric is passed with unlimited precision
2018-02-27 16:36:21 +01:00
Jiri Pechanec
4623c1d8c4
DBZ-606 Numeric NaN represented as null
2018-02-27 16:36:21 +01:00
Gunnar Morling
c900c7fc92
DBZ-623 Formatting and indentation
2018-02-22 21:16:00 +01:00
Jiri Pechanec
01e5f8da8c
DBZ-623 Extract parsing code to a separate method
2018-02-22 21:12:05 +01:00
Jiri Pechanec
80c74a9d7c
DBZ-623 Fix NPE for invlid restored LSNs
2018-02-22 21:12:05 +01:00
Gunnar Morling
eef1ad7c2a
DBZ-630 Refactoring around MongoDB task context;
...
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8
DBZ-630 Pulling up getClock() to ConnectorTaskContext
2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6
DBZ-630 Unifying common start-up logic across connectors
2018-02-21 12:04:41 +01:00
Jiri Pechanec
4e74e0a4ac
Revert default Postgres decoder plugin for testing
2018-02-20 10:29:06 +01:00
Jiri Pechanec
ad181493da
DBZ-626 Make caller method naming consistent
2018-02-20 09:26:47 +01:00
Gunnar Morling
7fe45db831
DBZ-626 Renaming AvroValidator to SchemaNameAdjuster;
...
Making apparent that this functionality doesn't solely validate but returns a new name actually
2018-02-20 09:26:47 +01:00
Gunnar Morling
cddd5fcc8a
DBZ-626 Passing schema validator as AvroValidator instance instead of Function
2018-02-20 09:26:47 +01:00
Jiri Pechanec
40a5422998
DBZ-605 Support for BYTEA datatype
2018-02-20 08:55:56 +01:00
Jiri Pechanec
6e00924ee9
DBZ-625 Fix parsing of float without decimals
2018-02-20 08:47:43 +01:00
Gunnar Morling
8369f10d52
DBZ-628 Adding MAX_QUEUE_SIZE, MAX_BATCH_SIZE and POLL_INTERVAL_MS to CommonConnectorConfig;
...
Also using these options in PG connector
2018-02-20 06:34:42 +01:00
Gunnar Morling
ab11f48843
DBZ-620 Creating envelope schemas only once per table type
2018-02-19 12:18:37 +01:00
Jenkins user
6d0cd88e12
[maven-release-plugin] prepare for next development iteration
2018-02-15 04:15:34 +00:00
Jenkins user
7d1e1a989e
[maven-release-plugin] prepare release v0.7.3
2018-02-15 04:15:34 +00:00
Gunnar Morling
cc64a7a616
DBZ-618 Removing unused method and field
2018-02-14 16:53:52 +01:00
Gunnar Morling
f54ed51897
DBZ-618 Removing superfluous method parameter
2018-02-14 16:43:52 +01:00