Chris Cranford
b4736a7e03
DBZ-1283 Only validate history topic name for affected connectors
2019-06-03 10:58:14 +02:00
Gunnar Morling
73590173b5
DBZ-1310 Removing methods only needed for testing
2019-05-31 18:09:41 -04:00
Jiri Pechanec
4b64c8e447
DBZ-1295 3state snapshot for MySQL
2019-05-31 10:56:05 +02:00
Jiri Pechanec
348b2415d5
DBZ-1295 3state snapshot for PostgreSQL
2019-05-31 10:56:05 +02:00
Jiri Pechanec
946019982e
DBZ-1303 Assembly without database drivers
2019-05-30 14:32:47 +02:00
Gunnar Morling
7759333fe3
DBZ-1092 Using ColumnNameFilter instead of Predicate<Column>
2019-05-29 15:49:49 +02:00
Jenkins user
ac5c76e431
[maven-release-plugin] prepare for next development iteration
2019-05-28 14:38:32 +00:00
Jenkins user
35f59ba460
[maven-release-plugin] prepare release v0.10.0.Alpha1
2019-05-28 14:38:32 +00:00
Gunnar Morling
8ecfbee77d
DBZ-1235 Adding TODO marker
2019-05-28 09:35:15 +02:00
Gunnar Morling
b8eede2f41
DBZ-1235 Misc. clean-up;
...
* Putting collection name after replica set name for MongoDB
* Removing redundant method for converting schema to string
* Removing redundant keys from PG SourceInfo
2019-05-28 09:35:15 +02:00
Jiri Pechanec
526e780513
DBZ-1235 Misc fixes
2019-05-28 09:35:15 +02:00
Gunnar Morling
eef37cb567
DBZ-1235 Misc adjustments;
...
* Removing superfluous parameter from getSourceInfoStructMaker()
* JavaDoc
* Renaming field to "source.struct.version"
2019-05-28 09:35:15 +02:00
Gunnar Morling
8f28f323e7
DBZ-1235 Using Instant to represent timestamps internally;
...
Only "rendered" as epoch milli-seconds at the system boundaries.
2019-05-28 09:35:15 +02:00
Jiri Pechanec
6f6c8ddf06
DBZ-1235 Refactored to unify the code
2019-05-28 09:35:15 +02:00
Jiri Pechanec
c1bf1752fe
DBZ-1235 Pluggable source maker for MongoDB
2019-05-28 09:35:15 +02:00
Jiri Pechanec
df16ed84cd
DBZ-1235 Pluggable source maker for PostgreSQL
2019-05-28 09:35:15 +02:00
Jiri Pechanec
f5757dbd9f
DBZ-1235 Extract serverName to common source
2019-05-28 09:35:15 +02:00
Gunnar Morling
4ed8b4636c
DBZ-1224 Completing implementation;
...
* Further unification of override handling;
* Undoing change related to SnapshotStatementFactory as it seems not needed
* Adding test
2019-05-27 08:44:42 +02:00
mokkara
b3de0250a9
DBZ-1224 Snapshot SELECT support for SQL Server
2019-05-27 08:44:42 +02:00
Gunnar Morling
a0e30b7c90
DBZ-1289 Making outbox router options more consistent
2019-05-24 13:18:55 +02:00
Chris Cranford
e2c74111ad
DBZ-1234 Removed deprecated configuration options
2019-05-22 10:19:05 +02:00
Gunnar Morling
4751b266f3
DBZ-1164 Misc. improvements;
...
* Moving stringToDuration() for the sake of re-use and testability
* Making pattern and durations constants
2019-05-21 09:38:23 +02:00
GraySmog
9b52ac86e8
DBZ-1164 Supporting TIME columns with value of "24:00:00"
2019-05-21 09:38:09 +02:00
Jenkins user
7748be6056
[maven-release-plugin] prepare for next development iteration
2019-05-02 16:04:56 +00:00
Jenkins user
5af083b86e
[maven-release-plugin] prepare release v0.9.5.Final
2019-05-02 16:04:55 +00:00
Gunnar Morling
d12926c404
DBZ-1214 Applying default status update interval
2019-05-02 11:30:16 +02:00
Gunnar Morling
6960c880ca
DBZ-1214 Misc. clean-up:
...
* Using parker instead of Thread#sleep()
* Using flag for executor cancelation
* Using unique context name in thread name
2019-05-02 11:30:16 +02:00
Addison Higham
e1570dd330
[DBZ-1214] Ensure PG connection keep alive
...
This adds a small change to the postgres connector which
works around a problem when large databases take more
than a minute to refresh the schema
This works by creating a simple background thread that ensures we send
statusUpdate messages periodically
I considered if we could refresh the schema before we create the
replication connection but I believe we choose our offset in the log
and then get the schema in order to ensure we don't get out of sync with
any potential schema changes
This also makes some minor refactors to use a Duration instead of an
integer for the statusUpdateInterval
2019-05-02 11:30:16 +02:00
Jiri Pechanec
3b78e0bebb
DBZ-1245 Fix empty heartbeat message test
2019-05-02 09:16:21 +02:00
Jiri Pechanec
b53f456912
DBZ-1245 Heartbeat only when offsets ready
2019-05-02 08:22:28 +02:00
Gunnar Morling
2f7cab6eb3
DBZ-1260 Removing some dead code
2019-05-02 07:28:23 +02:00
Chris Cranford
dc61e52bc5
DBZ-1260 Fix timetz microsecond precision support
2019-05-02 07:28:23 +02:00
Chris Cranford
7602136cf6
DBZ-1236 Removed millisecond truncation.
2019-05-01 14:16:17 +02:00
Chris Cranford
16840d7d59
DBZ-1236 Handle timez precision consistently.
2019-05-01 14:16:17 +02:00
Jiri Pechanec
3ff81348b8
DBZ-1236 Support full timestamptz precision
2019-05-01 14:16:17 +02:00
Gunnar Morling
a84230447e
DBZ-1247 Making defaultSnapshotFetchSize() abstract
2019-04-30 09:57:11 +02:00
jchipmunk
72b06366fe
Removing extra initialisation of "config" variable in PG config
2019-04-30 09:57:11 +02:00
jchipmunk
9fe033ce24
DBZ-1247 Specifying default snapshot fetch size for each connector config
2019-04-30 09:57:11 +02:00
jchipmunk
a5cc83c0d1
DBZ-1247 Adding property to specify snapshot fetch size
2019-04-30 09:57:11 +02:00
jorkzijlstra
d0e2a72cd1
DBZ-1241 Allowing null values again for commitTime
2019-04-30 09:45:10 +02:00
shubham
f55bec86d3
Added support for all range types
2019-04-24 07:40:03 +02:00
krizhan
70c564b4c2
DBZ-212 Moved SERVER_NAME to RelationalDatabaseConnectorConfig
2019-04-23 12:29:44 +02:00
krizhan
38f3856c31
DBZ-212 Remove PostgreSQL getLogicalName function
2019-04-23 12:29:44 +02:00
Dennis Campagna
173e8f5854
DBZ-1034 Support range column types on Postgres
...
Signed-off-by: sciamp <denniscampagna@gmail.com>
DBZ-1034 Support range column types on Postgres
Signed-off-by: sciamp <denniscampagna@gmail.com>
2019-04-12 09:44:17 +02:00
GraySmog
726a2afcd6
Add comments stating the reason for skipping tests
2019-04-12 09:27:45 +02:00
GraySmog
08ca02f8f4
DBZ-1193 Add support for network type MACADDR and MACADDR8 in PostgreSQL connnector
2019-04-12 09:27:45 +02:00
Jenkins user
cefc5ccfb9
[maven-release-plugin] prepare for next development iteration
2019-04-11 12:07:16 +00:00
Jenkins user
fc3026adc3
[maven-release-plugin] prepare release v0.9.4.Final
2019-04-11 12:07:15 +00:00
Gunnar Morling
2250c47b7f
DBZ-1163 Minor improvements
2019-04-10 10:53:22 +02:00
preethi
02873e8169
DBZ-1163 Enable postgres snapshot for tables without primary key
2019-04-10 10:48:11 +02:00
ShubhamRwt
762d908eef
DBZ-1193 Added support for cidr address type
2019-04-08 15:00:03 +02:00
jchipmunk
7c0ae3ee20
DBZ-1212 SLF4J usage issues
2019-04-04 21:32:12 +02:00
Jiri Pechanec
ff52f3713e
DBZ-1144 Test for geometry in public schema
2019-04-02 15:57:57 +02:00
Alexander Kovryga
4616df189e
DBZ-1189 Add support for inet data type.
2019-03-28 09:57:32 +01:00
ShubhamRwt
540a951211
DBZ- 362 Adding WhitespaceAfter check to Checkstyle
2019-03-28 09:24:11 +01:00
Jenkins user
8b0edd1871
[maven-release-plugin] prepare for next development iteration
2019-03-25 12:16:31 +00:00
Jenkins user
56f98a48b2
[maven-release-plugin] prepare release v0.9.3.Final
2019-03-25 12:16:30 +00:00
Renato Mefi
6300f2972f
DBZ-1169 Integration tests for Outbox EventRouter
...
Set the Snapshot Mode to NEVER provides more consistent tests
2019-03-25 11:27:44 +01:00
Renato Mefi
8b689d7f03
DBZ-1169 Implement RegexRouter based on config
2019-03-25 11:27:44 +01:00
Renato Mefi
f0f13ed4a4
DBZ-1169 Implement basic behavior
2019-03-25 11:27:44 +01:00
Renato Mefi
2ee1c98298
DBZ-1169 Setup EventRouter on Postgres Test IT
2019-03-25 11:27:44 +01:00
shubham
8cca21e969
DBZ-1039 Make all ifs with braces
2019-03-25 10:23:14 +01:00
Addison Higham
8e21139ca3
DBZ-1082 Add new custom recovery mode, more metadata
...
This commit does a few things:
- Refactors snapshot modes to be encapsulated by an interface and
to only use that interface in determining when to snapshot and in
determing the type of the `RecordProducer` interface to instantiate
- Refactors the configuration of existing snapshot modes to tie the
existing snapshot modes to their aligned implementation
- Adds a new snapshot.mode, custom, and a new configuration option to
specify a custom implementation that will be loaded by the class loader
- Changes the visibility of some classes to allow for custom snapshot
modes to get enough context to make an informed choice
- Adds some metadata about slots (the catalog_xmin) to give a full idea
of the state of slots which can be useful in implementing snapshot
modes (which is also configurable, as it can add some overhead)
Together, these changes allow for a much broader flexibility got end
users to implement a snapshot mode that can do more advanced snapshots,
such as partial recovery or for partial snapshots for tables where not
all records are needed.
This could also be seen as superseeding the
`snapshot.select.statement.overrides` to allow for users to dynamically
build queries based on the state of the slot and the offsets consumed.
2019-03-24 11:56:40 +01:00
Jiri Pechanec
b11f1b8d05
DBZ-1161 Start snapshot in async thread
2019-03-21 14:32:47 +01:00
trolison
2001b178d7
- Fixed bad formatting
...
- Added trace for when empty messages are processed
DBZ-1181
2019-03-19 10:46:36 -07:00
trolison
df0e789601
- Add support for processing WAL2JSON events that have no changes
...
DBZ-1181
2019-03-18 20:32:03 -07:00
Gunnar Morling
2ea87727cf
DBZ-1174 Using Instant instead of long for representing timestamps
2019-03-12 17:19:13 +01:00
Ashhar Hasan
0b07afd23e
DBZ-1174 Convert commitTime from nanos to micros for wal2json
...
- wal2json sends the txn commitTime using a function from PostgreSQL's C
library. The value that Debezium recevies is in nanoseconds.
- decoderbufs sends the txn commitTime in microseconds.
- RecordsSnapshotProducer updates SourceInfo.ts_usec by converting
System.currentTimeMillis() to microseconds.
- RecordsSnapshotProducer updates the SourceInfo's ts_usec field using
message.getCommitTime().
This means that when using wal2json, the value of SourceInfo.ts_usec
is in microseconds since epoch during snapshot but is in nanoseconds
during streaming. To fix this, we changed
Wal2JsonReplicationMessage.getCommitTime() to return in microseconds.
2019-03-12 17:19:13 +01:00
Josh Stanfield
e4b7b90637
DBZ-1162 fix to ensure hstore snapshots behave the same as streams
2019-03-04 13:39:50 +01:00
Jenkins user
50a7c568fa
[maven-release-plugin] prepare for next development iteration
2019-02-23 09:53:58 +00:00
Jenkins user
28f3839804
[maven-release-plugin] prepare release v0.9.2.Final
2019-02-23 09:53:58 +00:00
Jiri Pechanec
78bfda09a6
DBZ-1146 Test full column diff
2019-02-22 14:16:03 +01:00
Gunnar Morling
5043d41b7b
DBZ-1146 Expanding test
2019-02-22 14:16:03 +01:00
Jiri Pechanec
4dec844fa2
DBZ-1146 Fix table created in streaming test
2019-02-22 14:16:03 +01:00
Jiri Pechanec
fa1e632a90
DBZ-1146 Handle toastable columns for snapshotted tables
2019-02-22 14:16:03 +01:00
Jiri Pechanec
89feae32e7
DBZ-1141 Test for the default values
2019-02-21 15:02:51 +01:00
Gunnar Morling
6787d17d54
DBZ-1141 Misc. clean-up;
...
* Removing redundant check for date mapping type
* Always using String as fallback value for temporal values where needed
* Pulling fallback temporal values up to JdbcValueConverters
2019-02-21 15:02:51 +01:00
Jiri Pechanec
b9d326711d
DBZ-1141 Incorrect money scale
2019-02-21 15:02:51 +01:00
Jiri Pechanec
992c31fa1f
DBZ-1141 Add non-date types
2019-02-21 15:02:51 +01:00
Jiri Pechanec
41ba7128e3
DBZ-1141 Correct default falback values for connect date types
2019-02-21 15:02:51 +01:00
Keith Barber
95b2290df2
DBZ-1149 Ensure consistent Java types for fallback values when converting
2019-02-21 15:02:51 +01:00
taylor-rolison
17c6a7a82d
Config option to pass optional logical decoder stream params ( #783 )
...
DBZ-1130 Added config param slot.stream.params to the Postgres connector to allow passing optional params to logical decoder on stream start
2019-02-21 14:06:51 +01:00
Jenkins user
b96e5fee0d
[maven-release-plugin] prepare for next development iteration
2019-02-13 11:13:44 +00:00
Jenkins user
3de2d04bcd
[maven-release-plugin] prepare release v0.9.1.Final
2019-02-13 11:13:44 +00:00
Jiri Pechanec
9049078607
DBZ-997 Handle de-sync of message and JDBC PK info
2019-02-13 10:13:12 +01:00
Gunnar Morling
2a486ecfff
DBZ-1019 Removing legacy methods from Metronome
2019-02-11 18:11:34 +01:00
Ivan Lorenz
721e8112cd
DBZ-1126 Localize era display value to default locale
2019-02-06 15:34:30 +01:00
Gunnar Morling
7076deba5a
DBZ-1118 Adding JUnit rule to skip tests depending on Postgres version
2019-02-04 14:43:19 +01:00
Kevin Pullin
c414a8b942
Add test for DBZ-1118
2019-02-04 14:43:19 +01:00
Kevin Pullin
1330df8908
Postgres snapshot - On null records, clear prior value from currentRecord
2019-02-04 14:43:19 +01:00
Gunnar Morling
41b7fbe78e
Adding note on testing Postgres connector with external DB
2019-01-30 09:25:12 +01:00
Gunnar Morling
cc0df61665
DBZ-1111 Allowing to use drop-slot-on-close with PG 10 or newer
2019-01-29 14:44:10 +01:00
Gunnar Morling
e84c6e2238
DBZ-962 Avoiding dependency to JAXB
2019-01-29 10:28:48 +01:00
Jenkins user
c5b7d21d3e
[maven-release-plugin] prepare for next development iteration
2019-01-28 11:07:04 +00:00
Jenkins user
f1ae79ff73
[maven-release-plugin] prepare release v0.9.0.CR1
2019-01-28 11:07:04 +00:00
Gunnar Morling
ccff79e07d
DBZ-1105 Removing TableImpl#columnNames field
2019-01-28 10:23:56 +01:00
Gunnar Morling
9dc2c525c9
DBZ-1083 Passing "include-unchanged-toast" option only for wal2json
2019-01-25 11:04:43 +01:00
Addison Higham
26f836e0f6
DBZ-1080 new API for EmbeddedEngine for batch/async
...
This introduces a new API to the EmbeddedEngine, the ChangeConsumer,
which gives the user a more flexible option for consuming changes by
exposing groups of records as well as the ability to control the
comitting of those records.
This remainds completely backwards compatible with the old API as the
ChangeConsumer wraps the existing Consumer interface with a default
implementation.
2019-01-18 14:06:50 +01:00
Jiri Pechanec
1cf257478b
DBZ-781 Enable unstable tests
2019-01-17 08:59:42 +01:00
Tony Rizko
a921613877
Fix typo in PostgresConnection error message
2019-01-17 08:58:12 +01:00