Commit Graph

501 Commits

Author SHA1 Message Date
Renato Mefi
ebf36e1ddb DBZ-1169 Support skipping deletions and tombstones 2019-03-25 11:27:44 +01:00
Renato Mefi
6512c0b387 DBZ-1169 Add Configuration for Outbox Transform 2019-03-25 11:27:44 +01:00
Jiri Pechanec
9b405aa84d DBZ-1039 Add missing spaces 2019-03-25 10:23:14 +01:00
shubham
8cca21e969 DBZ-1039 Make all ifs with braces 2019-03-25 10:23:14 +01:00
Addison Higham
8e21139ca3 DBZ-1082 Add new custom recovery mode, more metadata
This commit does a few things:

- Refactors snapshot modes to be encapsulated by an interface and
to only use that interface in determining when to snapshot and in
determing the type of the `RecordProducer` interface to instantiate
- Refactors the configuration of existing snapshot modes to tie the
existing snapshot modes to their aligned implementation
- Adds a new snapshot.mode, custom, and a new configuration option to
specify a custom implementation that will be loaded by the class loader
- Changes the visibility of some classes to allow for custom snapshot
modes to get enough context to make an informed choice
- Adds some metadata about slots (the catalog_xmin) to give a full idea
of the state of slots which can be useful in implementing snapshot
modes (which is also configurable, as it can add some overhead)

Together, these changes allow for a much broader flexibility got end
users to implement a snapshot mode that can do more advanced snapshots,
such as partial recovery or for partial snapshots for tables where not
all records are needed.

This could also be seen as superseeding the
`snapshot.select.statement.overrides` to allow for users to dynamically
build queries based on the state of the slot and the offsets consumed.
2019-03-24 11:56:40 +01:00
Jon Casstevens
2efdae1ffd DBZ-1179 Set the default replication factor property for history topics when none is provided 2019-03-21 12:07:10 +01:00
Gunnar Morling
2ea87727cf DBZ-1174 Using Instant instead of long for representing timestamps 2019-03-12 17:19:13 +01:00
Jiri Pechanec
71765a2ab2 DBZ-1170 Detect and handle errors like ULE 2019-03-11 10:29:54 +01:00
Josh Stanfield
e4b7b90637 DBZ-1162 fix to ensure hstore snapshots behave the same as streams 2019-03-04 13:39:50 +01:00
Jenkins user
50a7c568fa [maven-release-plugin] prepare for next development iteration 2019-02-23 09:53:58 +00:00
Jenkins user
28f3839804 [maven-release-plugin] prepare release v0.9.2.Final 2019-02-23 09:53:58 +00:00
Gunnar Morling
6787d17d54 DBZ-1141 Misc. clean-up;
* Removing redundant check for date mapping type
* Always using String as fallback value for temporal values where needed
* Pulling fallback temporal values up to JdbcValueConverters
2019-02-21 15:02:51 +01:00
Jiri Pechanec
41ba7128e3 DBZ-1141 Correct default falback values for connect date types 2019-02-21 15:02:51 +01:00
Keith Barber
95b2290df2 DBZ-1149 Ensure consistent Java types for fallback values when converting 2019-02-21 15:02:51 +01:00
Jiri Pechanec
ff5aaf2e9a DBZ-1153 Support for special char names 2019-02-20 15:25:45 +01:00
Gunnar Morling
510e9415f9 DBZ-1137 Some clean-up;
* Using existing justifyLeft() method
* Giving expected values as Duration
* Removing unused methods
2019-02-19 18:41:09 +01:00
Jiri Pechanec
0d60e60243 DBZ-1137 Default for TIME can be generic interval value 2019-02-19 13:40:01 +01:00
Jenkins user
b96e5fee0d [maven-release-plugin] prepare for next development iteration 2019-02-13 11:13:44 +00:00
Jenkins user
3de2d04bcd [maven-release-plugin] prepare release v0.9.1.Final 2019-02-13 11:13:44 +00:00
Jiri Pechanec
9049078607 DBZ-997 Handle de-sync of message and JDBC PK info 2019-02-13 10:13:12 +01:00
tomaz.fernandes
4e6b44d925 DBZ-1122 Fixed a potential NPE in FileDatabaseHistory 2019-02-12 13:17:09 +01:00
Gunnar Morling
2a486ecfff DBZ-1019 Removing legacy methods from Metronome 2019-02-11 18:11:34 +01:00
Jiri Pechanec
fa6fcd8487 DBZ-1019 Guard against wall-clock change 2019-02-11 18:11:34 +01:00
Jiri Pechanec
4d76243227 DBZ-1121 Emit ALTER event for updated tables 2019-02-09 20:03:57 +01:00
Gunnar Morling
7076deba5a DBZ-1118 Adding JUnit rule to skip tests depending on Postgres version 2019-02-04 14:43:19 +01:00
Jiri Pechanec
16620f0bf9 DBZ-1040 Cleaning up event formatting 2019-02-04 12:39:07 +01:00
Gunnar Morling
99caad60d8 DBZ-1040 Changing resolution of "lag behind source" to milli-seconds;
Also using Duration/Instant types internally.
2019-02-04 12:39:07 +01:00
Gunnar Morling
f1c4234248 DBZ-1040 Removing value from metrics events 2019-02-04 12:39:07 +01:00
Gunnar Morling
913e0bc5c8 DBZ-1040 Making metadata provider a member of Metrics instead of passing it for each onEvent() call 2019-02-04 12:39:07 +01:00
Gunnar Morling
399f91ce84 DBZ-1040 Typo fix 2019-02-04 12:39:07 +01:00
Gunnar Morling
27274d664f DBZ-1040 Passing source as DataCollectionId instead of Object 2019-02-04 12:39:07 +01:00
Jiri Pechanec
de171917dc DBZ-1040 Improve event to string formatting 2019-02-04 12:39:07 +01:00
Jiri Pechanec
a621e43d8b DBZ-1040 Expose transaction id 2019-02-04 12:39:07 +01:00
Jiri Pechanec
213d4a42ca DBZ-1040 Metrics for internal queue 2019-02-04 12:39:07 +01:00
Jiri Pechanec
d993ee265e DBZ-1040 Metrics for lag behind source, offset and tx count 2019-02-04 12:39:07 +01:00
jchipmunk
acbc94e8b0 DBZ-1112 rename prepareQuery() to prepareQueryWithBlockingConsumer()
Reason:
reference to prepareQuery is ambiguous
[ERROR] both method prepareQuery(java.lang.String,io.debezium.jdbc.JdbcConnection.StatementPreparer,io.debezium.jdbc.JdbcConnection.BlockingResultSetConsumer) in io.debezium.jdbc.JdbcConnection and method prepareQuery(java.lang.String,io.debezium.jdbc.JdbcConnection.StatementPreparer,io.debezium.jdbc.JdbcConnection.ResultSetConsumer) in io.debezium.jdbc.JdbcConnection match
2019-01-30 09:36:09 +01:00
jchipmunk
ff4e38cc46 DBZ-1112 Strings.join() doesn't apply conversation for first element
Replace:
- Integer.parseInt() to Integer.valueOf()
- Short.parseShort() and new Short() to Short.valueOf()
- Long.parseLong() to Long.valueOf()
2019-01-30 09:36:09 +01:00
Gunnar Morling
2a80931d03 DBZ-962 Avoiding Short constructor 2019-01-29 10:28:48 +01:00
Gunnar Morling
e84c6e2238 DBZ-962 Avoiding dependency to JAXB 2019-01-29 10:28:48 +01:00
Jenkins user
c5b7d21d3e [maven-release-plugin] prepare for next development iteration 2019-01-28 11:07:04 +00:00
Jenkins user
f1ae79ff73 [maven-release-plugin] prepare release v0.9.0.CR1 2019-01-28 11:07:04 +00:00
Gunnar Morling
ccff79e07d DBZ-1105 Removing TableImpl#columnNames field 2019-01-28 10:23:56 +01:00
Gunnar Morling
a27c29765a DBZ-947 Formatting 2019-01-28 09:32:57 +01:00
Grzegorz Kołakowski
64c45c7490 DBZ-947 Reset initial transaction isolation level at snapshot end 2019-01-28 09:32:57 +01:00
Gunnar Morling
ebe5fd7e85 DBZ-1067 Adding SourceRecordAssert 2019-01-23 08:31:22 +01:00
Gunnar Morling
9665af3ebc DBZ-1086 Adding Ivan Kovbas to COPYRIGHT.txt 2019-01-18 21:43:45 +01:00
Ivan Kovbas
81f031cb7c DBZ-1086 Added functionality to gracefully handle key-less messages in ByLogicalTableRouter transformation. 2019-01-18 21:38:52 +01:00
Gunnar Morling
cedef89a48 DBZ-1089 Using Strings#isNullOrEmpty(); adding test 2019-01-18 10:07:20 +01:00
Jiri Pechanec
28c9734685 DBZ-1089 Use only schema name when both catalog and schema is present 2019-01-18 10:07:20 +01:00
Jiri Pechanec
25156730f0 DBZ-1088 use correct key for update of PK insert event 2019-01-17 14:28:18 +01:00