AndreyIg
77662a0dde
DBZ-137 contribution to the core, Oracle related
2020-09-14 12:33:43 +02:00
cooksey
8bcbdb639d
DBZ-2094 Allow Postgres snapshotter to set streaming start on resume
...
When a connector resumes after previously streaming and takes a
snapshot, through a new method on the snapshotter interface,
shouldStreamEventsStartingFromSnapshot, can choose whether
to resume streaming from the last known streaming position or the
beginning of the snapshot. This is helpful for snapshotters that
may not want to resnapshot every table in the whitelist/blacklist
but not miss event on the tables that are skipped.
2020-09-09 10:54:11 +02:00
Debezium Builder
9979db648f
[maven-release-plugin] prepare for next development iteration
2020-08-28 05:04:26 -04:00
Debezium Builder
5c58b23756
[maven-release-plugin] prepare release v1.3.0.Beta1
2020-08-28 05:04:26 -04:00
Jiri Pechanec
03a11efd8e
DBZ-2338 Atomic is not necessary
2020-08-24 21:16:37 +02:00
Jiri Pechanec
cf0db85436
DBZ-2338 Flush LSNs on tx boundaries
2020-08-24 21:16:37 +02:00
Jiri Pechanec
483a483569
DBZ-2338 Log exception; restart keepalive
2020-08-24 21:16:37 +02:00
Jiri Pechanec
f01ab234a4
DBZ-2338 Skip initial statements on reconnect
2020-08-24 21:16:37 +02:00
Jiri Pechanec
4d0ec1f848
DBZ-2397 Test for restart in tx
2020-08-24 21:16:37 +02:00
Jiri Pechanec
aead9a6612
DBZ-2338 Minor corrections
...
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
2020-08-24 21:16:37 +02:00
Jiri Pechanec
b5c2ed17ed
DBZ-2338 Support for pgoutput
2020-08-24 21:16:37 +02:00
Jiri Pechanec
39623aa799
DBZ-2338 Base WAL position detection on commit LSNs
2020-08-24 21:16:37 +02:00
Jiri Pechanec
31721c247b
DBZ-2200 Introduce LLSN domain type
2020-08-24 21:16:37 +02:00
rkerner
2c5c093587
DBZ-2171 avoid divisive language
2020-08-21 09:29:04 +02:00
Gunnar Morling
e89b65238f
DBZ-2423 Increasing visibility of some methods for UI PoC usage
2020-08-12 13:50:42 +02:00
Gunnar Morling
44efd1bc6b
DBZ-2423 Propagating port default values
2020-08-12 13:50:42 +02:00
Gunnar Morling
26b51e02cb
DBZ-2418 Misc. clean-up
2020-08-12 09:28:35 +02:00
Gunnar Morling
ebe5169a66
DBZ-2418 Delaying retrieval of DB charset;
...
This is called only once anyways, so there's no need for storing the info
in a field in PostgresConnection. This allows PostgresConnector#validate()
bring up a more meaningful error message in case of incorrect credentials
and other incorrect connection configuration.
2020-08-12 09:28:35 +02:00
Debezium Builder
c4c50f5fc6
[maven-release-plugin] prepare for next development iteration
2020-08-06 00:46:18 -04:00
Debezium Builder
94cbeb95fa
[maven-release-plugin] prepare release v1.3.0.Alpha1
2020-08-06 00:46:17 -04:00
rkerner
cad6cf62d3
DBZ-2315 fix BIT VARYING handling in Postgres connector
...
* removed little endian padding for BIT types in JdbcValueConverters (only used for Postgres yet)
* removed legacy format handling fot BIT related types in JdbcValueConverters#convertBits which were leftovers from PR #1408
* removed unnecessary zero-ing of a newly created byte array to improve performance for huge byte arrays
* updated Postgres connector docs
2020-08-03 14:58:31 +02:00
Björn Häuser
04254e3fe7
DBZ-2310: Return true when messages are skipped
...
See https://issues.redhat.com/browse/DBZ-2310 for more explanation.
Short summary: Whenever LSNs are skipped return back that messages have
been processed, which will cause two things:
* The replication can be advanced properly
* The connector does not fallback to the poll timeout waiting interval
I also think its more correct to say, "yes I have read messages, but
still skipped all of them".
2020-07-21 15:53:06 +02:00
Debezium Builder
e1ab38de84
[maven-release-plugin] prepare for next development iteration
2020-07-16 04:40:04 -04:00
Debezium Builder
69cf375faa
[maven-release-plugin] prepare release v1.2.1.Final
2020-07-16 04:40:04 -04:00
Jiri Pechanec
f06a14f1da
DBZ-2288 Rethrow exception in test
2020-07-16 09:51:00 +02:00
Jiri Pechanec
55ba8b8b89
DBZ-2288 Fix copy/paste errors
2020-07-16 09:51:00 +02:00
Jiri Pechanec
ea942a4268
DBZ-2288 Test for pgoutput
2020-07-16 09:51:00 +02:00
Jiri Pechanec
8b0b252eb9
DBZ-2288 Events in exported snapshot no longer filtered by LSN
2020-07-16 09:51:00 +02:00
Jos Huiting
da97be7842
DBZ-2307 - Improve logging when primary keys are missing
2020-07-15 11:18:48 -04:00
Chris Cranford
94f2932e95
DBZ-2271 Fix test failure - RecordsStreamProducerIT#testEmptyChangesProducesHeartbeat
...
This fixes a race condition with heartbeat emitted messages with empty DDLs
2020-07-14 17:25:44 +02:00
Chris Cranford
0e5122c69d
DBZ-2319 Fix test failure - PostgresConnectorIT#shouldOutputRecordsInCloudEventsFormat
2020-07-09 08:22:13 +02:00
Chris Cranford
4c6351e5e3
DBZ-2271 Fix test failure - RecordsStreamProducerIT#testEmptyChangesProducesHeartbeat
2020-07-08 17:23:58 +02:00
Debezium Builder
c0d30b87d9
[maven-release-plugin] prepare for next development iteration
2020-06-24 18:03:11 +02:00
Debezium Builder
0e2ac78301
[maven-release-plugin] prepare release v1.2.0.Final
2020-06-24 18:03:11 +02:00
Chris Cranford
41bfac49a3
DBZ-2230 Fix intermittent CI test failure - Postgres
2020-06-22 18:27:43 +02:00
Debezium Builder
b03a4d0e50
[maven-release-plugin] prepare for next development iteration
2020-06-19 00:39:01 -04:00
Debezium Builder
93e60258d5
[maven-release-plugin] prepare release v1.2.0.CR2
2020-06-19 00:39:00 -04:00
Gunnar Morling
f3b07a1374
DBZ-1916 Formatting
2020-06-16 13:28:49 +02:00
Chris Cranford
64bff85ad2
DBZ-1916 Add tests for datatype.propagate.source.type
2020-06-16 13:28:49 +02:00
cooksey
1282403c44
DBZ-2176 Open transaction test are more reliable
...
Checking pg_stat_activity for transactions with a state of "idle in transaction"
should be more reliable to find open abandoned transaction than looking for
a non-null backend_xmin. For example, an autovaccum process will have a
backend_xmin but will not have a stat of "idle in transaction"
2020-06-15 11:15:51 +02:00
Debezium Builder
65c39e9f11
[maven-release-plugin] prepare for next development iteration
2020-06-11 02:00:20 -04:00
Debezium Builder
bce332c900
[maven-release-plugin] prepare release v1.2.0.CR1
2020-06-11 02:00:19 -04:00
Lukas Krejci
a4689611f5
DBZ-2159 Fix a typo in the package name.
2020-06-09 19:28:33 +02:00
Lukas Krejci
90c637b1e3
DBZ-2159 Include the Postgres SPI in the API checks.
2020-06-09 19:28:33 +02:00
Gunnar Morling
8cac498a60
DBZ-2156 More meaningful exception in case of replication slot conflict
2020-06-08 10:28:08 +02:00
Jiri Pechanec
13419ae74a
DBZ-2118 Document type registry connection capability
2020-06-08 09:22:05 +02:00
Jiri Pechanec
d2ef7b7011
DBZ-2118 Slot dropping more resilient to race
2020-06-08 09:22:05 +02:00
Jiri Pechanec
642f4e7303
DBZ-2118 Switch to manual tx management
2020-06-08 09:22:05 +02:00
Jiri Pechanec
8c378400eb
DBZ-2118 Type registry provided only for dedicated connections
2020-06-08 09:22:05 +02:00
Jiri Pechanec
853ebafd7f
DBZ-2118 Close TX after type query
2020-06-08 09:22:05 +02:00
Chris Cranford
9b33264334
DBZ-2157 Fix intermittent postgres test failures
2020-06-05 20:02:14 +02:00
Chris Cranford
845a790593
DBZ-2060 Fix intermittent test failures (sqlserver & postgres)
2020-06-05 20:02:14 +02:00
Gunnar Morling
02b16e17ac
DBZ-1814 Adding tests for exporting unkown column type
2020-06-05 12:13:29 +02:00
Gunnar Morling
c54e377dd2
DBZ-1814 Exporting Hex and Base64 representations as string
2020-06-05 12:13:29 +02:00
Gunnar Morling
023064eedb
DBZ-1814 Renaming "raw" to "bytes"; also clarifying that this notion isn't Java-specific
2020-06-05 12:13:29 +02:00
Jeremy Finzel
5db614da21
DBZ-1814 added new binary handling mode option
2020-06-05 12:13:29 +02:00
Gunnar Morling
d7b3f0a7fb
DBZ-1813 Misc. clean-up;
...
* Removing unused method
* Simplifying tests
* Avoiding deprecated code
2020-06-05 06:24:25 +02:00
Gunnar Morling
59f34e7b16
DBZ-1813 Stabiling test
2020-06-05 06:24:25 +02:00
Brandon Brown
bd64ace7fc
DBZ-1813 Adding ability to configure pgoutput publication based on table filters
...
defined.
2020-06-05 06:24:25 +02:00
Debezium Builder
dc25c379e5
[maven-release-plugin] prepare for next development iteration
2020-05-19 03:40:25 -04:00
Debezium Builder
ad060181b9
[maven-release-plugin] prepare release v1.2.0.Beta2
2020-05-19 03:40:25 -04:00
Braden Groom
8439e70d09
DBZ-2090 Remove unused sourceInfo variable
2020-05-19 08:30:55 +02:00
Jiri Pechanec
97c3d3b6e9
DBZ-2088 Use POSIX tar
2020-05-18 14:29:38 +02:00
Chris Cranford
f449550b2d
DBZ-2068 Skip tests incompatible with Kafka 1.x
2020-05-13 08:22:07 +02:00
Debezium Builder
8d4203279e
[maven-release-plugin] prepare for next development iteration
2020-05-07 05:03:40 -04:00
Debezium Builder
a4e2c8f36c
[maven-release-plugin] prepare release v1.2.0.Beta1
2020-05-07 05:03:40 -04:00
Gunnar Morling
ab1d4bb1cd
DBZ-1970 Making change event API more consistent and uniform
2020-05-06 14:33:37 +02:00
Jiri Pechanec
d82307c331
DBZ-1970 Support for separate key/value embedded converter
2020-05-06 14:33:37 +02:00
Chris Cranford
cd43c4880f
DBZ-2014 Do not include outbox eventType field by default
2020-05-05 22:13:25 +02:00
Chris Cranford
81c3a808da
DBZ-1963 Suggested changes
2020-05-05 08:45:58 +02:00
Chris Cranford
a1cb06299c
DBZ-1963 Emit outbox events with value schema names
2020-05-05 08:45:58 +02:00
Jiri Pechanec
7f23d74d90
DBZ-1904 Add config param into a group
2020-05-04 12:31:05 +02:00
Jiri Pechanec
e350d74b17
DBZ-1904 Public facing schema history topic
2020-05-04 12:31:05 +02:00
Braden Groom
a8cb4e319c
DBZ-2044 Load Postgres enum values with a single query instead of N queries
2020-05-04 12:08:16 +02:00
Chris Cranford
96dda6c3cb
DBZ-1969 Add array support for time, timetz, timestamp, and timestamptz
2020-05-04 11:22:49 +02:00
Braden Groom
8093582487
DBZ-2038 Refactor postgres type registry queries to use the same base query
2020-04-30 14:29:14 -04:00
Braden Groom
63f96940fb
DBZ-2038 Factor out logic for converting postgres result sets to type builders
2020-04-30 14:29:14 -04:00
Braden Groom
50c61fe462
DBZ-2038 Prime postgres array and non-array types using same logic
2020-04-30 14:29:14 -04:00
Braden Groom
01dda2a8a9
DBZ-2038 Load unknown postgres types by name and oid using same logic
2020-04-30 14:29:14 -04:00
Braden Groom
905a732342
DBZ-2038 Resolve array and non-array postgres types with the same logic
2020-04-30 14:29:14 -04:00
Braden Groom
284a4357e0
DBZ-1969 Update convertLtreeArray to handle List instances
2020-04-29 13:06:43 -04:00
Braden Groom
1578ebc8bb
DBZ-1969 Have isArray return false for Postgres LTree arrays
2020-04-29 13:06:43 -04:00
Gunnar Morling
2310d6c792
DBZ-1969 Fixing incorrect assertion and utility which concealed this
2020-04-29 13:06:43 -04:00
Gunnar Morling
f718302d36
DBZ-1969 Simplifying array handling in PgProtoColumnValue
2020-04-29 13:06:43 -04:00
Braden Groom
0b429ce880
DBZ-1969 Add support for Postgres enum arrays
2020-04-29 13:06:43 -04:00
Chris Cranford
230ddaebdc
DBZ-1953 Generalized SkipWhenDatabaseVersion functionality
2020-04-29 12:27:38 +02:00
Gunnar Morling
3d0606b348
DBZ-2027 Small speed-up of PG test execution;
...
* Using Awaitility so we can use 100ms looping intervals; also it's more concise
* Avoiding creation of one temporary connection
2020-04-29 09:56:59 +02:00
Gunnar Morling
16b780fa3c
DBZ-2027 Upgrading Postgres JDBC driver to 42.2.12;
...
The driver upgrade mitigates some issues with using this connector with
Postgres on Azure. It comes with some behavioural changes, though:
* column metadata for DECIMAL without scale is returned differently by
the (see https://github.com/pgjdbc/pgjdbc/issues/1767 ): while it used
to be returned as 0, it's now returned as null. This should be
transparent to DBZ consumers
* snapshots for partitioned tables only export change events on the
partition topics now due to https://github.com/pgjdbc/pgjdbc/pull/1708 ;
this has an impact on consumers, but I think it's more reasonable than
exporting all change events twice, one partition table and main table
topics
2020-04-29 09:56:59 +02:00
Anton Kondratev
236ea1a15f
DBZ-1990 Support for Postgres Json and Jsonb array columns
2020-04-27 22:17:48 +03:00
JanHendrikDolling
33d83731af
DBZ-1972 move static Filed methods as static Filed constant to RelationalDatabaseConnectorConfig instead.
2020-04-27 12:31:19 +02:00
Gunnar Morling
ea7f8e918b
DBZ-1750 Removing unused method
2020-04-24 15:35:28 +02:00
Gunnar Morling
03ce403972
DBZ-1750 Separating config definition editor and immutable config definitions
2020-04-24 15:35:28 +02:00
Jiri Pechanec
37f0dbcaf8
DBZ-1750 Limit access to parent defs; dont share the parent defs
2020-04-24 15:35:28 +02:00
Jiri Pechanec
af90507fa6
DBZ-1750 Centralize shared config definitions
2020-04-24 15:35:28 +02:00
dajerome
ad8beb76d0
DBZ-1949 Return BitSet instead of short/int/long;
...
When converting Postgres bit varying column, convert to BitSet instead of short/int/long to handle cases where the value is larger than Long.MAX_VALUE
2020-04-21 11:26:49 +02:00
Jos Huiting
33fe631674
DBZ-1962 Fix PR feedback
2020-04-17 15:24:42 +02:00
Jos Huiting
a0f65830de
DBZ-1962 Remove TABLE_WHITELIST and TABLE_BLACKLIST in favor of definitions in RelationalDatabaseConnectorConfig
2020-04-17 15:24:42 +02:00
Jos Huiting
8ba21a98ef
DBZ-1962 Introduce column.whitelist for the Postgres connector
2020-04-17 15:24:42 +02:00
Debezium Builder
4ca9f92575
[maven-release-plugin] prepare for next development iteration
2020-04-16 02:22:00 -04:00
Debezium Builder
e704bccdaa
[maven-release-plugin] prepare release v1.2.0.Alpha1
2020-04-16 02:22:00 -04:00
Gunnar Morling
659d4a415b
DBZ-1807 Misc. adjustments;
...
* Adding @TestFor annotations
* Change -> ChangeEvent
2020-04-15 16:29:29 +02:00
Jiri Pechanec
c0914b245b
DBZ-1807 Support for CloudEvents converter
2020-04-15 16:29:29 +02:00
Jiri Pechanec
c06707f4c5
DBZ-1807 Support for Avro converters
2020-04-15 16:29:29 +02:00
Jiri Pechanec
4e2ac367a9
DBZ-1807 Support for JSON Converter
2020-04-15 16:29:29 +02:00
Chris Cranford
57af80afd5
DBZ-1859 Enhance MongoDB connector metrics
2020-04-15 08:04:23 +02:00
Jiri Pechanec
6219376bfe
DBZ-1960 Remove sources of busy wait
2020-04-14 12:59:10 +02:00
JanHendrikDolling
1e19270981
DBZ-1692 Added support for hash based column masking mode for string values
2020-04-08 21:21:42 +02:00
Chris Cranford
5f8719f201
DBZ-1925 Fix test regression after Confluent Platform 5.4.1 upgrade
2020-04-07 09:01:20 +02:00
Jiri Pechanec
cacd21c44a
DBZ-1922 wal2_json streaming uses NOOP messages
2020-04-02 17:29:16 +02:00
rk3rn3r
7f36b3c08f
DBZ-1531 added headers for primary key update events to reference the original key
2020-03-25 12:33:38 +01:00
rk3rn3r
6310e00675
DBZ-1531 added headers for primary key update events to reference the original key
2020-03-25 12:33:38 +01:00
rk3rn3r
fbd8b94e94
DBZ-1531 added headers for primary key update events to reference the original key
2020-03-25 12:33:38 +01:00
Debezium Builder
ab54ce062f
[maven-release-plugin] prepare for next development iteration
2020-03-24 01:23:05 -04:00
Debezium Builder
0b5a9117fb
[maven-release-plugin] prepare release v1.1.0.Final
2020-03-24 01:23:05 -04:00
Jiri Pechanec
224f97e380
DBZ-1857 Support for autorestart for embedded engine
2020-03-23 13:56:29 +01:00
Jiri Pechanec
b44cd8475e
DBZ-1815 Use separatae conection for db heartbeat
2020-03-23 12:02:31 +01:00
trolison
8784b41e22
DBZ-1815 Ading DatabaseHeartbeatImpl to allow for making queries to the DB when a heartbeat happens;
...
* Minor refactor to make DatabaseHeartbeatImpl usable
* Added integration test to verify heartbeat.action.query is being executed properly
2020-03-18 10:27:22 +01:00
Chris Cranford
5ce9d6fb14
DBZ-1755 Fix wal2json/pgoutput streaming of negative monetary values
2020-03-17 08:36:24 +01:00
Chris Cranford
f6d7778bb4
DBZ-1755 Suggested changes
...
* Add test case for snapshotting null currency value
* Restore previous exception handling for money types
2020-03-17 08:36:24 +01:00
Chris Cranford
0cc6e34264
DBZ-1755 Handle parsing negative monetary values during snapshot
2020-03-17 08:36:24 +01:00
Fabian Aussems
1be07ca4f4
DBZ-1755 Handle error conditions better for the Pg MONEY datatype
2020-03-17 08:36:24 +01:00
Jiri Pechanec
b114520f5e
DBZ-1875 Test stabilization;
...
* Snapshot metrics are present with snapshot never
* Skip extra empty TX
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
2020-03-13 11:40:00 +01:00
Jiri Pechanec
4ffed1fa46
DBZ-1730 Improve test limits description
2020-03-13 09:39:23 +01:00
Jiri Pechanec
ee91eac418
DBZ-1730 Stream from replica slot position not end of tx log
2020-03-13 09:39:23 +01:00
Gunnar Morling
449154beaa
DBZ-1863 Centralizing coordinator shutdown
2020-03-12 11:28:08 +01:00
Debezium Builder
86c1ed90ed
[maven-release-plugin] prepare for next development iteration
2020-03-11 09:54:44 -04:00
Debezium Builder
062c6abbe5
[maven-release-plugin] prepare release v1.1.0.CR1
2020-03-11 09:54:44 -04:00
Gunnar Morling
c51a5fb597
DBZ-1723 Restarting after connection losses in Postgres
2020-03-11 13:57:23 +01:00
Gunnar Morling
3437da00d7
DBZ-1723 Simplifying shutdown handling;
...
doStop() is already called from stop(), no need to do it again via the error handler.
2020-03-11 13:57:23 +01:00
Gunnar Morling
c156d30048
DBZ-1723 Centralizing stopping() logic in BaseSourceTask
2020-03-11 13:57:23 +01:00
Gunnar Morling
8ce69dc597
DBZ-1723 Encapsulating common poll() and stop() logic in BaseSourceTask
2020-03-11 13:57:23 +01:00
Chris Cranford
6f1eb49b32
DBZ-1726 Fix formatting failures
2020-03-11 10:15:00 +01:00
Chris Cranford
3a4788cc33
DBZ-1726 Introduced AbstractSnapshotChangeEventSource and refactor implementations
2020-03-11 10:15:00 +01:00
Chris Cranford
5506fadd2d
DBZ-1664 Verify correctness of JMX metrics
2020-03-09 23:30:24 +01:00
Jiri Pechanec
0248fdde16
DBZ-1850 Delegate signed/nonsigned conversion to driver
2020-03-09 13:06:30 +01:00
Jiri Pechanec
69e87170e3
DBZ-1849 Only log message was fixed
2020-03-09 07:43:18 +01:00
Jiri Pechanec
62e6a0add7
DBZ-1849 Avoid type refresh for unchanged root type
2020-03-06 18:01:56 +01:00
Jiri Pechanec
f99064fda1
DBZ-1824 Restart incomplete Tx for wal2json
2020-03-04 14:03:21 +01:00
Gunnar Morling
a8b796685b
DBZ-1840 Typo fix; commenting
2020-03-04 09:31:22 +01:00
Jiri Pechanec
10dfc12c99
DBZ-1840 Explanation for unresolvable tables
2020-03-04 09:25:57 +01:00
Jiri Pechanec
19d0a88cc9
DBZ-1840 Test logic was out of order
2020-03-04 09:25:57 +01:00
Jiri Pechanec
b2370c2d8e
DBZ-1840 Syntheitc message to replace messages dropped in decoder
2020-03-04 09:25:57 +01:00
Jiri Pechanec
6bab948e76
DBZ-1840 No nulls to stream producer
2020-03-04 09:25:57 +01:00
Jan-Hendrik Dolling
5cfbaab7a6
DBZ-1845 Typos in deployment.yaml and code readability
...
* improve readability
* fixed some typos
2020-03-04 08:34:55 +01:00
Gunnar Morling
c5422e6b31
DBZ-1840 Not passing null to replication message processor;
...
Note this means to that WAL offsets cannot be acknowledged in cases
where a null message is received from the server (e.g. wal2json for
DDL events). But as we cannot tell apart null events from the server
from readPending() simply having no events to read, the current
logic renders the pausing between two read calls without events
obsolete, so that's the lesser evil
2020-03-03 11:45:53 +01:00
Gunnar Morling
0f6d2c54fc
DBZ-1134 Avoiding concrete class in SPI by introducing receiver parameter
2020-02-28 21:58:39 +01:00
Jiri Pechanec
d13e48cc55
DBZ-1134 Make converted field type parametrized
2020-02-28 21:58:39 +01:00
Jiri Pechanec
06e1eca085
DBZ-1134 Support for user provided converters
2020-02-28 21:58:39 +01:00
Jiri Pechanec
91f292a1f2
DBZ-1825 Keep keys for non-matching table
2020-02-26 15:03:03 +01:00
Jiri Pechanec
ae6ee3d458
DBZ-234 Place Incubating into a unique package
2020-02-25 15:31:10 +01:00