Braden Groom
a8cb4e319c
DBZ-2044 Load Postgres enum values with a single query instead of N queries
2020-05-04 12:08:16 +02:00
Chris Cranford
96dda6c3cb
DBZ-1969 Add array support for time, timetz, timestamp, and timestamptz
2020-05-04 11:22:49 +02:00
Braden Groom
8093582487
DBZ-2038 Refactor postgres type registry queries to use the same base query
2020-04-30 14:29:14 -04:00
Braden Groom
63f96940fb
DBZ-2038 Factor out logic for converting postgres result sets to type builders
2020-04-30 14:29:14 -04:00
Braden Groom
50c61fe462
DBZ-2038 Prime postgres array and non-array types using same logic
2020-04-30 14:29:14 -04:00
Braden Groom
01dda2a8a9
DBZ-2038 Load unknown postgres types by name and oid using same logic
2020-04-30 14:29:14 -04:00
Braden Groom
905a732342
DBZ-2038 Resolve array and non-array postgres types with the same logic
2020-04-30 14:29:14 -04:00
Braden Groom
284a4357e0
DBZ-1969 Update convertLtreeArray to handle List instances
2020-04-29 13:06:43 -04:00
Braden Groom
1578ebc8bb
DBZ-1969 Have isArray return false for Postgres LTree arrays
2020-04-29 13:06:43 -04:00
Gunnar Morling
2310d6c792
DBZ-1969 Fixing incorrect assertion and utility which concealed this
2020-04-29 13:06:43 -04:00
Gunnar Morling
f718302d36
DBZ-1969 Simplifying array handling in PgProtoColumnValue
2020-04-29 13:06:43 -04:00
Braden Groom
0b429ce880
DBZ-1969 Add support for Postgres enum arrays
2020-04-29 13:06:43 -04:00
Chris Cranford
230ddaebdc
DBZ-1953 Generalized SkipWhenDatabaseVersion functionality
2020-04-29 12:27:38 +02:00
Gunnar Morling
3d0606b348
DBZ-2027 Small speed-up of PG test execution;
...
* Using Awaitility so we can use 100ms looping intervals; also it's more concise
* Avoiding creation of one temporary connection
2020-04-29 09:56:59 +02:00
Gunnar Morling
16b780fa3c
DBZ-2027 Upgrading Postgres JDBC driver to 42.2.12;
...
The driver upgrade mitigates some issues with using this connector with
Postgres on Azure. It comes with some behavioural changes, though:
* column metadata for DECIMAL without scale is returned differently by
the (see https://github.com/pgjdbc/pgjdbc/issues/1767 ): while it used
to be returned as 0, it's now returned as null. This should be
transparent to DBZ consumers
* snapshots for partitioned tables only export change events on the
partition topics now due to https://github.com/pgjdbc/pgjdbc/pull/1708 ;
this has an impact on consumers, but I think it's more reasonable than
exporting all change events twice, one partition table and main table
topics
2020-04-29 09:56:59 +02:00
Anton Kondratev
236ea1a15f
DBZ-1990 Support for Postgres Json and Jsonb array columns
2020-04-27 22:17:48 +03:00
JanHendrikDolling
33d83731af
DBZ-1972 move static Filed methods as static Filed constant to RelationalDatabaseConnectorConfig instead.
2020-04-27 12:31:19 +02:00
Gunnar Morling
ea7f8e918b
DBZ-1750 Removing unused method
2020-04-24 15:35:28 +02:00
Gunnar Morling
03ce403972
DBZ-1750 Separating config definition editor and immutable config definitions
2020-04-24 15:35:28 +02:00
Jiri Pechanec
37f0dbcaf8
DBZ-1750 Limit access to parent defs; dont share the parent defs
2020-04-24 15:35:28 +02:00
Jiri Pechanec
af90507fa6
DBZ-1750 Centralize shared config definitions
2020-04-24 15:35:28 +02:00
dajerome
ad8beb76d0
DBZ-1949 Return BitSet instead of short/int/long;
...
When converting Postgres bit varying column, convert to BitSet instead of short/int/long to handle cases where the value is larger than Long.MAX_VALUE
2020-04-21 11:26:49 +02:00
Jos Huiting
33fe631674
DBZ-1962 Fix PR feedback
2020-04-17 15:24:42 +02:00
Jos Huiting
a0f65830de
DBZ-1962 Remove TABLE_WHITELIST and TABLE_BLACKLIST in favor of definitions in RelationalDatabaseConnectorConfig
2020-04-17 15:24:42 +02:00
Jos Huiting
8ba21a98ef
DBZ-1962 Introduce column.whitelist for the Postgres connector
2020-04-17 15:24:42 +02:00
Debezium Builder
4ca9f92575
[maven-release-plugin] prepare for next development iteration
2020-04-16 02:22:00 -04:00
Debezium Builder
e704bccdaa
[maven-release-plugin] prepare release v1.2.0.Alpha1
2020-04-16 02:22:00 -04:00
Gunnar Morling
659d4a415b
DBZ-1807 Misc. adjustments;
...
* Adding @TestFor annotations
* Change -> ChangeEvent
2020-04-15 16:29:29 +02:00
Jiri Pechanec
c0914b245b
DBZ-1807 Support for CloudEvents converter
2020-04-15 16:29:29 +02:00
Jiri Pechanec
c06707f4c5
DBZ-1807 Support for Avro converters
2020-04-15 16:29:29 +02:00
Jiri Pechanec
4e2ac367a9
DBZ-1807 Support for JSON Converter
2020-04-15 16:29:29 +02:00
Chris Cranford
57af80afd5
DBZ-1859 Enhance MongoDB connector metrics
2020-04-15 08:04:23 +02:00
Jiri Pechanec
6219376bfe
DBZ-1960 Remove sources of busy wait
2020-04-14 12:59:10 +02:00
JanHendrikDolling
1e19270981
DBZ-1692 Added support for hash based column masking mode for string values
2020-04-08 21:21:42 +02:00
Chris Cranford
5f8719f201
DBZ-1925 Fix test regression after Confluent Platform 5.4.1 upgrade
2020-04-07 09:01:20 +02:00
Jiri Pechanec
cacd21c44a
DBZ-1922 wal2_json streaming uses NOOP messages
2020-04-02 17:29:16 +02:00
rk3rn3r
7f36b3c08f
DBZ-1531 added headers for primary key update events to reference the original key
2020-03-25 12:33:38 +01:00
rk3rn3r
6310e00675
DBZ-1531 added headers for primary key update events to reference the original key
2020-03-25 12:33:38 +01:00
rk3rn3r
fbd8b94e94
DBZ-1531 added headers for primary key update events to reference the original key
2020-03-25 12:33:38 +01:00
Debezium Builder
ab54ce062f
[maven-release-plugin] prepare for next development iteration
2020-03-24 01:23:05 -04:00
Debezium Builder
0b5a9117fb
[maven-release-plugin] prepare release v1.1.0.Final
2020-03-24 01:23:05 -04:00
Jiri Pechanec
224f97e380
DBZ-1857 Support for autorestart for embedded engine
2020-03-23 13:56:29 +01:00
Jiri Pechanec
b44cd8475e
DBZ-1815 Use separatae conection for db heartbeat
2020-03-23 12:02:31 +01:00
trolison
8784b41e22
DBZ-1815 Ading DatabaseHeartbeatImpl to allow for making queries to the DB when a heartbeat happens;
...
* Minor refactor to make DatabaseHeartbeatImpl usable
* Added integration test to verify heartbeat.action.query is being executed properly
2020-03-18 10:27:22 +01:00
Chris Cranford
5ce9d6fb14
DBZ-1755 Fix wal2json/pgoutput streaming of negative monetary values
2020-03-17 08:36:24 +01:00
Chris Cranford
f6d7778bb4
DBZ-1755 Suggested changes
...
* Add test case for snapshotting null currency value
* Restore previous exception handling for money types
2020-03-17 08:36:24 +01:00
Chris Cranford
0cc6e34264
DBZ-1755 Handle parsing negative monetary values during snapshot
2020-03-17 08:36:24 +01:00
Fabian Aussems
1be07ca4f4
DBZ-1755 Handle error conditions better for the Pg MONEY datatype
2020-03-17 08:36:24 +01:00
Jiri Pechanec
b114520f5e
DBZ-1875 Test stabilization;
...
* Snapshot metrics are present with snapshot never
* Skip extra empty TX
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
2020-03-13 11:40:00 +01:00
Jiri Pechanec
4ffed1fa46
DBZ-1730 Improve test limits description
2020-03-13 09:39:23 +01:00
Jiri Pechanec
ee91eac418
DBZ-1730 Stream from replica slot position not end of tx log
2020-03-13 09:39:23 +01:00
Gunnar Morling
449154beaa
DBZ-1863 Centralizing coordinator shutdown
2020-03-12 11:28:08 +01:00
Debezium Builder
86c1ed90ed
[maven-release-plugin] prepare for next development iteration
2020-03-11 09:54:44 -04:00
Debezium Builder
062c6abbe5
[maven-release-plugin] prepare release v1.1.0.CR1
2020-03-11 09:54:44 -04:00
Gunnar Morling
c51a5fb597
DBZ-1723 Restarting after connection losses in Postgres
2020-03-11 13:57:23 +01:00
Gunnar Morling
3437da00d7
DBZ-1723 Simplifying shutdown handling;
...
doStop() is already called from stop(), no need to do it again via the error handler.
2020-03-11 13:57:23 +01:00
Gunnar Morling
c156d30048
DBZ-1723 Centralizing stopping() logic in BaseSourceTask
2020-03-11 13:57:23 +01:00
Gunnar Morling
8ce69dc597
DBZ-1723 Encapsulating common poll() and stop() logic in BaseSourceTask
2020-03-11 13:57:23 +01:00
Chris Cranford
6f1eb49b32
DBZ-1726 Fix formatting failures
2020-03-11 10:15:00 +01:00
Chris Cranford
3a4788cc33
DBZ-1726 Introduced AbstractSnapshotChangeEventSource and refactor implementations
2020-03-11 10:15:00 +01:00
Chris Cranford
5506fadd2d
DBZ-1664 Verify correctness of JMX metrics
2020-03-09 23:30:24 +01:00
Jiri Pechanec
0248fdde16
DBZ-1850 Delegate signed/nonsigned conversion to driver
2020-03-09 13:06:30 +01:00
Jiri Pechanec
69e87170e3
DBZ-1849 Only log message was fixed
2020-03-09 07:43:18 +01:00
Jiri Pechanec
62e6a0add7
DBZ-1849 Avoid type refresh for unchanged root type
2020-03-06 18:01:56 +01:00
Jiri Pechanec
f99064fda1
DBZ-1824 Restart incomplete Tx for wal2json
2020-03-04 14:03:21 +01:00
Gunnar Morling
a8b796685b
DBZ-1840 Typo fix; commenting
2020-03-04 09:31:22 +01:00
Jiri Pechanec
10dfc12c99
DBZ-1840 Explanation for unresolvable tables
2020-03-04 09:25:57 +01:00
Jiri Pechanec
19d0a88cc9
DBZ-1840 Test logic was out of order
2020-03-04 09:25:57 +01:00
Jiri Pechanec
b2370c2d8e
DBZ-1840 Syntheitc message to replace messages dropped in decoder
2020-03-04 09:25:57 +01:00
Jiri Pechanec
6bab948e76
DBZ-1840 No nulls to stream producer
2020-03-04 09:25:57 +01:00
Jan-Hendrik Dolling
5cfbaab7a6
DBZ-1845 Typos in deployment.yaml and code readability
...
* improve readability
* fixed some typos
2020-03-04 08:34:55 +01:00
Gunnar Morling
c5422e6b31
DBZ-1840 Not passing null to replication message processor;
...
Note this means to that WAL offsets cannot be acknowledged in cases
where a null message is received from the server (e.g. wal2json for
DDL events). But as we cannot tell apart null events from the server
from readPending() simply having no events to read, the current
logic renders the pausing between two read calls without events
obsolete, so that's the lesser evil
2020-03-03 11:45:53 +01:00
Gunnar Morling
0f6d2c54fc
DBZ-1134 Avoiding concrete class in SPI by introducing receiver parameter
2020-02-28 21:58:39 +01:00
Jiri Pechanec
d13e48cc55
DBZ-1134 Make converted field type parametrized
2020-02-28 21:58:39 +01:00
Jiri Pechanec
06e1eca085
DBZ-1134 Support for user provided converters
2020-02-28 21:58:39 +01:00
Jiri Pechanec
91f292a1f2
DBZ-1825 Keep keys for non-matching table
2020-02-26 15:03:03 +01:00
Jiri Pechanec
ae6ee3d458
DBZ-234 Place Incubating into a unique package
2020-02-25 15:31:10 +01:00
Jiri Pechanec
2b6e50daff
DBZ-1760 Allow skipping of invalid events
2020-02-25 10:43:11 +01:00
jcechace
3209de3e5f
DBZ-924 Adding integration tests for OpenShift and Strimzi
2020-02-20 12:48:08 +01:00
Gunnar Morling
eff3ffc345
DBZ-1758 Documentation update
2020-02-14 13:14:25 +01:00
Debezium Builder
f75746fb7b
[maven-release-plugin] prepare for next development iteration
2020-02-13 03:18:20 -05:00
Debezium Builder
1d2df32743
[maven-release-plugin] prepare release v1.1.0.Beta2
2020-02-13 03:18:20 -05:00
Gunnar Morling
dd5c2b8be9
DBZ-1766 Unifying handling of offset commits
2020-02-12 15:44:45 +01:00
Melissa Winstanley
ef55c9157d
DBZ-1766 Update lastOffset to commit only after a record is committed
2020-02-12 15:44:45 +01:00
Jiri Pechanec
d4d38dcbec
DBZ-1783 Ignore empty transactions
2020-02-12 14:58:15 +01:00
Jiri Pechanec
40506d5352
[ci] Reduce postgres test logging for CI
2020-02-12 14:49:55 +01:00
Debezium Builder
9c424b2404
[maven-release-plugin] prepare for next development iteration
2020-02-05 08:54:52 -05:00
Debezium Builder
9f2caa3b7c
[maven-release-plugin] prepare release v1.1.0.Beta1
2020-02-05 08:54:52 -05:00
Jiri Pechanec
7c9d78e597
DBZ-1727 Fix unused imports
2020-02-05 13:33:29 +01:00
Igor Gabaydulin
3890038c21
DBZ-1727 Reduce heartbeat interval in test
2020-02-05 13:33:29 +01:00
Igor Gabaydulin
a2a6525c84
DBZ-1727 Update heartbeat on changing non whitelisted table test
2020-02-05 13:33:29 +01:00
Igor Gabaydulin
2f6e522b0f
DBZ-1727 Fix PostgresConnectorIT#shouldFlushLsnOnEmptyMessage test
2020-02-05 13:33:29 +01:00
Igor Gabaydulin
b62acfd144
DBZ-1727 Apply code formatter
2020-02-05 13:33:29 +01:00
Igor Gabaydulin
9bea1e2638
DBZ-1727 Move null check to AbstractMessageDecoder
2020-02-05 13:33:29 +01:00
Igor Gabaydulin
fcc7a05579
DBZ-1727 Debezium Postgres connector does not apply keepalive lsn
2020-02-05 13:33:29 +01:00
Jiri Pechanec
77f9a61b95
DBZ-1751 Centralize envelope schema naming
2020-02-04 20:37:19 +01:00
Chris Cranford
c5902c3c32
DBZ-1158 Add test for REPLICA IDENTITY fallback values
2020-02-04 10:37:12 -05:00
Jiri Pechanec
1d9ac71a9e
DBZ-1698 Fix test timing issue
2020-02-03 22:38:02 +01:00
Jiri Pechanec
2a6ea1c41f
DBZ-1698 Handle toasted unknown column values
2020-02-03 22:38:02 +01:00
Jiri Pechanec
34d2979123
DBZ-1052 Stabilize test for changed heartbeats
2020-02-03 09:29:10 +01:00
Jiri Pechanec
010e0c6fa6
DBZ-1052 Flush LSN for skipped tx messages
2020-02-03 09:29:10 +01:00
Gunnar Morling
b66f314b3c
DBZ-1052 Misc. clean-up
...
* Doc updates
* Changed static method to instance
2020-02-03 09:29:10 +01:00
Jiri Pechanec
67e0079646
DBZ-1052 Expand known protobuf messages list
2020-02-03 09:29:10 +01:00
Jiri Pechanec
4fdda6c101
DBZ-1052 Add tx enable option to connector definitions
2020-02-03 09:29:10 +01:00
Jiri Pechanec
ba10a229b6
DBZ-1052 CE support for tx metadata in data messages
2020-02-03 09:29:10 +01:00
Jiri Pechanec
0e671d6167
DBZ-1052 Add counts per table to END
2020-02-03 09:29:10 +01:00
Jiri Pechanec
60056b0a94
DBZ-1052 Transaction block moved to envelope
2020-02-03 09:29:10 +01:00
Jiri Pechanec
5749acda7e
DBZ-1052 Remove unused fields; added JavaDoc
2020-02-03 09:29:10 +01:00
Jiri Pechanec
072a37e790
DBZ-1052 Support for full Postgres/protobuf
2020-02-03 09:29:10 +01:00
Jiri Pechanec
c0557b325b
DBZ-1052 Skip CE for transaction struct
2020-02-03 09:29:10 +01:00
Jiri Pechanec
c466143f35
DBZ-1052 Test for limited Postgres/protobuf
2020-02-03 09:29:10 +01:00
Jiri Pechanec
fcafd2f193
DBZ-1052 Transaction metadata for Postgres/wal2json
2020-02-03 09:29:10 +01:00
Jiri Pechanec
cd6a1f942d
DBZ-1052 Transaction metadata for Postgres/pgoutput
2020-02-03 09:29:10 +01:00
Jiri Pechanec
b0eb571f9f
DBZ-1052 Transaction metadata for SQL Server
2020-02-03 09:29:10 +01:00
Jiri Pechanec
7283f5d6c1
DBZ-1746 Gracefully handle unknown protobuf messages
2020-01-30 12:24:52 +01:00
Marc Zbyszynski
507aac0e0f
DBZ-1738 replaced casts to PgConnection with casts to interface BaseConnection
...
DBZ-1738 replaced casts to PgConnection with casts to interface BaseConnection
2020-01-27 16:47:27 +01:00
Debezium Builder
274bba9dc9
[maven-release-plugin] prepare for next development iteration
2020-01-16 05:40:24 -05:00
Debezium Builder
076df30108
[maven-release-plugin] prepare release v1.1.0.Alpha1
2020-01-16 05:40:24 -05:00
Gunnar Morling
8c33a4e147
DBZ-1292 Catering for the differences in data schema;
...
* MongoDB doesn't have "before", but "patch"
* Also removing some redundancies and making RecordParser immutable
* Reducing log verbosity during tests
2020-01-16 08:01:56 +01:00
Gunnar Morling
1a18bdbc81
DBZ-1292 Misc. refactorings;
...
* Protecting against Avro as CE format and JSON as data format
* Reducing mutability
* Single Avro client
2020-01-16 08:00:45 +01:00
GraySmog
d0e4b3c0af
DBZ-1292 Delete data.schema.registry.url config option
2020-01-16 08:00:45 +01:00
GraySmog
915c8cafd9
DBZ-1292 CloudEvents converter for JSON and Avro format
2020-01-16 08:00:45 +01:00
Gunnar Morling
07e89de91d
DBZ-1707 Ensuring correct Kafka record timestamp for outbox events;
...
Depending on the width of the configured TIMESTAMP column, a microsecond or nanosecond timestamp could have been returned before, whereas this should always be milliseconds.
2020-01-15 07:32:55 +01:00
Chris Cranford
1d41d53636
DBZ-1685 Document and add support for column.mask.with for PostgreSQL connector
2020-01-14 09:30:34 +01:00
Chris Cranford
9fc1eada11
DBZ-1600 Remove deprecated "slot.drop_on_stop" postgres configuration option
2020-01-14 09:27:32 +01:00
Chris Cranford
cf60466b02
DBZ-1684 Make sure publication initialized prior to slot
...
This is to avoid the need to restart the connector when streaming begins due to
an error that the publication can't be found. It would seem based on testing
that since we create the replication slot early on when it does not exist, then
when streaming starts its confused about trying to marry a newly created
publication to an slot. By creating the publication before the slot, this
works around the problem.
2020-01-09 15:55:22 +01:00
Chris Cranford
3413d674fb
DBZ-1684 Create missing publication when replication slot exists
2020-01-07 15:41:15 +01:00
Debezium Builder
9723c93db9
[maven-release-plugin] prepare for next development iteration
2019-12-18 01:44:33 -05:00
Debezium Builder
09049516f2
[maven-release-plugin] prepare release v1.0.0.Final
2019-12-18 01:44:33 -05:00
Chris Cranford
5c6f8fdd70
DBZ-1680 Fix enum value resolution across all PostgreSQL decoders
2019-12-17 18:21:17 +01:00
Chris Cranford
debcb95fa2
DBZ-1677 Include root cause when PostgreSQL schema refresh fails during streaming
2019-12-17 07:19:02 +01:00
Gunnar Morling
d40641b3ab
DBZ-1666 Formatting
2019-12-12 16:39:10 +01:00
Gunnar Morling
4619f5eb9d
DBZ-1666 Wording fix
2019-12-12 11:56:05 +01:00
Jiri Pechanec
ffa15b100f
DBZ-1666 Warn for drop slot on stop
2019-12-12 11:52:30 +01:00
Debezium Builder
b1c62fe306
[maven-release-plugin] prepare for next development iteration
2019-12-10 06:39:17 -05:00
Debezium Builder
ad2efdb9a6
[maven-release-plugin] prepare release v1.0.0.CR1
2019-12-10 06:39:17 -05:00
Chris Cranford
c32b771e1b
DBZ-1413 Fix format violation
2019-12-10 10:09:35 +01:00
Chris Cranford
19fd8dd1d9
DBZ-1413 Resolve column length/scale when reading table column metadata from TypeRegistry
2019-12-10 10:09:35 +01:00
Chris Cranford
372d199a71
DBZ-1413 Revert AbstractColumnValue changes in commit 774f2ed
2019-12-10 10:09:35 +01:00
Chris Cranford
365eb0d76e
DBZ-1413 Suggested Javadoc changes, added/fix tests, changed how Column native/jdbc type resolution happens
2019-12-10 10:09:35 +01:00
Chris Cranford
9685990fb8
DBZ-920 Add support for PostgreSQL enum types
2019-12-10 10:09:35 +01:00
Chris Cranford
46de58ae75
DBZ-1413 Updated tests
2019-12-10 10:09:35 +01:00
Chris Cranford
ef9b4a7431
DBZ-1413 Delay building type hierarchy after all base types built
2019-12-10 10:09:35 +01:00
Chris Cranford
547ba8e0db
DBZ-1413 Align domain type support across all 3 decoders
2019-12-10 10:09:35 +01:00
Chris Cranford
9586be7d4e
DBZ-1413 Support PostgreSQL domain types
2019-12-10 10:09:35 +01:00
Thomas Deblock
17e44d4c0c
DBZ-1648 outbox EventRouter remove topic toLowerCase
...
- kafka topics names are case sensitive. We should not perform
a toLowerCase on the topic name. It can cause some issue if
topic name have a upper case
2019-12-04 21:39:25 +01:00
Jiri Pechanec
d471f2f6ab
DBZ-1637 Support for UUID array
2019-11-28 09:06:23 +01:00
Brandon Brown
a0f7f6ec6b
DBZ-1402 Support for JDK 11
2019-11-25 11:53:19 +01:00
Jiri Pechanec
a56ab78b3f
DBZ-825 Unify decimal upscaling
2019-11-20 18:23:55 +01:00
Debezium Builder
38279cc2dd
[maven-release-plugin] prepare for next development iteration
2019-11-14 02:56:07 -05:00