Chris Cranford
ada157d117
DBZ-4321 Introduce BeanRegistry
and ServiceRegistry
...
As a part of this work to handle injection in a cleaner way, this commit
adds two new broad concepts called `BeanRegistry` and `ServiceRegistry`.
A BeanRegistry is a glorified registry of different objects that are not
necessarily services but may be desired by a service. This contract will
allow Debezium to integrate in the future with other CDI providers.
A ServiceRegistry is more of an internal concept, where various systems
can be started based on their dependency order and provides a universal
way to split larger parts of the code into smaller, focused modules that
can be accessed using the Service Locator pattern.
2023-12-13 11:27:40 -05:00
Chris Cranford
26b3881941
DBZ-4321 Add MongoDB hook & polish injection handler
2023-12-13 11:27:40 -05:00
Chris Cranford
59027ed5ed
DBZ-4321 New PostProcessor
contract and Column Reselection
2023-12-13 11:27:40 -05:00
Chris Cranford
aea6cc003d
[ci] Use a relaxed property check for tests
2023-12-06 18:04:59 -05:00
mfvitale
874ab18017
DBZ-6834 Enable REPLICA IDENTITY on debezium_signal test table
2023-12-06 14:10:26 +01:00
Debezium Builder
0fd1c0dc9a
[maven-release-plugin] prepare for next development iteration
2023-12-04 13:55:35 +00:00
Debezium Builder
3e2d75f0da
[maven-release-plugin] prepare release v2.5.0.Beta1
2023-12-04 13:55:35 +00:00
Prabhu19
8b0487fb5c
DBZ-7193 Added testcases for tables toasted array columns with replica identity FULL.
2023-12-01 13:52:21 +01:00
Prabhu19
c60cbbd4aa
DBZ-7193 Unchanged toasted array columns are substituted with unavailable.value.placeholder, even when REPLICA IDENTITY FULL is configured.
...
This PR fixes the above issue.
2023-12-01 13:52:21 +01:00
Gunnar Morling
0ff6c730d6
DBZ-7181 Avoiding queries not available in recovery mode
2023-11-27 10:47:16 +01:00
rkerner
aeaba3ae70
DBZ-6762 Add "validate filters" endpoint/s for connector-specific Connect REST Extensions
...
closes to https://issues.redhat.com/browse/DBZ-6762
2023-11-22 06:10:40 +01:00
Debezium Builder
1521445908
[maven-release-plugin] prepare for next development iteration
2023-11-10 10:26:05 +00:00
Debezium Builder
6c6f6e9138
[maven-release-plugin] prepare release v2.5.0.Alpha2
2023-11-10 10:26:05 +00:00
Vojtech Juranek
9d26dd4cf4
DBZ-7007 Move embedded engine config into separate interface
...
This config will be re-used by possible other implementations of
DebeiumEngine API in the embedded package. As DebeziumEngine API
can have completely different implementations and thus also config,
the class is called `EmbeddedEngineConfig` as it's assumed to be used
only by embedded engine "family" of implementations.
To keep backward compatibility, the config options are extracted into
an interface and `EmbeddedEngine` implements this interface, thus
allowing to use these options in custom classes without any need for the
code changes.
2023-11-06 10:50:21 +01:00
Vojtech Juranek
e073dbd1be
DBZ-7007 Use DebeziumEngine API instead of EmbeddedEngine in postgres tests
2023-11-06 10:50:21 +01:00
Debezium Builder
78c5204444
[maven-release-plugin] prepare for next development iteration
2023-10-26 15:39:35 +00:00
Debezium Builder
356e488e83
[maven-release-plugin] prepare release v2.5.0.Alpha1
2023-10-26 15:39:35 +00:00
Roman Kudryashov
215f3098b7
DBZ-7016 Refactoring
2023-10-19 11:35:18 +02:00
Roman Kudryashov
ee64c70546
DBZ-7016 id
and type
of a CloudEvent can be retrieved from headers
2023-10-19 11:35:18 +02:00
Jiri Pechanec
899ac0d285
DBZ-7015 Hide the option in UI
2023-10-16 15:37:29 +02:00
Praveen Burgu
f9b07e6517
DBZ-7015 Make slot validation check opt-in
2023-10-16 15:37:29 +02:00
Praveen Burgu
7fd68a5a0d
DBZ-7015 Enable replication slot advance check
...
This reverts commit 925de0184c
.
2023-10-16 15:37:29 +02:00
Vojtech Juranek
d7e8943d4e
DBZ-6778 Initialize ElapsedTimeStrategy upon its creation
...
Currently, newly created `ElapsedTimeStrategy` is uninitialized and its
`hasElapsed()` has to be called once `ElapsedTimeStrategy` is created to
initialize the strategy. This is confusing and error prone.
Move initialization of `ElapsedTimeStrategy` into it's constructor, so
it's initialized once it's created.
2023-10-05 11:57:59 +02:00
Roman Kudryashov
01485aceb3
DBZ-6982 Provide configuration option to exclude extension attributes from a CloudEvent
2023-10-04 13:58:53 +02:00
Debezium Builder
40a8f4026d
[maven-release-plugin] prepare for next development iteration
2023-10-03 14:18:54 +00:00
Debezium Builder
412c6f7979
[maven-release-plugin] prepare release v2.4.0.Final
2023-10-03 14:18:54 +00:00
Vojtech Juranek
496495bc1c
DBZ-6987 Create schema in Postgres test if not exists
2023-10-03 07:52:18 +02:00
Vojtěch Juránek
2776814cd5
DBZ-6986 Provide more clear reason why the test is skipped
...
Co-authored-by: Fiore Mario Vitale <mvitale86@gmail.com>
2023-10-03 07:49:08 +02:00
Vojtech Juranek
5abe3f2e52
DBZ-6986 Skip RecordsStreamProducerIT#shouldReceiveChangesForInfinityNumericWithInfinity on Postgres < 14
...
Infinity support for numeric types was added in Postgres 14, see
https://www.postgresql.org/docs/release/14.0/
2023-10-03 07:49:08 +02:00
Vojtech Juranek
c31aeef854
DBZ-6216 Use quay.io for base postgres image
2023-10-03 07:49:08 +02:00
Roman Kudryashov
98c426dec4
DBZ-3642 CloudEventsConverter can retrieve metadata info from headers
2023-09-26 13:06:09 +02:00
Debezium Builder
cc1de0aa51
[maven-release-plugin] prepare for next development iteration
2023-09-22 09:00:45 +00:00
Debezium Builder
6809978083
[maven-release-plugin] prepare release v2.4.0.CR1
2023-09-22 09:00:45 +00:00
mfvitale
5d7ba3e951
DBZ-6957 Remove spaces in Signal and Notification JMX ObjectName
2023-09-21 12:04:33 -04:00
rkerner
6d3528d4fb
DBZ-4395 Add connector specific Debezium Connect REST Extension/s and move logic from UI backend to the Debezium Connect REST Extension (part 2 of the re-architecturing/re-factoring)
...
DBZ-6761 Add "validate connection" endpoints in connector specific Connect REST extensions
closes to https://issues.redhat.com/browse/DBZ-4395
closes to https://issues.redhat.com/browse/DBZ-6761
2023-09-21 13:29:59 +02:00
mfvitale
ae199d2053
DBZ-6828 Blocking snapshot will generate schema events only for signaled tables
2023-09-20 08:18:33 +02:00
Jiri Pechanec
29e45b0ce0
DBZ-6911 Support for PostgreSQL 16
2023-09-18 15:33:11 +02:00
xiaowu
e50d0eb87a
DBZ-6935 fix logger named
2023-09-18 12:34:47 +02:00
Debezium Builder
65169a9315
[maven-release-plugin] prepare for next development iteration
2023-09-13 09:41:46 +00:00
Debezium Builder
845e3e9d09
[maven-release-plugin] prepare release v2.4.0.Beta2
2023-09-13 09:41:46 +00:00
Jiri Pechanec
f1e7999431
DBZ-6481 Support Postgres enum as key in incremental snapshot
2023-09-11 06:08:57 +02:00
Debezium Builder
d97a18e8dc
[maven-release-plugin] prepare for next development iteration
2023-08-29 11:41:08 +00:00
Debezium Builder
5ce25f2cfe
[maven-release-plugin] prepare release v2.4.0.Beta1
2023-08-29 11:41:08 +00:00
Jiri Pechanec
dac07d53c3
DBZ-6482 TimescaleDB transformation
2023-08-29 11:45:27 +02:00
Vojtech Juranek
28b06e6795
DBZ-6839 Fix PostgresConnectorIT#shouldAddNewFieldToSourceInfo
...
PostgresConnectorIT#shouldAddNewFieldToSourceInfo fails only when run
together with other tests and the failure is random. It seem there is
a caching issue in Apicuro, when `test_server.s1.a-value` artifact
references `io.debezium.connector.postgresql.Source` version 1, which
hasnt't `newField` field and this reference is used also in
`shouldAddNewFieldToSourceInfo` where artifact with version 2 should
be used. Using dedicated table and thus creating new artifact in
Apucurio should fix this issue.
Also remove unused variable from `CustomPostgresSourceInfoStructMaker`.
2023-08-25 15:02:18 +02:00
mfvitale
542b3619fa
DBZ-6731 Blocking snapshot takes configuration from signal payload
2023-08-15 12:43:43 +02:00
Vojtech Juranek
6ca842b772
DBZ-6635 Send heartbeats also before processing first event
...
We should send heartbeats no matter if there is already any DB record
processed or not. This prevents situation when after the start there is
no new record in the Db and Debezium is not sending neither DB events
nor heartbeats.
2023-08-14 18:45:43 +02:00
harveyyue
76738a6402
DBZ-6758 Add it test for (+/-) infinity values for numeric/decimal types
2023-08-14 09:04:33 +02:00
harveyyue
15e81c1a84
DBZ-6758 When using pgoutput in postgres connector, (+/-)Infinity is not supported in decimal values
2023-08-14 09:04:33 +02:00
rajdangwal
b3098de0fd
DBZ-6720: minor nit
2023-08-14 08:59:54 +02:00
rajdangwal
153834c650
DBZ-6720: handle unchanged toasted uuid array
2023-08-14 08:59:54 +02:00
Debezium Builder
3c72f071b0
[maven-release-plugin] prepare for next development iteration
2023-08-09 10:33:06 +00:00
Debezium Builder
915c465440
[maven-release-plugin] prepare release v2.4.0.Alpha2
2023-08-09 10:33:06 +00:00
harveyyue
b16d991a25
DBZ-6684 Rename back fields with avro_unicode
2023-08-08 10:22:10 +02:00
Jiri Pechanec
a8664ae3e9
DBZ-6767 Create publication for pgoutput only
2023-08-08 08:18:17 +02:00
Jiri Pechanec
3d263d645e
DBZ-6767 Provide bytea suport
2023-08-08 08:18:17 +02:00
Jiri Pechanec
3aa6759ebd
DBZ-6767 Add missing type name
2023-08-08 08:18:17 +02:00
mfvitale
61651a94b5
DBZ-6566 Force blocking snapshot even if a snapshot has already executed
2023-08-03 09:22:24 +02:00
mfvitale
def7379a0d
DBZ-6566 Support Blocking snapshot for MySQL
2023-08-03 09:22:24 +02:00
mfvitale
6e0b323e83
DBZ-6566 Set correct log context during blocking snapshot
2023-08-03 09:22:24 +02:00
mfvitale
70f16af7c4
DBZ-6566 Support Blocking snapshot for PostgreSQL
2023-08-03 09:22:24 +02:00
jcechace
699a1612ae
DBZ-6670 Replace error handler instead of recreate in order to keep retry counter
2023-07-25 13:46:56 +02:00
mfvitale
8ea2b3c5b7
DBZ-6669 Snapshot will get data when only signal data collection table is specified
2023-07-24 10:02:13 -04:00
Debezium Builder
267dfda8a6
[maven-release-plugin] prepare for next development iteration
2023-07-17 10:40:52 +00:00
Debezium Builder
0d3cef6a29
[maven-release-plugin] prepare release v2.4.0.Alpha1
2023-07-17 10:40:52 +00:00
Jiri Pechanec
4ca2a1c5d0
DBZ-6517 Use correct constructor
2023-07-14 09:11:10 +02:00
jcechace
318c55cc74
DBZ-6659 Fixing junit dependencies
2023-07-14 09:11:10 +02:00
Vojtech Juranek
5cf61444c0
DBZ-6648 Use String fallback when interval handling mode is String
...
The root cause of DBZ-6648 is using long number as a fallback value,
while String is expected and thus falling conversion and returning
`null`.
Use String fallback when interval handling mode is String.
2023-07-14 06:03:59 +02:00
Jiri Pechanec
764427081b
DBZ-6517 DBZ-6654 Make CE converter compatible with Kafka 3.5
2023-07-13 11:26:21 +02:00
harveyyue
58d3e150b9
DBZ-6603 Add integration test for custom metric tags
2023-07-13 09:52:33 +02:00
harveyyue
5bc7fd1f7c
DBZ-6603 Support for custom tags in the connector metrics
2023-07-13 09:52:33 +02:00
jcechace
5cb579bf12
DBZ-6525 Removing runtime dependency on core for testcontainers module in order to prevent logback.xml downstream conflicts
2023-07-11 12:07:04 +02:00
mfvitale
0baad2f7b7
DBZ-6417 Add test for custom action
2023-06-23 15:03:27 +02:00
mfvitale
d5febcc054
DBZ-6417 Provide extensible signal action mechanism through SPI
2023-06-23 15:03:27 +02:00
mfvitale
9ed928a1c9
DBZ-6416 Improve tests stability
2023-06-22 13:22:34 +02:00
mfvitale
34e28ac52d
DBZ-6416 Add initial snapshot notifications
2023-06-22 13:22:34 +02:00
Anatolii Popov
b207f96320
DBZ-6577 Don't try to alter publication created for all tables when configuration changes to 'filtered'
2023-06-21 15:58:11 +02:00
Debezium Builder
60f730d383
[maven-release-plugin] prepare for next development iteration
2023-06-20 13:21:31 +00:00
Debezium Builder
150396e4c8
[maven-release-plugin] prepare release v2.3.0.Final
2023-06-20 13:21:31 +00:00
Jiri Pechanec
067683e56a
DBZ-6076 Consistent config for all connectors
2023-06-15 11:55:28 +02:00
Anil Dasari
0431a8b9fc
DBZ-6076: Add support for custom SourceInfoStructMaker for adding new fields to source field
2023-06-15 11:55:28 +02:00
Vojtech Juranek
78668df5c1
DBZ-6547 Declare exactly-once semantic support for Postgres connector
2023-06-12 12:44:12 +02:00
Debezium Builder
5d1b00588a
[maven-release-plugin] prepare for next development iteration
2023-06-09 09:23:38 +00:00
Debezium Builder
d1e6107517
[maven-release-plugin] prepare release v2.3.0.CR1
2023-06-09 09:23:37 +00:00
mfvitale
7338fde84e
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
...
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
2023-06-07 07:36:58 +02:00
mfvitale
769d9361e7
DBZ-6424 Add JMX Notification channel
...
This channel permits to send Debezium notification to JMX bean
DBZ-6424 Add JMX Signal channel
This channel permits to send signal to Debezium through the JMX operation
2023-06-07 07:36:58 +02:00
mfvitale
537cc50d65
DBZ-6524 Add support for ltree
and isbn
PostgreSQL data-types in case INCLUDE_UNKNOWN_DATATYPES
is set to false
2023-06-06 06:31:00 +02:00
Ronak Jain
3f3967c2bb
DBZ-6366: Code Improvements for skip.messages.without.change
...
Addresses/Closes DBZ-6366
2023-06-05 14:50:25 +02:00
Debezium Builder
aa5b2131fc
[maven-release-plugin] prepare for next development iteration
2023-05-26 10:35:20 +00:00
Debezium Builder
9cc9b3c29a
[maven-release-plugin] prepare release v2.3.0.Beta1
2023-05-26 10:35:20 +00:00
Debezium Builder
a5b44767bc
[maven-release-plugin] prepare for next development iteration
2023-05-11 11:05:24 +00:00
Debezium Builder
1b1a7531f2
[maven-release-plugin] prepare release v2.3.0.Alpha1
2023-05-11 11:05:24 +00:00
mfvitale
7429b4e6f1
DBZ-1973 Add incremental snapshot notifications
2023-05-10 12:22:02 +02:00
mfvitale
811b800f55
DBZ-1973 Add support for Notifications
...
DBZ-1973 Add more tests for other connectors
DBZ-1973 Add send method with offset parameter
DBZ-1973 Instantiate NotificationService in the task class
DBZ-1973 Instantiate NotificationService in the task class
DBZ-4027 Move specific sink channel configuration to SinkNotificationChannel
DBZ-4027 Remove not used SPI file
DBZ-1973 Moved SPI file definition to debezium-core
DBZ-1973 Rename KafkaNotificationChannel to more generic SinkNotificationChannel
DBZ-1973 Code refactor
DBZ-1973 Improve configuration property description
DBZ-1973 Improve test
DBZ-1973 Add close method to NotificationChannel
DBZ-1973 Implement KafkaNotificationChannel
DBZ-1973 Add NotificationService and LogNotificationChannel
2023-05-10 12:22:02 +02:00
mfvitale
0ca87d8031
DBZ-4027 During event dispatch process only signal from source channel
2023-05-05 13:35:40 +02:00
mfvitale
952340286e
DBZ-4027 Make signalling channel configurable
...
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Code style
DBZ-4027 Make SignalPayload more generic and extensible
DBZ-4027 Rename DatabaseSignalChannel to SourceSignalChannel
DBZ-4027 Improve logging
DBZ-4027 Moved SPI file definition to debezium-core
DBZ-4027 Move SignalProcessor synchronization point to be processed only when a signal cdc event arrives.
DBZ-4027 Add EventDispatcher constructor without signalProcessor for spanner connector
DBZ-4027 Fix NPE
DBZ-4027 Fix NPE
DBZ-4027 Formatting
DBZ-4027 Correctly manage signal on not supported connector
DBZ-4027 Use the correct MongoDbOffset
DBZ-4027 Correctly initialize offset for Oracle and SqlServer connectors
DBZ-4027 Register SPI implementations
DBZ-4027 Improve SignalProcessor instantiation
DBZ-4027 Pass source info in case of SchemaChanges action
DBZ-4027 Manage close event in a synchronous way
DBZ-4027 Correctly init offset context also in case of snapshot mode 'never'
DBZ-4027 Fix MySqlMetricsIT test
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Set pass offset context after initial snapshot to SignalProcessor
DBZ-4027 Pass OffsetContext to signal processor
DBZ-4027 Pass CommonConnectorConfig to SignalChannelReader init method
DBZ-4027 Move Incremental snapshot window actions to dedicated package
DBZ-4027 Align SignalsIT test with new code
DBZ-4027 Fix SignalsIT test
DBZ-4027 Fix SignalProcessor scheduling
DBZ-4027 Moved DatabaseSignalChannel and SignalChannelReader to dedicated package
DBZ-4027 Start SignalProcessor from ChangeEventSourceCoordinator
DBZ-4027 Create SignalProcessor and renamed Signal to DatabaseSignalChannel
DBZ-4027 Initial refactoring of signal feature
2023-05-05 13:35:40 +02:00
Jiri Pechanec
ab5d643aa8
DBZ-6353 Use correct SQL codes for other cases
2023-05-03 12:15:18 +02:00
umachi
6ced91fe53
DBZ-6353 Add a conditions of 'OBJECT_IN_USE', 'NOT_IMPLEMENTED'.
2023-05-03 12:15:18 +02:00
miguelbirdie
63cc36e9a2
DBZ-6112 Set Replica Identity for PostgreSQL tables
...
Introduces a new PostgreSQL configuration parameter called `replica.autoset.type`. With this parameter, you can easily specify the Replica Identity value for each table that is captured by the connector
2023-05-02 14:33:46 +02:00
ani-sha
a453c3ca19
DBZ-6259 Add missing description of various snapshot modes in the connector field definitions
2023-05-01 11:03:04 -04:00
Nir Levy
34aa5ed8b9
DBZ-6396 Honor slot.max.retries when committing LSN before starting replication
2023-04-28 15:01:26 +02:00
Frederic Laurent
5139957f91
DBZ-6340 Add support to the Postgres connector for allow and prefer SSL modes (with prefer as the new default value)
2023-04-26 12:48:54 +02:00
Bertrand Paquet
82cf55e3ec
DBZ-6384: Fix snapshotting with hstore in Map mode
2023-04-26 12:43:21 +02:00
Jiri Pechanec
643a82e4ee
DBZ-6122 Refactor for more safety
2023-04-24 07:06:42 +02:00
Bertrand Paquet
979d4529b7
DBZ-6122 Toasted varying character array and date array are not correctly processed
2023-04-24 07:06:42 +02:00
Jiri Pechanec
28563b7244
DBZ-6353 Use SQL state to identify missing function
2023-04-24 06:36:58 +02:00
Jiri Pechanec
07947e1540
DBZ-6379 Fix imports
2023-04-24 06:03:56 +02:00
Bertrand Paquet
8a224e6588
DBZ-6379: Fix handling of toasted hstore
2023-04-24 06:03:56 +02:00
Debezium Builder
bb1b79f3bd
[maven-release-plugin] prepare for next development iteration
2023-04-20 09:09:43 +02:00
Debezium Builder
5483b4de99
[maven-release-plugin] prepare release v2.2.0.Final
2023-04-20 09:09:43 +02:00
Mark Bereznitsky
83629089a0
DBZ-6265 always check the WAL_LEVEL but fail if streaming is required
2023-04-19 10:54:39 +02:00
Jiri Pechanec
f4c200ac3e
DBZ-2979 Fix tests
2023-04-19 09:04:29 +02:00
Ronak Jain
465b355dd0
DBZ-2979 If column.include.list/column.exclude.list are used and the target table receives an update for the excluded (or not included) column - such events should be ignored
...
* Add More Test Cases
Addresses/Closes DBZ-2979
2023-04-19 09:04:29 +02:00
Ronak Jain
762122ecf5
DBZ-2979 If column.include.list/column.exclude.list are used and the target table receives an update for the excluded (or not included) column - such events should be ignored
...
* Add Mysql and SqlServer Connector Changes
* Change Docs
* Clean Test Case
* Formatting and Checkstyle
Addresses/Closes DBZ-2979
2023-04-19 09:04:29 +02:00
Debezium Builder
976a47a3a3
[maven-release-plugin] prepare for next development iteration
2023-04-14 08:46:06 +00:00
Debezium Builder
1ca8399e8b
[maven-release-plugin] prepare release v2.2.0.CR1
2023-04-14 08:46:06 +00:00
Jiri Pechanec
925de0184c
DBZ-6191 Do not fail if slot cannot be advanced
2023-04-14 07:44:16 +02:00
Vojtech Juranek
44f2f40c56
DBZ-6216 Update testcontainers to use quay.io images
...
When needed update also version.
Update also leftovers in readme files.
2023-04-12 14:07:35 +02:00
Jiri Pechanec
75e680cfb0
DBZ-6265 Correctly format error message
2023-04-04 07:05:55 +02:00
Mark Bereznitsky
6b82ee67e1
DBZ-6265 warn in case no wal support and no cdc
2023-04-04 07:05:55 +02:00
Mark Bereznitsky
4fea0cc895
DBZ-6265 skip the wal level check if no cdc is to occur
2023-04-04 07:05:55 +02:00
Jiri Pechanec
77c971880c
DBZ-6261 Allow different version for protoc
2023-04-04 05:40:27 +02:00
Debezium Builder
5c650fd054
[maven-release-plugin] prepare for next development iteration
2023-03-31 10:00:16 +00:00
Debezium Builder
68ffc4833b
[maven-release-plugin] prepare release v2.2.0.Beta1
2023-03-31 10:00:16 +00:00
mfvitale
673a42b541
DBZ-6232 Support 'binary.handling.mode' also on bytea array
2023-03-27 15:55:38 +02:00
mfvitale
ed67ff6eb9
DBZ-6232 Manage BYTEA_ARRAY in PostgresValueConverter
2023-03-27 15:55:38 +02:00
Anatolii Popov
930d9b0971
DBZ-6251 Ignore Debezium internal transactions during fallback to restart_lsn in Postgres connector if confirmed_flush_lsn is NULL
2023-03-27 15:55:13 +02:00
Nir Levy
c7e00d3881
DBZ-6249 Fix error thrown when snapshot.custom_class=custom and no snapshot.custom.class
2023-03-27 10:53:39 +02:00
Vojtech Juranek
cfca3e41a3
DBZ-6216 Update postgres notes to use quay.io images
...
Also update postgres version currently being used
2023-03-23 13:29:19 +01:00
Jiri Pechanec
d4c3e23767
DBZ-6092 Simplify the code; change logging
2023-03-16 12:58:03 +01:00
Anatolii Popov
cbb785d67a
DBZ-6092 Fallback to restart_lsn in Postgres connector if confirmed_flush_lsn is NULL and no long-running transactions were committed during timeout
2023-03-16 12:58:03 +01:00
Vojtěch Juránek
42b374613a
DBZ-6204 Fix typo
...
Co-authored-by: Jiri Pechanec <jpechane@redhat.com>
2023-03-16 11:32:52 +01:00
Vojtech Juranek
3b5740f1a8
DBZ-6204 Avoid skipping record when determining resume LSN
...
When the last operation before connector stop is tx BEGIN, subsequent
INSERT done after connector stop can have same LSN as BEGIN and thus it
was possible to skip this INSERT when we search for the resume LSN. This
issue was described and fixed in DBZ-5915. However, the same LSN can
have also previous tx COMMIT, so the WAL can look like this:
LSN | operation
------------------
123 | COMMIT
123 | BEGIN
123 | INSERT
124 | COMMIT
When the last stored LSN is 123 for COMMIT operation, we skip also BEGIN
and INSERT operations with LSN 123 and resume from COMMIT with LSN 124.
Adjust condition introduced in DBZ-5915 to consider also the case when
the last processed operation is COMMIT.
2023-03-16 11:32:52 +01:00
Debezium Builder
55d65e9618
[maven-release-plugin] prepare for next development iteration
2023-03-08 11:38:17 +00:00
Debezium Builder
6d183c3dc1
[maven-release-plugin] prepare release v2.2.0.Alpha3
2023-03-08 11:38:16 +00:00
Jiri Pechanec
3da2434c6c
DBZ-823 Restrict the type with generics
2023-03-03 12:47:15 +01:00
Jiri Pechanec
5e666f00ee
DBZ-823 Add JavaDoc to new interfaces
2023-03-03 12:47:15 +01:00
Jiri Pechanec
7d92dc0211
DBZ-823 Don't use getter based name
2023-03-03 12:47:15 +01:00
Jiri Pechanec
0f4adb9fe6
DBZ-823 Use better name for CF with main connection
2023-03-03 12:47:15 +01:00
ggaborg
e875e784a0
DBZ-823 CR changes
2023-03-03 12:47:15 +01:00
ggaborg
065dc997b9
DBZ-823 Optionally parallelize initial snapshots
2023-03-03 12:47:15 +01:00
Chris Cranford
a9d4663456
DBZ-6163 Properly handle zoned timestamp within PostgreSQL
2023-02-28 13:32:57 +01:00
Debezium Builder
2245ceda44
[maven-release-plugin] prepare for next development iteration
2023-02-16 11:28:25 +00:00
Debezium Builder
20f3b16e8a
[maven-release-plugin] prepare release v2.2.0.Alpha2
2023-02-16 11:28:25 +00:00
Yang Wu
d7ab7146f0
DBZ-6089 Expose sequence in postgres parser
2023-02-08 10:13:19 +01:00
harveyyue
57b902ff8c
DBZ-5743 Introduce alternative AVRO_UNICODE mode for field name selector
2023-02-02 13:21:45 +01:00
Jiri Pechanec
8f3ca36e23
DBZ-6026 Don't update last offsets in statistics
2023-01-24 14:26:05 +01:00
Vojtech Juranek
f00f2a8735
DBZ-5917 Fix querying column metadata for tables with back slash in the name
...
If the table contains back slash, which is ANSI SQL escape chracter,
in its name, querying column metadata would fail in some cases (*)
which would result into NPE. Fix table name before the query and
escape escape character if the table name contains it.
Default backslash works for all currently supported databases as it's
ANSI SQL standard,
(*) everything works when we collect column metadata for all tables,
i.e. when there are no excluded tables
2023-01-20 00:01:11 +01:00
Debezium Builder
04f5291a2d
[maven-release-plugin] prepare for next development iteration
2023-01-19 11:32:10 +00:00
Debezium Builder
027527963e
[maven-release-plugin] prepare release v2.2.0.Alpha1
2023-01-19 11:32:10 +00:00
Anil Dasari
e1f9d64fbb
DBZ-5852:Snapshotter#snapshotCompleted is invoked regardless of snapshot result
2023-01-19 09:24:02 +01:00
Jeremy Ford
8da86b78e8
DBZ-5879 Fix unit tests
2023-01-18 14:46:14 +01:00
Jeremy Ford
f2a6f0b8db
DBZ-5879 Support retrying database connection failures during connector start
...
Reworked when task requests start from subclasses
Add support for restarting
2023-01-18 14:46:14 +01:00
Jiri Pechanec
1ee4850273
DBZ-6012 Allow skipping of LSN check failure
2023-01-18 10:34:56 +01:00
Jiri Pechanec
107f09a2e7
DBZ-6001 DBZ-5991 Use strict class for money
2023-01-17 14:52:03 +01:00
harveyyue
93b82ab7b4
DBZ-6001 Postgresql connector parses the null of the money type into 0
2023-01-17 14:52:03 +01:00
harveyyue
bde7af1423
DBZ-5991 There is a problem with postgresql connector parsing the boundary value of money type
2023-01-17 14:52:03 +01:00
Jiri Pechanec
bcd6787cdc
DBZ-6008 Revert "DBZ-3653 Introduce incremental.snapshot.isolation.mode config"
...
This reverts commit 6d8bcd42fd
.
2023-01-17 10:40:20 +01:00
Animesh Kumar
90cd3fdb6d
DBZ-5988 Impose null check on schema and name values prior to applying lowercase operation
2023-01-11 09:04:31 +01:00
govi20
6d8bcd42fd
DBZ-3653 Introduce incremental.snapshot.isolation.mode config
2023-01-11 08:55:33 +01:00
Jiri Pechanec
8ca5e0a9b5
DBZ-4669 Reformat after plugin version consolidation
2023-01-11 06:44:53 +01:00
Mark Lambert
e5d16b14a6
DBZ-2439 Remove redundant modifiers
2023-01-11 06:44:53 +01:00
erdinc
5a063867df
DBZ-5976 Use as much as possible "replace" method rather than "replaceAll"
2023-01-05 07:30:49 +01:00
rajdangwal
b03b4cdd19
DBZ-5915: correctly determine the startStreamingLsn with messageType
2023-01-04 14:50:14 +01:00
rajdangwal
5dde1ab3b5
DBZ-5915: correctly determine the startStreamingLsn
2023-01-04 14:50:14 +01:00
Debezium Builder
cf1720e4cb
[maven-release-plugin] prepare for next development iteration
2022-12-22 10:11:54 +00:00
Debezium Builder
b4ea9b46cd
[maven-release-plugin] prepare release v2.1.0.Final
2022-12-22 10:11:53 +00:00
jchipmunk
189f3c023e
DBZ-5949 Support partitioned tables when updating PG publication
...
PostgresConnection.getAllTableIds(String) method was added to unify getting all tables in database catalog, including partitioned tables.
2022-12-20 09:28:05 +01:00
Praveen Burgu
31745a7dd5
DBZ-5936 Add test for json[] column
2022-12-20 09:18:34 +01:00
Praveen Burgu
48e1db8fc3
DBZ-5936 Add Javadoc
2022-12-20 09:18:34 +01:00
Jiri Pechanec
fff4719fb1
DBZ-5936 Remove separate config option
2022-12-20 09:18:34 +01:00
Praveen Burgu
4cada81661
DBZ-5936 Handle json and jsonb arrays
2022-12-20 09:18:34 +01:00
Praveen Burgu
28d12a836f
DBZ-5936 Move all placeholder logic into one class
2022-12-20 09:18:34 +01:00
Praveen Burgu
54c0cfc14a
DBZ-5936 Handle TOASTed int and bigint array columns
2022-12-20 09:18:34 +01:00
Debezium Builder
177a60a9d3
[maven-release-plugin] prepare for next development iteration
2022-12-16 09:33:38 +00:00
Debezium Builder
f805db8c3c
[maven-release-plugin] prepare release v2.1.0.Beta1
2022-12-16 09:33:38 +00:00
Praveen Burgu
0b4769c793
DBZ-5944 Toasted varchar array is not correctly processed
2022-12-16 08:50:32 +01:00
Praveen Burgu
b4be663168
DBZ-4941 add a comment
2022-12-14 14:26:17 +01:00
Praveen Burgu
e61a75ff4e
DBZ-4941 Create a separate table for test
2022-12-14 14:26:17 +01:00
Praveen Burgu
4bdcd66d48
DBZ-4941 Handle Toasted array columns
2022-12-14 14:26:17 +01:00
xiaowu
08119e5368
Clean up useless methods
2022-12-14 08:34:44 +01:00
Vojtech Juranek
9494552a3a
DBZ-5838 Add missing Apicurio converter dependency
2022-12-12 14:44:05 +01:00
Vojtech Juranek
68b624cf7b
DBZ-5811 DBZ-5914 Don't manipulate with LSN when LSN flushing LSN is disabled
...
Befre we start streamig, we adjust `confirmed_flush_lsn` in Postgres to
the position where we determined that the streaming should start. When
user disables LSN flushing, which is not desired as it's the user
reponsibility to flush LSN in Postgres.
Also adjust related tests, as when LSN flushing is not disabled, flushed
LSN in Postgres may change (as we e.g. determined start of streaming
from offset or from `xlogpos`) and can result in the test failure. When
flushing is enabled, only LSNs after start of streaming can be compared.
2022-12-09 21:03:19 +01:00
Jiri Pechanec
7f05ef44f9
DBZ-5895 Remove hard-coded plugin
2022-12-09 21:03:19 +01:00
Jiri Pechanec
67ff871569
DBZ-5895 Wait for REPLICA IDENTITY set completed
2022-12-09 21:03:19 +01:00
Jiri Pechanec
f437788325
DBZ-5895 Remove race condition between table reading and re-creating
2022-12-09 21:03:19 +01:00
Marcelo Avancini
7754163c5d
DBZ-2432: Fixing misuse of assertj
...
- Some usages of assertj was not callig the actual assert of the DSL, fixing it
- Changing the Assertions to be staticlly imported
2022-12-08 13:10:26 +01:00
Vojtech Juranek
e1b65cd6f7
DBZ-5889 Run integration tests in specified order
...
Run the tests alwyas in thr same order to make it more easy to debug
failures. If needed, the order can be changed (e.g. to `random`) by
overriding propeperty `runOrder`.
2022-12-08 13:00:41 +01:00
Jiri Pechanec
574c1b0015
DBZ-5811 Use validate method
2022-12-06 13:35:52 +01:00
subodh
66fb51b55a
DBZ-5811 Implement support for skipping LSN commit on postgres source based on config
2022-12-06 13:35:52 +01:00
Debezium Builder
81896ed334
[maven-release-plugin] prepare for next development iteration
2022-11-30 07:52:24 +00:00
Debezium Builder
1aac97b87c
[maven-release-plugin] prepare release v2.1.0.Alpha2
2022-11-30 07:52:23 +00:00
Praveen Burgu
2158ea22d4
DBZ-5739 Pass the exception as argument to the logger
2022-11-30 06:55:35 +01:00
Praveen Burgu
ad6ec2be6a
DBZ-5739 Handle permission error from RDS Postgres
2022-11-30 06:55:35 +01:00
Jiri Pechanec
5eb147129d
DBZ-5874 Fix syntax errors
2022-11-25 11:43:49 +01:00
Gunnar Morling
e7aba47a24
DBZ-5874 Making sure trace message gets logged
2022-11-25 11:43:49 +01:00
Praveen Burgu
2b73ad6216
DBZ-5739 Seek to LSN based on lastCommitStoredLsn
2022-11-22 14:50:11 +01:00
Praveen Burgu
00321ccb4d
DBZ-5739 Fix remaining failing tests
2022-11-22 14:50:11 +01:00
Praveen Burgu
cb535d2ce6
DBZ-5739 Fix another test with expected error message
2022-11-22 14:50:11 +01:00
Praveen Burgu
8e3867f23f
DBZ-5739 Fix other failing tests
2022-11-22 14:50:11 +01:00