Commit Graph

1474 Commits

Author SHA1 Message Date
Debezium Builder
65d63ed42d [maven-release-plugin] prepare for next development iteration 2024-01-21 10:12:45 +00:00
Debezium Builder
485fa82a8f [maven-release-plugin] prepare release v2.6.0.Alpha1 2024-01-21 10:12:44 +00:00
Jiri Pechanec
2d1ce13d7c DBZ-7316 Formatting and cotnributor 2024-01-16 11:00:04 +01:00
James Johnston
20fbbd6452 DBZ-7316 Send heartbeats while searching for WAL position
If the logical replication slot has absolutely zero events in it, then
the searchWalPosition could be looping for quite some time.  During
this time, it wasn't sending heartbeats.

This commit fixes that function to send heartbeats.
2024-01-16 11:00:04 +01:00
mfvitale
5d088cffca DBZ-7312 On demand blocking snapshot will not set transaction snapshot 2024-01-15 14:29:31 +01:00
mfvitale
20fbe73bc8 DBZ-7311 Use a more meaningful name for identify an on demand blocking snapshot 2024-01-08 12:52:03 +01:00
mfvitale
47cbdee526 DBZ-7311 Permits to execute a blocking snapshot even if snapshot.mode=never 2024-01-08 12:52:03 +01:00
Mickael Maison
70e7186868 DBZ-7298 Add service loader manifests for all Connect plugins
Add manifests for source/sink connectors, converters and transformations.
2024-01-04 09:22:29 +01:00
Debezium Builder
3853d20f44 [maven-release-plugin] prepare for next development iteration 2023-12-21 06:52:01 +00:00
Debezium Builder
5d35e9caaa [maven-release-plugin] prepare release v2.5.0.Final 2023-12-21 06:52:01 +00:00
ani-sha
8bb7869c7e DBZ-7177 DBZ-6764 Remove unused loggers, sort imports 2023-12-20 08:56:56 +01:00
ani-sha
4b36ed9975 DBZ-7177 DBZ-6764 Use correct context name and add suggestions for code review 2023-12-20 08:56:56 +01:00
ani-sha
5a27d42257 DBZ-7177 Add isStreaming check for all tests, remove unused files 2023-12-20 08:56:56 +01:00
ani-sha
a12b1a9872 DBZ-6764 Connector metrics implementation for REST extension 2023-12-20 08:56:56 +01:00
ani-sha
e535e5b78d DBZ-7177 Add suggestions from code review, adjust code style changes 2023-12-20 08:56:56 +01:00
ani-sha
2555a1cee2 DBZ-6764 DBZ-7178 DBZ-7177 Use MBeanServer to query metrics for REST endpoint and create new metrics descriptor 2023-12-20 08:56:56 +01:00
ani-sha
2165d9bed1 DBZ-6764 Connector metrics implementation for REST extension 2023-12-20 08:56:56 +01:00
Roman Kudryashov
516aa87fad DBZ-7235 Add option to customize CloudEvents schema name 2023-12-20 06:53:37 +01:00
Jiri Pechanec
5a10199e12 DBZ-7251 Enable fitering of PostgreSQL connector 2023-12-19 17:29:37 +01:00
harveyyue
225277c5c0 DBZ-7251 Return generic DataCollectionId instead of REST response DataCollection in getMatchingCollections method 2023-12-19 12:53:57 +01:00
harveyyue
4719396ffc DBZ-7145 Introduce a common parent connect for all Debezium source connectors 2023-12-19 12:53:57 +01:00
harveyyue
5553059fb8 DBZ-7167 Provide a public API from the connector implementations to retrieve the list of matching collections or tables based on the different include-/exclude lists 2023-12-19 12:53:57 +01:00
Vojtech Juranek
37ebb8ecae DBZ-7275 Close connection registerd with bean registry
New connection created for bean registry is never closed. Close it
during when stopping the task.
2023-12-19 10:22:02 +01:00
Debezium Builder
2c1def7241 [maven-release-plugin] prepare for next development iteration 2023-12-14 09:43:13 +00:00
Debezium Builder
ef8260f802 [maven-release-plugin] prepare release v2.5.0.CR1 2023-12-14 09:43:12 +00:00
Chris Cranford
2b02b3982e DBZ-4321 Rework configuration options 2023-12-13 11:27:40 -05:00
Chris Cranford
ada157d117 DBZ-4321 Introduce BeanRegistry and ServiceRegistry
As a part of this work to handle injection in a cleaner way, this commit
adds two new broad concepts called `BeanRegistry` and `ServiceRegistry`.

A BeanRegistry is a glorified registry of different objects that are not
necessarily services but may be desired by a service. This contract will
allow Debezium to integrate in the future with other CDI providers.

A ServiceRegistry is more of an internal concept, where various systems
can be started based on their dependency order and provides a universal
way to split larger parts of the code into smaller, focused modules that
can be accessed using the Service Locator pattern.
2023-12-13 11:27:40 -05:00
Chris Cranford
26b3881941 DBZ-4321 Add MongoDB hook & polish injection handler 2023-12-13 11:27:40 -05:00
Chris Cranford
59027ed5ed DBZ-4321 New PostProcessor contract and Column Reselection 2023-12-13 11:27:40 -05:00
Chris Cranford
aea6cc003d
[ci] Use a relaxed property check for tests 2023-12-06 18:04:59 -05:00
mfvitale
874ab18017 DBZ-6834 Enable REPLICA IDENTITY on debezium_signal test table 2023-12-06 14:10:26 +01:00
Debezium Builder
0fd1c0dc9a [maven-release-plugin] prepare for next development iteration 2023-12-04 13:55:35 +00:00
Debezium Builder
3e2d75f0da [maven-release-plugin] prepare release v2.5.0.Beta1 2023-12-04 13:55:35 +00:00
Prabhu19
8b0487fb5c DBZ-7193 Added testcases for tables toasted array columns with replica identity FULL. 2023-12-01 13:52:21 +01:00
Prabhu19
c60cbbd4aa DBZ-7193 Unchanged toasted array columns are substituted with unavailable.value.placeholder, even when REPLICA IDENTITY FULL is configured.
This PR fixes the above issue.
2023-12-01 13:52:21 +01:00
Gunnar Morling
0ff6c730d6 DBZ-7181 Avoiding queries not available in recovery mode 2023-11-27 10:47:16 +01:00
rkerner
aeaba3ae70 DBZ-6762 Add "validate filters" endpoint/s for connector-specific Connect REST Extensions
closes to https://issues.redhat.com/browse/DBZ-6762
2023-11-22 06:10:40 +01:00
Debezium Builder
1521445908 [maven-release-plugin] prepare for next development iteration 2023-11-10 10:26:05 +00:00
Debezium Builder
6c6f6e9138 [maven-release-plugin] prepare release v2.5.0.Alpha2 2023-11-10 10:26:05 +00:00
Vojtech Juranek
9d26dd4cf4 DBZ-7007 Move embedded engine config into separate interface
This config will be re-used by possible other implementations of
DebeiumEngine API in the embedded package. As DebeziumEngine API
can have completely different implementations and thus also config,
the class is called `EmbeddedEngineConfig` as it's assumed to be used
only by embedded engine "family" of implementations.

To keep backward compatibility, the config options are extracted into
an interface and `EmbeddedEngine` implements this interface, thus
allowing to use these options in custom classes without any need for the
code changes.
2023-11-06 10:50:21 +01:00
Vojtech Juranek
e073dbd1be DBZ-7007 Use DebeziumEngine API instead of EmbeddedEngine in postgres tests 2023-11-06 10:50:21 +01:00
Debezium Builder
78c5204444 [maven-release-plugin] prepare for next development iteration 2023-10-26 15:39:35 +00:00
Debezium Builder
356e488e83 [maven-release-plugin] prepare release v2.5.0.Alpha1 2023-10-26 15:39:35 +00:00
Roman Kudryashov
215f3098b7 DBZ-7016 Refactoring 2023-10-19 11:35:18 +02:00
Roman Kudryashov
ee64c70546 DBZ-7016 id and type of a CloudEvent can be retrieved from headers 2023-10-19 11:35:18 +02:00
Jiri Pechanec
899ac0d285 DBZ-7015 Hide the option in UI 2023-10-16 15:37:29 +02:00
Praveen Burgu
f9b07e6517 DBZ-7015 Make slot validation check opt-in 2023-10-16 15:37:29 +02:00
Praveen Burgu
7fd68a5a0d DBZ-7015 Enable replication slot advance check
This reverts commit 925de0184c.
2023-10-16 15:37:29 +02:00
Vojtech Juranek
d7e8943d4e DBZ-6778 Initialize ElapsedTimeStrategy upon its creation
Currently, newly created `ElapsedTimeStrategy` is uninitialized and its
`hasElapsed()` has to be called once `ElapsedTimeStrategy` is created to
initialize the strategy. This is confusing and error prone.

Move initialization of `ElapsedTimeStrategy` into it's constructor, so
it's initialized once it's created.
2023-10-05 11:57:59 +02:00
Roman Kudryashov
01485aceb3 DBZ-6982 Provide configuration option to exclude extension attributes from a CloudEvent 2023-10-04 13:58:53 +02:00
Debezium Builder
40a8f4026d [maven-release-plugin] prepare for next development iteration 2023-10-03 14:18:54 +00:00
Debezium Builder
412c6f7979 [maven-release-plugin] prepare release v2.4.0.Final 2023-10-03 14:18:54 +00:00
Vojtech Juranek
496495bc1c DBZ-6987 Create schema in Postgres test if not exists 2023-10-03 07:52:18 +02:00
Vojtěch Juránek
2776814cd5 DBZ-6986 Provide more clear reason why the test is skipped
Co-authored-by: Fiore Mario Vitale <mvitale86@gmail.com>
2023-10-03 07:49:08 +02:00
Vojtech Juranek
5abe3f2e52 DBZ-6986 Skip RecordsStreamProducerIT#shouldReceiveChangesForInfinityNumericWithInfinity on Postgres < 14
Infinity support for numeric types was added in Postgres 14, see
https://www.postgresql.org/docs/release/14.0/
2023-10-03 07:49:08 +02:00
Vojtech Juranek
c31aeef854 DBZ-6216 Use quay.io for base postgres image 2023-10-03 07:49:08 +02:00
Roman Kudryashov
98c426dec4 DBZ-3642 CloudEventsConverter can retrieve metadata info from headers 2023-09-26 13:06:09 +02:00
Debezium Builder
cc1de0aa51 [maven-release-plugin] prepare for next development iteration 2023-09-22 09:00:45 +00:00
Debezium Builder
6809978083 [maven-release-plugin] prepare release v2.4.0.CR1 2023-09-22 09:00:45 +00:00
mfvitale
5d7ba3e951 DBZ-6957 Remove spaces in Signal and Notification JMX ObjectName 2023-09-21 12:04:33 -04:00
rkerner
6d3528d4fb DBZ-4395 Add connector specific Debezium Connect REST Extension/s and move logic from UI backend to the Debezium Connect REST Extension (part 2 of the re-architecturing/re-factoring)
DBZ-6761 Add "validate connection" endpoints in connector specific Connect REST extensions

closes to https://issues.redhat.com/browse/DBZ-4395
closes to https://issues.redhat.com/browse/DBZ-6761
2023-09-21 13:29:59 +02:00
mfvitale
ae199d2053 DBZ-6828 Blocking snapshot will generate schema events only for signaled tables 2023-09-20 08:18:33 +02:00
Jiri Pechanec
29e45b0ce0 DBZ-6911 Support for PostgreSQL 16 2023-09-18 15:33:11 +02:00
xiaowu
e50d0eb87a DBZ-6935 fix logger named 2023-09-18 12:34:47 +02:00
Debezium Builder
65169a9315 [maven-release-plugin] prepare for next development iteration 2023-09-13 09:41:46 +00:00
Debezium Builder
845e3e9d09 [maven-release-plugin] prepare release v2.4.0.Beta2 2023-09-13 09:41:46 +00:00
Jiri Pechanec
f1e7999431 DBZ-6481 Support Postgres enum as key in incremental snapshot 2023-09-11 06:08:57 +02:00
Debezium Builder
d97a18e8dc [maven-release-plugin] prepare for next development iteration 2023-08-29 11:41:08 +00:00
Debezium Builder
5ce25f2cfe [maven-release-plugin] prepare release v2.4.0.Beta1 2023-08-29 11:41:08 +00:00
Jiri Pechanec
dac07d53c3 DBZ-6482 TimescaleDB transformation 2023-08-29 11:45:27 +02:00
Vojtech Juranek
28b06e6795 DBZ-6839 Fix PostgresConnectorIT#shouldAddNewFieldToSourceInfo
PostgresConnectorIT#shouldAddNewFieldToSourceInfo fails only when run
together with other tests and the failure is random. It seem there is
a caching issue in Apicuro, when `test_server.s1.a-value` artifact
references `io.debezium.connector.postgresql.Source` version 1, which
hasnt't `newField` field and this reference is used also in
`shouldAddNewFieldToSourceInfo` where artifact with version 2 should
be used. Using dedicated table and thus creating new artifact in
Apucurio should fix this issue.

Also remove unused variable from `CustomPostgresSourceInfoStructMaker`.
2023-08-25 15:02:18 +02:00
mfvitale
542b3619fa DBZ-6731 Blocking snapshot takes configuration from signal payload 2023-08-15 12:43:43 +02:00
Vojtech Juranek
6ca842b772 DBZ-6635 Send heartbeats also before processing first event
We should send heartbeats no matter if there is already any DB record
processed or not. This prevents situation when after the start there is
no new record in the Db and Debezium is not sending neither DB events
nor heartbeats.
2023-08-14 18:45:43 +02:00
harveyyue
76738a6402 DBZ-6758 Add it test for (+/-) infinity values for numeric/decimal types 2023-08-14 09:04:33 +02:00
harveyyue
15e81c1a84 DBZ-6758 When using pgoutput in postgres connector, (+/-)Infinity is not supported in decimal values 2023-08-14 09:04:33 +02:00
rajdangwal
b3098de0fd DBZ-6720: minor nit 2023-08-14 08:59:54 +02:00
rajdangwal
153834c650 DBZ-6720: handle unchanged toasted uuid array 2023-08-14 08:59:54 +02:00
Debezium Builder
3c72f071b0 [maven-release-plugin] prepare for next development iteration 2023-08-09 10:33:06 +00:00
Debezium Builder
915c465440 [maven-release-plugin] prepare release v2.4.0.Alpha2 2023-08-09 10:33:06 +00:00
harveyyue
b16d991a25 DBZ-6684 Rename back fields with avro_unicode 2023-08-08 10:22:10 +02:00
Jiri Pechanec
a8664ae3e9 DBZ-6767 Create publication for pgoutput only 2023-08-08 08:18:17 +02:00
Jiri Pechanec
3d263d645e DBZ-6767 Provide bytea suport 2023-08-08 08:18:17 +02:00
Jiri Pechanec
3aa6759ebd DBZ-6767 Add missing type name 2023-08-08 08:18:17 +02:00
mfvitale
61651a94b5 DBZ-6566 Force blocking snapshot even if a snapshot has already executed 2023-08-03 09:22:24 +02:00
mfvitale
def7379a0d DBZ-6566 Support Blocking snapshot for MySQL 2023-08-03 09:22:24 +02:00
mfvitale
6e0b323e83 DBZ-6566 Set correct log context during blocking snapshot 2023-08-03 09:22:24 +02:00
mfvitale
70f16af7c4 DBZ-6566 Support Blocking snapshot for PostgreSQL 2023-08-03 09:22:24 +02:00
jcechace
699a1612ae DBZ-6670 Replace error handler instead of recreate in order to keep retry counter 2023-07-25 13:46:56 +02:00
mfvitale
8ea2b3c5b7 DBZ-6669 Snapshot will get data when only signal data collection table is specified 2023-07-24 10:02:13 -04:00
Debezium Builder
267dfda8a6 [maven-release-plugin] prepare for next development iteration 2023-07-17 10:40:52 +00:00
Debezium Builder
0d3cef6a29 [maven-release-plugin] prepare release v2.4.0.Alpha1 2023-07-17 10:40:52 +00:00
Jiri Pechanec
4ca2a1c5d0 DBZ-6517 Use correct constructor 2023-07-14 09:11:10 +02:00
jcechace
318c55cc74 DBZ-6659 Fixing junit dependencies 2023-07-14 09:11:10 +02:00
Vojtech Juranek
5cf61444c0 DBZ-6648 Use String fallback when interval handling mode is String
The root cause of DBZ-6648 is using long number as a fallback value,
while String is expected and thus falling conversion and returning
`null`.

Use String fallback when interval handling mode is String.
2023-07-14 06:03:59 +02:00
Jiri Pechanec
764427081b DBZ-6517 DBZ-6654 Make CE converter compatible with Kafka 3.5 2023-07-13 11:26:21 +02:00
harveyyue
58d3e150b9 DBZ-6603 Add integration test for custom metric tags 2023-07-13 09:52:33 +02:00
harveyyue
5bc7fd1f7c DBZ-6603 Support for custom tags in the connector metrics 2023-07-13 09:52:33 +02:00
jcechace
5cb579bf12 DBZ-6525 Removing runtime dependency on core for testcontainers module in order to prevent logback.xml downstream conflicts 2023-07-11 12:07:04 +02:00
mfvitale
0baad2f7b7 DBZ-6417 Add test for custom action 2023-06-23 15:03:27 +02:00
mfvitale
d5febcc054 DBZ-6417 Provide extensible signal action mechanism through SPI 2023-06-23 15:03:27 +02:00