nicholas-fwang
2bdeec099a
DBZ-7143 Add document for event.converting.failure.handling.mode
2024-01-29 13:48:19 +01:00
nicholas-fwang
7d99605886
DBZ-7143 Add case when EventConvertingFailureHandlingMode is null
2024-01-29 13:48:19 +01:00
nicholas-fwang
4f2bbb023a
DBZ-7143 fix checkstyle format
2024-01-29 13:48:19 +01:00
nicholas-fwang
3a5f35594c
DBZ-7143 Add description of event.converting.failure.handling.mode
2024-01-29 13:48:19 +01:00
nicholas-fwang
57a46943af
DBZ-7143 throw exception in JdbcValueConverters
2024-01-29 13:48:19 +01:00
nicholas-fwang
eeea0f1e70
DBZ-7143 Add event.converting.failure.handling.mode option
2024-01-29 13:48:19 +01:00
Animesh Kumar
2f8114029e
DBZ-7380 Offset transaction id only when it's non null
2024-01-24 15:32:34 +01:00
ani-sha
5be134eb42
DBZ-7022 Use prefix names from Envelope class
2024-01-22 10:49:38 +01:00
ani-sha
78a54a3ff7
DBZ-7022 Adjust prefix names, simplify null check
2024-01-22 10:49:38 +01:00
ani-sha
17d3df62c0
DBZ-7022 Allow conversion of source block timestamp fields
2024-01-22 10:49:38 +01:00
Chris Cranford
c1b7e68319
DBZ-7358 Use relational table primary key by default
...
This fix uses the relational table primary key by default; however, as a
user can define `message.key.columns` to even override the primary key
configuration or to handle keyless tables, the user can override the
`reselect.use.event.key` option with `true` so use the event key fields
instead for the re-select so that keyless tables can also participate
with the column reselection process.
2024-01-18 05:53:37 +01:00
Peter Hamer
a4e660f979
DBZ-7347 use identifiers for initial snapshot notifications
2024-01-17 09:42:27 +01:00
“vsantonastaso”
c045fab3dd
DBZ-7249 Add Notification ID in log channel
2024-01-12 06:34:48 +01:00
Roman Kudryashov
0c80f1f38d
DBZ-7284 Provide config option to customize CloudEvents.data schema name
2024-01-11 13:20:35 +01:00
Jakub Cechacek
5dbe073d3b
DBZ-7260 Offset value conversion extracted into common util class
2024-01-11 09:58:24 +01:00
mfvitale
20fbe73bc8
DBZ-7311 Use a more meaningful name for identify an on demand blocking snapshot
2024-01-08 12:52:03 +01:00
mfvitale
47cbdee526
DBZ-7311 Permits to execute a blocking snapshot even if snapshot.mode=never
2024-01-08 12:52:03 +01:00
Chris Cranford
84fe328a91
DBZ-7315 Introduce trimmed variant helpers
2024-01-05 10:52:32 +01:00
Mickael Maison
70e7186868
DBZ-7298 Add service loader manifests for all Connect plugins
...
Add manifests for source/sink connectors, converters and transformations.
2024-01-04 09:22:29 +01:00
mfvitale
b770ed4a31
DBZ-7256 Enhance logs for incremental snapshot
2024-01-03 09:19:05 +01:00
ani-sha
4b36ed9975
DBZ-7177 DBZ-6764 Use correct context name and add suggestions for code review
2023-12-20 08:56:56 +01:00
rkerner
e06f96f339
DBZ-6764 Fix waitForStreamingRunning/isStreamingRunning
2023-12-20 08:56:56 +01:00
ani-sha
a12b1a9872
DBZ-6764 Connector metrics implementation for REST extension
2023-12-20 08:56:56 +01:00
ani-sha
e535e5b78d
DBZ-7177 Add suggestions from code review, adjust code style changes
2023-12-20 08:56:56 +01:00
ani-sha
2555a1cee2
DBZ-6764 DBZ-7178 DBZ-7177 Use MBeanServer to query metrics for REST endpoint and create new metrics descriptor
2023-12-20 08:56:56 +01:00
ani-sha
2165d9bed1
DBZ-6764 Connector metrics implementation for REST extension
2023-12-20 08:56:56 +01:00
Roman Kudryashov
516aa87fad
DBZ-7235 Add option to customize CloudEvents schema name
2023-12-20 06:53:37 +01:00
“vsantonastaso”
9dac11e1cd
DBZ-7164 Support offset commit for the signal topic
2023-12-19 15:51:10 +01:00
harveyyue
225277c5c0
DBZ-7251 Return generic DataCollectionId instead of REST response DataCollection in getMatchingCollections method
2023-12-19 12:53:57 +01:00
harveyyue
4719396ffc
DBZ-7145 Introduce a common parent connect for all Debezium source connectors
2023-12-19 12:53:57 +01:00
harveyyue
5553059fb8
DBZ-7167 Provide a public API from the connector implementations to retrieve the list of matching collections or tables based on the different include-/exclude lists
2023-12-19 12:53:57 +01:00
Jiri Pechanec
66bb795860
DBZ-7271 Make db name optional in RDS filter regexes
2023-12-19 06:20:33 +01:00
Chris Cranford
985e5c8600
DBZ-4321 Use Closeable rather than Stoppable
2023-12-13 11:27:40 -05:00
Chris Cranford
5c6b437f24
DBZ-4321 Rename getServiceNoThrow as tryGetService
2023-12-13 11:27:40 -05:00
Chris Cranford
21641d76c0
DBZ-4321 Use InjectService in PostProcessorRegistry
2023-12-13 11:27:40 -05:00
Chris Cranford
2b02b3982e
DBZ-4321 Rework configuration options
2023-12-13 11:27:40 -05:00
Chris Cranford
8b016adb70
DBZ-4321 Adjust ReselectColumnsPostProcessor column value null check
2023-12-13 11:27:40 -05:00
Chris Cranford
82b317df92
DBZ-4321 Relax service existence in EventDispatcher
2023-12-13 11:27:40 -05:00
Chris Cranford
ada157d117
DBZ-4321 Introduce BeanRegistry
and ServiceRegistry
...
As a part of this work to handle injection in a cleaner way, this commit
adds two new broad concepts called `BeanRegistry` and `ServiceRegistry`.
A BeanRegistry is a glorified registry of different objects that are not
necessarily services but may be desired by a service. This contract will
allow Debezium to integrate in the future with other CDI providers.
A ServiceRegistry is more of an internal concept, where various systems
can be started based on their dependency order and provides a universal
way to split larger parts of the code into smaller, focused modules that
can be accessed using the Service Locator pattern.
2023-12-13 11:27:40 -05:00
Chris Cranford
26b3881941
DBZ-4321 Add MongoDB hook & polish injection handler
2023-12-13 11:27:40 -05:00
Chris Cranford
59027ed5ed
DBZ-4321 New PostProcessor
contract and Column Reselection
2023-12-13 11:27:40 -05:00
harveyyue
c5ce1cf945
DBZ-7251 Length value is not removed when changing a column's type
2023-12-12 10:18:21 +01:00
harveyyue
b6924e6efc
DBZ-7145 Introduce a new SMT to extract table schema information for jdbc sink connector
2023-12-08 13:33:15 +01:00
Roman Kudryashov
c7b3197bc1
DBZ-7216 Fix data schema name in case of metadata is in headers
2023-12-06 14:22:24 +01:00
mfvitale
5ed16284f4
DBZ-6834 Provide INSERT/DELETE semantics for MongoDb incremental snapshot watermarking
2023-12-06 14:10:26 +01:00
mfvitale
4fedfbba03
DBZ-6834 Provide INSERT/DELETE semantics for incremental snapshot watermarking
2023-12-06 14:10:26 +01:00
Jakub Cechacek
8a5007a738
DBZ-7184 Refactored streaming loop of MongoDB connector
2023-12-04 06:58:28 +01:00
Jakub Cechacek
93e8570fb6
DBZ-7184 Implementation of buffering change stream cursor
2023-12-04 06:58:28 +01:00
Chris Cranford
17a9245772
DBZ-7197 Checkstyle fixes
2023-11-29 12:10:04 -05:00
Matt Vance
f26c24e82f
DBZ-7197 Add "table.fields.additional.error.on.missing" config support to Outbox EventRouter
2023-11-29 12:10:04 -05:00
Roman Kudryashov
6474c68e42
DBZ-7159 Simplify CE validator API by setting serializer type at the configuration phase
2023-11-29 15:09:10 +01:00
“vsantonastaso”
8c1c369449
DBZ-6878 add table specific notification in initial snapshot
2023-11-29 08:32:21 +01:00
Roman Kudryashov
cf3b74bcfb
DBZ-7159 Code review
2023-11-23 14:19:37 +01:00
Roman Kudryashov
1992c1e7e4
DBZ-7159 Fail fast during deserialization if a value is not a CloudEvent
2023-11-23 14:19:37 +01:00
rkerner
aeaba3ae70
DBZ-6762 Add "validate filters" endpoint/s for connector-specific Connect REST Extensions
...
closes to https://issues.redhat.com/browse/DBZ-6762
2023-11-22 06:10:40 +01:00
ruslan
daf7f57d77
DBZ-7142 Improved Outbox Event Router SMT to handle whitespaces inside of fields.additional.placement setting
2023-11-16 11:57:34 +01:00
mfvitale
a049ea92fa
DBZ-7141 Remove ComputePartition SMT related code
2023-11-15 12:57:11 +01:00
Jiri Pechanec
e654dc228f
DBZ-7066 Code reorganization
2023-11-14 11:56:16 +01:00
harveyyue
223b6e4335
DBZ-7066 Extract common creating transformation methods to ConnectRecordUtil
2023-11-14 11:56:16 +01:00
harveyyue
4c3b5540ec
DBZ-7066 Add NewExtractNewDocumentStateTest and NewExtractNewDocumentStateTestIT for new option "delete.tombstone.handling.mode"
2023-11-14 11:56:16 +01:00
harveyyue
d21d5838fc
DBZ-7066 Add NewExtractNewRecordStateTest for new option "delete.tombstone.handling.mode"
2023-11-14 11:56:16 +01:00
harveyyue
5e8287eb40
DBZ-7066 The DefaultDeleteHandlingStrategy couldn't add the rewrite "__deleted" field to a non-struct value
2023-11-14 11:56:16 +01:00
mfvitale
33dc8e0ec6
DBZ-6317 Document tombstone behaviour
2023-11-10 06:43:18 +01:00
rkerner
68481038b3
DBZ-7105 Fix exceptions that are triggered by RelationalBaseSourceConnector#validate when calling #validateConnection() with an invalid configuration. Other, not-checked fields can cause an exception when the ConnectorConfig class is instantiated to setup the connection. Now we are only allowing connection validation when all config options are free of errors.
...
closes to https://issues.redhat.com/browse/DBZ-7105
2023-11-06 15:02:51 +01:00
Roman Kudryashov
3d56341375
DBZ-7060 Rename metadata.location
-> metadata.source
2023-10-27 13:55:18 +02:00
harveyyue
767c342506
DBZ-7062 Update transformation property "delete.tombstone.handling.mode" to debezium doc
2023-10-27 11:17:02 +02:00
Jiri Pechanec
13d1b4e0b2
DBZ-4999 Provide hook to allow Informix to avoid blocking
2023-10-20 11:15:17 +02:00
Roman Kudryashov
215f3098b7
DBZ-7016 Refactoring
2023-10-19 11:35:18 +02:00
Roman Kudryashov
b0d7d60b5d
DBZ-7016 Optimize retrieval of id and type
2023-10-19 11:35:18 +02:00
Roman Kudryashov
89e2700cbf
DBZ-7016 Fix dataschema()
2023-10-19 11:35:18 +02:00
Roman Kudryashov
22f6b083b1
DBZ-7016 Change metadata.location property syntax
2023-10-19 11:35:18 +02:00
Roman Kudryashov
ee64c70546
DBZ-7016 id
and type
of a CloudEvent can be retrieved from headers
2023-10-19 11:35:18 +02:00
Jiri Pechanec
9b098615ab
DBZ-6907 Change package; use better strategy names
2023-10-19 09:35:15 +02:00
harveyyue
4836fcf1ea
DBZ-6907 Introduce a new handling deleted record option "delete.tombstone.handling.mode"
2023-10-19 09:35:15 +02:00
harveyyue
f64ea1a5f8
DBZ-6907 Extract an abstract transformation class from ExtractNewRecordState
2023-10-19 09:35:15 +02:00
Jiri Pechanec
6dfbdb4f69
DBZ-7033 Upgrade Kafka to 3.6.0
2023-10-18 07:23:42 +02:00
ani-sha
49ff78ae9a
DBZ-7020 Modify the updated schema envelope, adjust tests
2023-10-16 10:06:38 +02:00
ani-sha
d28bedb710
DBZ-7020 Use single record schema definition
2023-10-16 10:06:38 +02:00
ani-sha
c4fe510f00
DBZ-7020 Add source block struct for transformed records
2023-10-16 10:06:38 +02:00
ani-sha
8cb02eb16a
DBZ-7020 Use optional schema for Timezone Converter tests
2023-10-16 10:06:38 +02:00
Vojtech Juranek
3cc70c8a52
DBZ-6778 Remove unused code
2023-10-05 11:57:59 +02:00
Vojtech Juranek
d7e8943d4e
DBZ-6778 Initialize ElapsedTimeStrategy upon its creation
...
Currently, newly created `ElapsedTimeStrategy` is uninitialized and its
`hasElapsed()` has to be called once `ElapsedTimeStrategy` is created to
initialize the strategy. This is confusing and error prone.
Move initialization of `ElapsedTimeStrategy` into it's constructor, so
it's initialized once it's created.
2023-10-05 11:57:59 +02:00
Roman Kudryashov
e8e7c705af
DBZ-6982 Fix property name
2023-10-04 13:58:53 +02:00
Roman Kudryashov
01485aceb3
DBZ-6982 Provide configuration option to exclude extension attributes from a CloudEvent
2023-10-04 13:58:53 +02:00
harveyyue
e80c82622f
DBZ-6983 Add the ability to sanitize field name when inferencing json schema
2023-10-04 13:53:18 +02:00
Jakub Cechacek
baafe602ce
DBZ-6973 Literal matching mode for database/collection filters and ability to override internal aggregation pipeline for MongoDB conenctor
2023-10-02 08:25:53 +02:00
Jiri Pechanec
b65582f2c0
DBZ-6966 Fix reverted filtering logic
2023-09-28 14:27:16 -04:00
mfvitale
56795732b5
DBZ-6793 Add timestamp for notifications into documentation
2023-09-28 14:20:17 -04:00
mfvitale
993413955c
DBZ-6793 Add timestamp for notifications
2023-09-28 14:20:17 -04:00
ani-sha
c20f297fe1
DBZ-6940 Refactor & add check for unsupported schema types
2023-09-28 13:58:13 -04:00
Jiri Pechanec
c7caa6726f
DBZ-3642 Use Java naming conventions
2023-09-26 13:06:09 +02:00
Roman Kudryashov
98c426dec4
DBZ-3642 CloudEventsConverter can retrieve metadata info from headers
2023-09-26 13:06:09 +02:00
Animesh Kumar
fcbff074f4
DBZ-6787 Dedupe table list while incremental snapshot
2023-09-26 10:50:32 +02:00
nancyxu123
b08acc6043
DBZ-6870 throw producer exception during poll() even if the queue isn't empty yet
2023-09-26 10:05:48 +02:00
Jiri Pechanec
cd2df01fe8
DBZ-6956 Add parenthesis around additional condition
2023-09-25 06:10:35 +02:00
mfvitale
5d7ba3e951
DBZ-6957 Remove spaces in Signal and Notification JMX ObjectName
2023-09-21 12:04:33 -04:00
mfvitale
b4661881c9
DBZ-6945 Remove Table object for DROP schema event
2023-09-21 13:32:38 +02:00
rkerner
32161c9874
DBZ-4395 Add connector specific Debezium Connect REST Extension/s and move logic from UI backend to the Debezium Connect REST Extension (part 1 of the re-architecturing/re-factoring)
...
closes to https://issues.redhat.com/browse/DBZ-4395
2023-09-21 13:29:59 +02:00
harveyyue
e341910d2c
DBZ-6910 Values being omitted from list of JSON object
2023-09-21 05:31:36 +02:00
mfvitale
68eb4b2df2
DBZ-6828 Process drop table events during blocking snapshot
2023-09-20 08:18:33 +02:00
mfvitale
ae199d2053
DBZ-6828 Blocking snapshot will generate schema events only for signaled tables
2023-09-20 08:18:33 +02:00
mfvitale
19ee1070f5
DBZ-6937 Incremental snapshot context is initialized in processSchemaChange method to avoid NPE when no context from initial snapshot exists
2023-09-20 08:14:36 +02:00
Jiri Pechanec
d83139a54a
DBZ-6901 Fix formatting
2023-09-19 07:00:14 +02:00
harveyyue
07abd6dbc2
DBZ-6901 ExtractNewRecordState's schema cache is not updated with arrival of the ddl change event
2023-09-19 07:00:14 +02:00
Chris Cranford
46fa3bef4d
DBZ-2543 Refactor value conversion logic
2023-09-12 18:20:07 -04:00
Jiri Pechanec
f1e7999431
DBZ-6481 Support Postgres enum as key in incremental snapshot
2023-09-11 06:08:57 +02:00
ani-sha
087362bfcc
DBZ-6567 Use optional seconds in TIMEZONE_OFFSET_PATTERN, handle sub-milliseconds precision loss, correct the DST time values
2023-09-07 11:00:26 +02:00
ani-sha
d46744aa06
DBZ-6567 Add a test case for checking DST
2023-09-07 11:00:26 +02:00
ani-sha
78477e7622
DBZ-6567 Handle DST for test cases, use correct zone rules
2023-09-07 11:00:26 +02:00
ani-sha
22500839de
DBZ-6567 SMT for handling timezone based fields
2023-09-07 11:00:26 +02:00
Chris Cranford
c86b2648a5
DBZ-6864 Checkstyle fixes
2023-09-04 09:02:16 +02:00
Breno Moreira
3a2e141c56
DBZ-6864 - Debezium-MySQL not filtering AWS RDS internal events
2023-09-04 09:02:16 +02:00
Jiri Pechanec
0dd0ff7306
DBZ-6865 Clarify error message
2023-09-04 08:59:31 +02:00
harveyyue
fe300784b0
DBZ-6865 Avoid getting NPE when executing the arrived method in ExecuteSnapshot
2023-09-04 08:59:31 +02:00
“vsantonastaso”
3ec6a3f18e
DBZ-6843 change skipped change event log to debug
2023-08-29 12:18:17 +02:00
Chris Cranford
ed88a73510
DBZ-6840 Add test case
2023-08-29 11:56:12 +02:00
harveyyue
c03b2249d8
DBZ-6840 Wrong filtered comments
2023-08-29 11:56:12 +02:00
Vojtech Juranek
c2b48312a6
DBZ-6841 Fix randomly failing BaseSourceTaskTest#verifyTaskRestartsSuccessfully
...
Test was passing locally and in the CI as the start was very fast and
after the first start retriable wait condition wasn't met, so we
actually sleep and while the sleep we doesn't increment the counter
and the state is still `RESTARTING`. On slower machines when start
takes longer time and we don't wait, the test is actualy wrong.
The correct test either should remove last assert for `RESTARTING`
state or increase number of failures and add a sleep time after initial
start, so that there is no retry wait time.
This PR implements the later option.
2023-08-29 10:27:56 +02:00
“vsantonastaso”
43adc59292
DBZ-6742 use JSON format for JMX Notitication userData
2023-08-22 06:42:51 +02:00
Chris Beard
2d6490e3f8
[docs] Fix minor typos in ExtractNewRecordState docs
2023-08-20 22:37:03 -04:00
Matan Cohen
40ac4662fb
DBZ-6712 fix snapshot to history internal topic to respect schema.history.internal.store.only.captured.databases.ddl flag
2023-08-15 19:34:40 +02:00
mfvitale
542b3619fa
DBZ-6731 Blocking snapshot takes configuration from signal payload
2023-08-15 12:43:43 +02:00
“vsantonastaso”
64eaf72665
DBZ-6689 make kafka signal consumer group configurable
2023-08-11 13:20:07 +02:00
Anil Dasari
b7e0e1cd2a
DBZ-6641: Schema name changed with Custom topic naming strategy
2023-08-10 11:53:31 +02:00
Anil Dasari
a4c0bf5614
DBZ-6641: Schema name changed with Custom topic naming strategy
2023-08-10 11:53:31 +02:00
Jiri Pechanec
ec100d6127
DBZ-6240 Rename vars to reflect the semantics
2023-08-10 07:04:17 +02:00
sunxiaojian
43c7c0a3f6
DBZ-6240 Rename the configuration name 'schema.change.event.include.list' to 'schema.change.event.exclude.list'
2023-08-10 07:04:17 +02:00
sunxiaojian
6451c93256
DBZ-6240 Add an STM to filter schema change event
2023-08-10 07:04:17 +02:00
david remy
565075a320
DBZ-2862 Improved OpenTelemetry Handling in Debezium
...
- Implemented a dynamic solution for handling version changes in OpenTelemetry, using Java introspection to determine the correct interceptor class at runtime.
- Utilized a custom "ProducerInterceptor" to delegate logic based on the presence of specific OpenTelemetry classes.
- Removed direct usage of OpenTelemetry's internal API 'ConfigUtils'.
- Aligned solution with Debezium's aim of ensuring compatibility with Strimzi.
This update ensures a more robust handling of OpenTelemetry's version transitions.
2023-08-08 21:34:04 +02:00
REMY David
15fdcb5027
DBZ-2862 Performance optimization
...
- Changing the traceRecord method to be static. This avoids creating an instance on each call.
- Caching the results of isOpenTelemetryJavaagentEnable() and isOpenTelemetryApiEnable().
- Using isDebugEnabled() to avoid the costs of constructing strings when it's not necessary.
- Reorganizing some conditions to minimize the operations performed.
- Moving the static instances TEXT_MAP_PROPAGATOR, SETTER, and GETTER into the traceRecord method. This will prevent them from being held in memory when not in use.
2023-08-08 21:34:04 +02:00
REMY David
76c536ec23
DBZ-2862 Adaptation of traceability with OpenTelemetry dependencies in the scope provided. Moved the interceptor to a new module.
2023-08-08 21:34:04 +02:00
REMY David
5846446672
DBZ-2862 Switch tracing to OpenTelemetry, Rewrite SMT with Otel API, Add Interceptor for e2e tracing, add Otel TextMap getter and setter
2023-08-08 21:34:04 +02:00
mfvitale
df52a1068a
DBZ-6777 Fix wrong static notification and signals lists that lead to be wrongly shared between connectors instances
2023-08-08 13:01:37 +02:00
Vojtech Juranek
41cdaeb170
DBZ-6213 Adjust connector restart
...
* Try to start the connector in BaseSourceTask::start(). This is the
root cause of DBZ-6213 as EmbeddedEngine calls start() and assumes
it really tries to start the connector.
* Simplify the code by removing unnecessary checks.
2023-08-08 10:27:33 +02:00
Jiri Pechanec
e5ca8c5000
DBZ-6684 Use more descriptive class name
2023-08-08 10:22:10 +02:00
harveyyue
b16d991a25
DBZ-6684 Rename back fields with avro_unicode
2023-08-08 10:22:10 +02:00
tyrantlucifer
32d2bf1fac
DBZ-6745 Remove redundant filter logic from SingleProcessor
2023-08-03 16:52:41 +02:00
mfvitale
524987227f
DBZ-6566 Shutdown blockingSnapshotExecutor when ChangeEventSourceCoordinator stops
2023-08-03 09:22:24 +02:00
mfvitale
61651a94b5
DBZ-6566 Force blocking snapshot even if a snapshot has already executed
2023-08-03 09:22:24 +02:00
mfvitale
def7379a0d
DBZ-6566 Support Blocking snapshot for MySQL
2023-08-03 09:22:24 +02:00
mfvitale
6e0b323e83
DBZ-6566 Set correct log context during blocking snapshot
2023-08-03 09:22:24 +02:00
mfvitale
70f16af7c4
DBZ-6566 Support Blocking snapshot for PostgreSQL
2023-08-03 09:22:24 +02:00
jcechace
699a1612ae
DBZ-6670 Replace error handler instead of recreate in order to keep retry counter
2023-07-25 13:46:56 +02:00
mfvitale
8ea2b3c5b7
DBZ-6669 Snapshot will get data when only signal data collection table is specified
2023-07-24 10:02:13 -04:00
Chris Cranford
de8cf36b2c
DBZ-6660 Introduce internal.log.mining.max.scn.deviation.ms
...
We hypothesize that there could be a situation where we may be mining precisely
around the CURRENT_SCN and this may lead to situations where LGWR may not have
flushed all records for the same SCN before being mined by the connector.
2023-07-17 10:22:46 +02:00
Chris Cranford
56ddbe21d5
DBZ-6650 Avoid ConcurrentModificationException during dispatch
2023-07-14 11:21:39 +02:00
harveyyue
e3f684170e
DBZ-6472 MySqlSnapshotChangeEventSource parallel execute createSchemaEventsForTables
2023-07-13 13:21:59 +02:00
Adam Strickland
0b15fe4556
[docs] Doc of type of return value
2023-07-13 13:08:41 +02:00
Jiri Pechanec
764427081b
DBZ-6517 DBZ-6654 Make CE converter compatible with Kafka 3.5
2023-07-13 11:26:21 +02:00
harveyyue
aa139203aa
DBZ-6603 Update property "custom.metric.tags" to connector docs
2023-07-13 09:52:33 +02:00
harveyyue
5bc7fd1f7c
DBZ-6603 Support for custom tags in the connector metrics
2023-07-13 09:52:33 +02:00
jcechace
5cb579bf12
DBZ-6525 Removing runtime dependency on core for testcontainers module in order to prevent logback.xml downstream conflicts
2023-07-11 12:07:04 +02:00
Chris Cranford
6a41761f85
DBZ-6573 Restore spacing
2023-06-30 09:39:32 +02:00
tjwornjs
2767f469aa
DBZ-6573 add max retries config to common config
2023-06-30 09:39:32 +02:00
bdbene
6f29ce08ef
DBZ-6605 Fix DataCollections for snapshot completion notification
...
DBZ-6605 Use new 'scanned_collection' field for snapshot completion signal
Updated the docs for fixed snapshot completion notification
2023-06-27 12:01:26 +02:00
mfvitale
d5febcc054
DBZ-6417 Provide extensible signal action mechanism through SPI
2023-06-23 15:03:27 +02:00
mfvitale
9ed928a1c9
DBZ-6416 Improve tests stability
2023-06-22 13:22:34 +02:00
mfvitale
34e28ac52d
DBZ-6416 Add initial snapshot notifications
2023-06-22 13:22:34 +02:00
mfvitale
7d28833ba8
DBZ-6588 Skip tombstone record on HeaderToValue SMT
2023-06-21 14:11:44 +02:00
jcechace
8f70de42c1
DBZ-6578 Cleaning up connection string propagation between discovery and tasks
2023-06-21 12:22:23 +02:00
mfvitale
91152c4c5d
DBZ-6559 Add test to cover topic name strategy
2023-06-16 11:48:34 +02:00
Jiri Pechanec
7f81f4bc6b
DBZ-6559 Handle keeping the first char as exception
2023-06-16 11:48:34 +02:00
mfvitale
87217a1474
DBZ-6559 Restore backward compatibility for field that stars with a digit
2023-06-16 11:48:34 +02:00
Jiri Pechanec
067683e56a
DBZ-6076 Consistent config for all connectors
2023-06-15 11:55:28 +02:00
Anil Dasari
0431a8b9fc
DBZ-6076: Add support for custom SourceInfoStructMaker for adding new fields to source field
2023-06-15 11:55:28 +02:00
Chris Cranford
ef8059f3ff
DBZ-6528 Fix Oracle NPE when storing only captured table schemas with signals
2023-06-14 10:48:19 +02:00
mfvitale
b7eb4b5c62
DBZ-6543 Fix NPE due to optional fields in PartitionRouting SMT
2023-06-14 07:22:27 +02:00
Andrei Isac
573a91be07
DBZ-6533: fix table order on partial match when snapshotting
2023-06-12 15:40:03 +02:00
kate
24d3425fdf
DBZ-6552 Notify when incremental snapshot fails
2023-06-12 15:14:40 +02:00
mfvitale
3aaca16024
DBZ-6543 Fix NPE due to optional fields in PartitionRouting SMT
2023-06-09 13:47:18 +02:00
Vojtech Juranek
b795519514
DBZ-6529 Use Murmur hash for PartitionRouting SMT
...
Add option to use better hash function than default Java Object::hash
function to get better hashes which would be more equally spred over
the hash space and thus more equally over the Kafka partitions.
To preserve backward compatibility, previous Java `hashCode` function
is used a default.
Add Murmur3 hash function for computing hashes of the fields.
Murmur3 implementation is taken from from Infinispan project code base.
To allow users eventually use their own make `computePartition()`
protected so it can be overriden in the subclasses.
2023-06-07 11:48:30 +02:00
mfvitale
61e75cc00d
DBZ-6424 Move JMX register/unregister code from Metrics to JmxUtils
2023-06-07 07:36:58 +02:00
mfvitale
b0b968fcce
DBZ-6424 Move JMX register/unregister code from Metrics to JmxUtils
2023-06-07 07:36:58 +02:00
mfvitale
394a9e32d8
DBZ-6424 Move JMX register/unregister code from Metrics to JmxUtils
2023-06-07 07:36:58 +02:00
mfvitale
7338fde84e
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
...
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
DBZ-6424 Emit JMX Notification when Debezium generate its own notification
2023-06-07 07:36:58 +02:00
mfvitale
769d9361e7
DBZ-6424 Add JMX Notification channel
...
This channel permits to send Debezium notification to JMX bean
DBZ-6424 Add JMX Signal channel
This channel permits to send signal to Debezium through the JMX operation
2023-06-07 07:36:58 +02:00
mfvitale
f93affc3ca
DBZ-6511 Add java docs for SignalProcessor getSignalChannel
method
2023-06-05 17:33:07 +02:00
mfvitale
ba8a3e5062
DBZ-6511 Expose reset method on SignalChannelReader to reset processed signals
2023-06-05 17:33:07 +02:00
mfvitale
4d56bbd15a
DBZ-6511 Restore Kafka signal offset only when KafkaSignalChannel is enabled
2023-06-05 17:33:07 +02:00
Ronak Jain
3f3967c2bb
DBZ-6366: Code Improvements for skip.messages.without.change
...
Addresses/Closes DBZ-6366
2023-06-05 14:50:25 +02:00
Jiri Pechanec
8a1cd95086
DBZ-6515 Add units to the description
2023-06-05 11:27:57 +02:00
mfvitale
155207dc9b
DBZ-6509 Add FileSignalChannel to META-INF/services
2023-05-31 14:39:35 +02:00
mfvitale
e1c1e65336
DBZ-6408 Add documentation for notifications
2023-05-30 17:17:52 +02:00
mfvitale
ed2157bfc9
DBZ-6486 ExtractNewRecordState skip fields that not exists on payload
2023-05-26 11:30:37 +02:00
Jiri Pechanec
b72f084eac
DBZ-6486 Check schema fields
2023-05-26 11:30:37 +02:00
mfvitale
08122371eb
DBZ-6486 ExtractNewRecordState can now take arbitrary fields from payload
2023-05-26 11:30:37 +02:00
mfvitale
7ba9f57035
DBZ-6496 Fix signal.poll.interval.ms
default value
2023-05-24 18:51:49 +02:00
mfvitale
dc00f64bbf
DBZ-6447 Incremental snapshot notification ids are now equal to the id sent in the execute-snapshot
signal.
2023-05-23 09:57:05 +02:00
harveyyue
016daf6fd8
DBZ-6490 Convert the field BigDecimal value to bytes failure of ApproximateStructSizeCalculator
2023-05-23 08:38:06 +02:00
Chris Cranford
d40da4f810
DBZ-6457 Use PDB for parallel snapshot connections
2023-05-19 14:04:14 +02:00
Ismail Simsek
2683b31394
DBZ-4027 implement FileSignalChannel
2023-05-15 15:34:43 +02:00
Vojtech Juranek
ee4ebaa957
DBZ-6439 Don't scan all the tables when loading schema
...
Currently we scan all the tables, which may result into a substantial
delay in initial snapshot when the database is very large. We need to
filter out tables which we are not interested in.
Add back table filter when loading schema of tables. As per comment of
this block of code, passing all tables and table filter should be faster
than passing only list of tables we are interested in.
2023-05-15 14:11:47 +02:00
Vojtech Juranek
f4596fa2e0
DBZ-6460 Enable setting log level in tests via log interceptor
2023-05-15 14:11:47 +02:00
mfvitale
85278e76af
DBZ-6462 Stop SignalProcessor after test execution in SignalProcessorTest
2023-05-15 08:51:02 +02:00
mfvitale
e41bbe7191
DBZ-6462 Schedule SignalProcessor task with a more relaxed scheduling. Removes parallelStreams while processing signalChannelReaders
2023-05-15 08:51:02 +02:00
mfvitale
7429b4e6f1
DBZ-1973 Add incremental snapshot notifications
2023-05-10 12:22:02 +02:00
mfvitale
811b800f55
DBZ-1973 Add support for Notifications
...
DBZ-1973 Add more tests for other connectors
DBZ-1973 Add send method with offset parameter
DBZ-1973 Instantiate NotificationService in the task class
DBZ-1973 Instantiate NotificationService in the task class
DBZ-4027 Move specific sink channel configuration to SinkNotificationChannel
DBZ-4027 Remove not used SPI file
DBZ-1973 Moved SPI file definition to debezium-core
DBZ-1973 Rename KafkaNotificationChannel to more generic SinkNotificationChannel
DBZ-1973 Code refactor
DBZ-1973 Improve configuration property description
DBZ-1973 Improve test
DBZ-1973 Add close method to NotificationChannel
DBZ-1973 Implement KafkaNotificationChannel
DBZ-1973 Add NotificationService and LogNotificationChannel
2023-05-10 12:22:02 +02:00
mfvitale
0ca87d8031
DBZ-4027 During event dispatch process only signal from source channel
2023-05-05 13:35:40 +02:00
mfvitale
952340286e
DBZ-4027 Make signalling channel configurable
...
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Add an Incremental snapshot test with kafka signaling
DBZ-4027 Code style
DBZ-4027 Make SignalPayload more generic and extensible
DBZ-4027 Rename DatabaseSignalChannel to SourceSignalChannel
DBZ-4027 Improve logging
DBZ-4027 Moved SPI file definition to debezium-core
DBZ-4027 Move SignalProcessor synchronization point to be processed only when a signal cdc event arrives.
DBZ-4027 Add EventDispatcher constructor without signalProcessor for spanner connector
DBZ-4027 Fix NPE
DBZ-4027 Fix NPE
DBZ-4027 Formatting
DBZ-4027 Correctly manage signal on not supported connector
DBZ-4027 Use the correct MongoDbOffset
DBZ-4027 Correctly initialize offset for Oracle and SqlServer connectors
DBZ-4027 Register SPI implementations
DBZ-4027 Improve SignalProcessor instantiation
DBZ-4027 Pass source info in case of SchemaChanges action
DBZ-4027 Manage close event in a synchronous way
DBZ-4027 Correctly init offset context also in case of snapshot mode 'never'
DBZ-4027 Fix MySqlMetricsIT test
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Move KafkaSignalChannel to core
DBZ-4027 Set pass offset context after initial snapshot to SignalProcessor
DBZ-4027 Pass OffsetContext to signal processor
DBZ-4027 Pass CommonConnectorConfig to SignalChannelReader init method
DBZ-4027 Move Incremental snapshot window actions to dedicated package
DBZ-4027 Align SignalsIT test with new code
DBZ-4027 Fix SignalsIT test
DBZ-4027 Fix SignalProcessor scheduling
DBZ-4027 Moved DatabaseSignalChannel and SignalChannelReader to dedicated package
DBZ-4027 Start SignalProcessor from ChangeEventSourceCoordinator
DBZ-4027 Create SignalProcessor and renamed Signal to DatabaseSignalChannel
DBZ-4027 Initial refactoring of signal feature
2023-05-05 13:35:40 +02:00
Jiri Pechanec
58ef4f0b98
DBZ-6157 Disable potentially dangerous MySQL JDBC props
2023-05-04 11:54:04 +02:00