alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
6,518 Commits 3 Branches 0 Tags 51 MiB
1355067ef0
Commit Graph

1343 Commits

Author SHA1 Message Date
Jiri Pechanec
8f5bb31eab DBZ-3700 Strings a bytes only when converters present 
When database was used with non-UTF8 charset the snapshot was sending
byte array with UTF-8 bytes but MySQL converter was using database based
charset to convert it to String leading to malformed data.
It is necessary to convert strings using the database charset but it
might be a problem in case of non-charset matching.
The current solution
1) Uses strings by default
2) Switches to byte array only when converters are present to cover
original use case
3) If the conversion fails it switches back to string
 2022-01-19 16:20:32 +01:00
Jiri Pechanec
8fd3243a86 DBZ-2224 Test logging based on logback 2022-01-18 11:24:21 +01:00
Sergei Morozov
17fba7258b DBZ-4541: Promote failure to register metrics to exception 2022-01-14 19:50:57 +01:00
Sergei Morozov
6d97433d18 DBZ-4541: Remove logger from Metrics register/unregister signatures 2022-01-14 19:50:57 +01:00
Chris Cranford
610e8eba73 DBZ-4548 Fix DatabaseVersionResolver comparison logic 2022-01-13 13:29:34 -05:00
Sergei Morozov
0329859cda DBZ-4459: Introduce interfaces and default implementations for change event source metrics 2022-01-12 14:32:19 +01:00
Gunnar Morling
f0b4a5ba9f DBZ-4351 Misc. clean-up 2022-01-06 19:03:46 +01:00
harveyyue
291e8ac221 DBZ-4351 Add create/update/delete event seen metrics for monitor upstream dml operation 2022-01-06 19:03:46 +01:00
harveyyue
831217053c DBZ-4503 Drop the primary key column getting exception 2022-01-03 10:57:07 +01:00
harveyyue
3670b0c6ef DBZ-4497 Parse multiple signed/unsigned keyword from ddl statement failed 2022-01-03 10:47:03 +01:00
Gunnar Morling
92665f1799 DBZ-4472 Improving log output 2021-12-20 15:58:14 +01:00
Jiri Pechanec
d13e6f546c DBZ-4467 Skip unsupported test for Db2 2021-12-20 11:29:30 +01:00
Debezium Builder
67c9663c98 [maven-release-plugin] prepare for next development iteration 2021-12-16 10:11:29 +00:00
Debezium Builder
8d532d895f [maven-release-plugin] prepare release v1.8.0.Final 2021-12-16 10:11:28 +00:00
Chris Cranford
f99ad7512f DBZ-3940 Add connector-driven EventRouter configuration 2021-12-13 13:52:31 +01:00
Jiri Pechanec
740b6b21f6 DBZ-4298 Support non-JDBC logical values in keys for incremental snapshots 
Incremental snapshot source gets the PK values after they are processed
by value converter. This is usally correct solution. The values are
passed to chunk queries. There are few datatypes like unconstrained
NUMERIC in PostgreSQL that must be converted to a Debezium specific
class to keep and enrich the value. Such class is not known to JDBC
driver. This commit intorduce a marker interface that enables the code
to identify the value not recognized by JDBC and provides a method to
obtain the original value.
 2021-12-13 12:29:21 +01:00
Gunnar Morling
0023cb10a5
DBZ-4393 Create a Debezium Schema Generator for Debezium connectors 
* added an API generator for Debezium connectors and static API definitions for connectors in a separate module
* added Maven plug-in
* added GH workflow for debezium-schema-generator

Co-authored-by: rkerner <rkerner.mobil@gmail.com>
Co-authored-by: Anisha Mohanty <anishamohanty23@gmail.com>
 2021-12-10 12:35:09 +01:00
Gunnar Morling
73cfe71342 DBZ-4423 Micro-optimizing Strings::duration() 2021-12-10 11:30:18 +01:00
Debezium Builder
f53e6c00df [maven-release-plugin] prepare for next development iteration 2021-12-09 20:16:04 +00:00
Debezium Builder
826837701a [maven-release-plugin] prepare release v1.8.0.CR1 2021-12-09 20:16:04 +00:00
Jiri Pechanec
b1dcfcb0b8 DBZ-3342 Incremental snapshot support for MongoDB 2021-12-09 10:03:41 +01:00
Tom Billiet
ec9f122e0a DBZ-4385 Truncate support for Oracle 2021-12-08 15:00:53 -05:00
Gunnar Morling
1f8fd22685 DBZ-4341 Improving logging; 
Also fixing incorrect JavaDoc.
 2021-12-08 13:12:54 +01:00
Chris Cranford
8f007e08a2 DBZ-4388 Don't set default value if resolution fails 2021-12-07 16:38:22 +01:00
Debezium Builder
b5439b54bf [maven-release-plugin] prepare for next development iteration 2021-11-30 09:38:47 +00:00
Debezium Builder
125f6e5aa2 [maven-release-plugin] prepare release v1.8.0.Beta1 2021-11-30 09:38:47 +00:00
Jiri Pechanec
2789dd610a DBZ-4272 Throw exception for non matching schema 2021-11-30 09:28:44 +01:00
Jiri Pechanec
5e40e1cdf5 DBZ-4272 Skip connector test using annotations 2021-11-30 09:28:44 +01:00
Jiri Pechanec
358eb0379b DBZ-4272 Disable tests for SQL Server 2021-11-30 09:28:44 +01:00
Chris Cranford
0a10d5b484 DBZ-4272 Trigger schema refresh before snapshot; only for PostgreSQL 2021-11-30 09:28:44 +01:00
Chris Cranford
3a8d1ff838 DBZ-4272 Incrementally snapshot after schema change without NPE 2021-11-30 09:28:44 +01:00
Gunnar Morling
2157256398 DBZ-4364 Awaiting table metadata to become queryable 2021-11-29 09:23:18 +01:00
Jiri Pechanec
1c5f3b2330 DBZ-4364 Make byte[] and ByteBuffer comparable 2021-11-29 09:23:18 +01:00
Jiri Pechanec
4a2a60db74 DBZ-4364 Fix missing method call 2021-11-29 09:23:18 +01:00
Gunnar Morling
560ef2babd DBZ-2363 Misc. adjustments; 
* Adding Strings::isBlank()
* Simplifying test
* Using long in cases where we know it's present
* Removing some unused code
* Testing binary handling mode
* Avoiding TX id value overflow
* Assigning names to schemas to avoid issues as in DBZ-650
 2021-11-26 14:29:15 +01:00
lairen
a0898ed440 DBZ-2363 Adding support for pg_logical_emit messages in Postgres connector for the pgoutput plugin 2021-11-26 14:29:15 +01:00
Jacob Gminder
f7f8748062 DBZ-2975: Move the poll interval logic to the ChangeEventSourceCoordinator. 2021-11-22 12:30:31 +01:00
Gunnar Morling
a7c6d18489 DBZ-4309 Removing legacy logger 2021-11-19 08:04:44 +01:00
Andrey Yegorov
b6d2c41fb0 DBZ-4309 Avoid calling 'LoggerFactory.getLogger(getClass())' for each new instance of the RelationalChangeRecordEmitter 2021-11-18 22:37:51 +01:00
Chris Cranford
2730187a4b DBZ-4012 Deprecate table.field.event.payload.id 2021-11-18 21:45:41 +01:00
Chris Cranford
ab86cd18a9 DBZ-4012 Suggested changes 2021-11-18 21:45:41 +01:00
Chris Cranford
349b9690f4 DBZ-4012 Suggested changes 2021-11-18 21:45:41 +01:00
Chris Cranford
0890202539 DBZ-4012 Remove incubating from EventRouter 2021-11-18 21:45:41 +01:00
Chris Cranford
0df5c6219b DBZ-4012 Bubble up SMT validation failures to Kafka Connect 2021-11-18 21:45:41 +01:00
Chris Cranford
970792bc40 DBZ-4012 Deprecate/Rename debezium namespace config options 2021-11-18 21:45:41 +01:00
Chris Cranford
7a4140c6aa DBZ-4012 ActivateTracingSpan is pass-through when OpenTracing unavailable 2021-11-18 21:45:41 +01:00
Gunnar Morling
ca95b424fb DBZ-3528 Misc. clean-up 2021-11-17 16:02:06 +01:00
Sungho Hwang
78c92c3772 DBZ-3528 turned access modifier of jsonNodeToSchemaBuilder() back to private 2021-11-17 16:02:06 +01:00
Gunnar Morling
03433b663b DBZ-3528 Misc. fixes 2021-11-17 16:02:06 +01:00
Sungho Hwang
7c95636668 DBZ-3528 Chaged variable naming for better readability 2021-11-17 16:02:06 +01:00
Sungho Hwang
c9eb0f9157 DBZ-3528 Refactored several parts according to the feedbacks 
A delegate class for common logic between two Outbox Event Routers has been created.
Documentation has been fixed.
The configuration properties for MongoDB Outbox Event Router have been changed to use existing Outbox Event Router's.
 2021-11-17 16:02:06 +01:00
Sungho Hwang
0b9fc88ea6 DBZ-3528 Applied the code style rules 2021-11-17 16:02:06 +01:00
Sungho Hwang
2c279b07f6 DBZ-3528 Wrote MongoDB Outbox Event Router with documentation and its unit & integration tests 2021-11-17 16:02:06 +01:00
Chris Cranford
e3da449303 DBZ-4276 Expose unavailable value placeholder for PG and Oracle only 2021-11-16 20:29:59 +01:00
Chris Cranford
ca17352221 DBZ-4276 Unify unavailable value placeholder handling 
Deprecate the `toasted.value.placeholder` option in PostgreSQL in favor
of the new `unavailable.value.placeholder` option that is shared by all
relational connectors.
 2021-11-16 20:29:59 +01:00
Kate
44073cf7d8 DBZ-4196 Support schema changes during incremental snapshot 2021-11-16 14:56:06 +01:00
camilesing
dfabe8cda8 DBZ-4275 fix AbstractDatabaseHistory typo 2021-11-14 19:21:37 +01:00
Debezium Builder
f93e901dce [maven-release-plugin] prepare for next development iteration 2021-11-11 09:21:59 +00:00
Debezium Builder
895a16bf19 [maven-release-plugin] prepare release v1.8.0.Alpha2 2021-11-11 09:21:59 +00:00
Chris Cranford
85101f0ad7 DBZ-3611 Avoid NPE with database include/exclude lists 2021-11-11 09:24:05 +01:00
Kate
992ae537c8 DBZ-4244 Quote signal table name in MySQL connector 2021-11-10 10:18:26 +01:00
Chris Cranford
5afda302ec DBZ-3710 Javadoc updates 2021-11-10 09:43:25 +01:00
Chris Cranford
5ec9381702 DBZ-3710 Unify DefaultValueMapper (Oracle/SQL Server/PostgreSQL) 2021-11-10 09:43:25 +01:00
Chris Cranford
343318b5d4 DBZ-3710 Support Oracle column default values 2021-11-10 09:43:25 +01:00
harveyyue
aa3ae05e22 DBZ-4107 Incremental snapshot doesn't work without primary key 2021-11-05 13:31:26 +01:00
Gunnar Morling
1226c3b2ab DBZ-4029 Misc. clean-up; 
* Exposing heartbeatInterval() on CommonConnectorConfig
* Closing heartbeat connection
* Making test more resilient
* Correct member ordering and visibility
* Doc fixes
 2021-11-05 13:27:09 +01:00
ahodavdekar
91b7ef2c36 DBZ-4029 Adding support for heartbeat action queries to MySQL connector 2021-11-05 13:27:09 +01:00
Gunnar Morling
b78e318399 DBZ-3966 Expanding test 2021-11-04 10:47:24 +01:00
jiabao.sun
2987e96c44 DBZ-3966 Serializing hasDefaultValue in JsonTableChangeSerializer 2021-11-04 10:47:17 +01:00
Gunnar Morling
d1cae3f3b7 DBZ-3966 Making defaultValueExpression() return Optional<String> from editor too 2021-11-04 10:47:17 +01:00
jiabao.sun
24ead16c24 DBZ-3966 Remove defaultValue and keep defaultValueExpression 2021-11-04 10:47:17 +01:00
jiabao.sun
77c4113697 DBZ-3966 JsonTableChangeSerializer support serialization for defaultValue and enumValues 2021-11-04 10:47:17 +01:00
Kate
a645d0bc7e DBZ-4197 Process transaction started/committed in R/O snapshot 2021-11-04 06:19:58 +01:00
ahodavdekar
1d81b6db19 DBZ-4077 Exposing the transaction metadata topic name as a config option 2021-11-02 06:07:05 +01:00
Gunnar Morling
fd06a4e2a4 DBZ-2975 Using Offsets in recover() contract; 
Keeping legacy method until the legacy MySQL connector implementation has been removed.
 2021-11-01 10:17:49 +01:00
Mike Kamornikov
16c07e915b DBZ-2975: Recover database schema from multiple partitions 2021-11-01 10:17:49 +01:00
Debezium Builder
2026a48302 [maven-release-plugin] prepare for next development iteration 2021-10-27 06:47:27 +00:00
Debezium Builder
97106c1fd9 [maven-release-plugin] prepare release v1.8.0.Alpha1 2021-10-27 06:47:26 +00:00
Jiri Pechanec
a1084360da DBZ-435 Too fast delete can prevent from obtaining the full document 2021-10-26 18:13:16 +02:00
Jiri Pechanec
cb36b575db DBZ-435 Support for Change Streams capture 2021-10-26 18:13:16 +02:00
Sungho Hwang
54d5c16b69 DBZ-4191 Fixed a variable name in an unit test of Outbox Event Router to enhance readability. 2021-10-22 07:57:24 +02:00
harveyyue
90e9bea93f DBZ-4000 Support parse table and columns comment 2021-10-20 15:19:31 +02:00
Anisha Mohanty
2eae7a742c DBZ-4045 Use listener props for deprecated host & port props 2021-10-20 11:51:34 +02:00
Gunnar Morling
3d093e2a3b DBZ-4164 Import ordering 2021-10-18 14:45:00 +02:00
lujiefsi
12ccc24e17
DBZ-4164 Fixing resource leak due to Files::lines 2021-10-18 14:31:39 +02:00
Juan Fiallo
346e0f3af2 DBZ-3517 Making database.user optional for SQL Server. 
SQL Server doesn't require username/password when using Kerberos authentication, which can be configured using pass-through database properties. Changing the database.user config to be optional for SQL Server Connector to reflect that.
 2021-10-18 12:06:56 +02:00
Juan Fiallo
5ac93ac20e DBZ-3517 Making database.user optional for SQL Server. 
SQL Server doesn't require username/password when using Kerberos authentication, which can be configured using pass-through database properties. Changing the database.user config to be optional for SQL Server Connector to reflect that.
 2021-10-18 12:06:56 +02:00
Juan Fiallo
e041b599d8 DBZ-3517 Making database.user optional for SQL Server. 
SQL Server doesn't require username/password when using Kerberos authentication, which can be configured using pass-through database properties. Changing the database.user config to be optional for SQL Server Connector to reflect that.
 2021-10-18 12:06:56 +02:00
Jiri Pechanec
0ccc83db21 DBZ-3860 Provide correct boundaries for multi PK tables 2021-10-14 17:26:50 +02:00
Gunnar Morling
26f13d83d5 DBZ-3993 Reflectively invoking Runtime.version(); 
As that API is only available from Java 9 onwards, calling it will cause
a compilation failure when compiling with --release.
 2021-10-14 09:46:06 -04:00
Gunnar Morling
6b3d2c3b76 DBZ-3993 Formatting fix 2021-10-13 10:59:23 +02:00
Gunnar Morling
c3eed6051b DBZ-3993 Skipping object size based test on Java 16+ 2021-10-13 10:59:23 +02:00
Sergei Morozov
4a03c5df17 DBZ-2975: Include database name into topic and schema names 
Co-authored-by: Mike Kamornikov <mikekamornikov@gmail.com>
 2021-10-12 15:40:02 +02:00
Jiri Pechanec
657c5540b3 DBZ-4083 Add filter to MongoDB cloud converter 2021-10-11 11:19:30 +02:00
Jiri Pechanec
aec2080415 DBZ-3895 Config returns Patterns not Strings 2021-10-11 10:57:33 +02:00
Jiri Pechanec
68f148c3f3 DBZ-3895 Make snapshot collection list matching case insensitive 2021-10-11 10:57:33 +02:00
Chris Cranford
abc3bb42d6 DBZ-4106 Don't continuously reconfigure DatabaseHistory 2021-10-05 20:15:58 +02:00
Gunnar Morling
62715c4e1d DBZ-4033 Doc update 2021-10-05 10:54:01 +02:00
unalsurmeli
9a13d80074 DBZ-4033 A new hash version has been added so that it can be used to have the same hash value in different places; 
In V2, the byte array only has the actual value, unlike the serialization based V1 implementation which exposed Java's internal serialization format.
 2021-10-05 10:52:30 +02:00
Vivek Wassan
82dad32e2a DBZ-4078 Checking connection is valid before commit in readChunk() 2021-10-04 10:00:27 +02:00