alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
7,724 Commits 3 Branches 0 Tags 51 MiB
f437788325
Commit Graph

1213 Commits

Author SHA1 Message Date
Gunnar Morling
560ef2babd DBZ-2363 Misc. adjustments; 
* Adding Strings::isBlank()
* Simplifying test
* Using long in cases where we know it's present
* Removing some unused code
* Testing binary handling mode
* Avoiding TX id value overflow
* Assigning names to schemas to avoid issues as in DBZ-650
 2021-11-26 14:29:15 +01:00
Gunnar Morling
b85a9d2f20 DBZ-2363 Extracting separate test class for logical decoding message tests 2021-11-26 14:29:15 +01:00
Lairen Hightower
450998b49e DBZ-2363 Replace logicalDecodingMessageHandlingMode with a prefix filtering configuration 2021-11-26 14:29:15 +01:00
Gunnar Morling
fcfc9bc785 DBZ-2363 Misc. adjustments; 
* Removing unused method
* Unifying update() handling
* Member ordering
* Avoiding NPE in error handler
 2021-11-26 14:29:15 +01:00
lairen
a0898ed440 DBZ-2363 Adding support for pg_logical_emit messages in Postgres connector for the pgoutput plugin 2021-11-26 14:29:15 +01:00
Jiri Pechanec
dc3ee488f1 DBZ-4343 Use schema component for incremental snapshot TableId 2021-11-24 10:25:33 +01:00
Andrey Yegorov
b6d2c41fb0 DBZ-4309 Avoid calling 'LoggerFactory.getLogger(getClass())' for each new instance of the RelationalChangeRecordEmitter 2021-11-18 22:37:51 +01:00
Vojtech Juranek
c0a65743f8 DBZ-1042 Allow Postgres connector to use numeric slot names 
To be able to use numeric slot names, the slot name has to be quoted.
This may be consider as s bug in Postgres driver as Debezium passes the
slot name as string. To workaround this issue, quote the slot name on
the Debezium side.
 2021-11-18 10:27:48 +01:00
Vojtech Juranek
1ca917a428 DBZ-1042 Allow numeric slot names 2021-11-18 10:27:48 +01:00
Vojtech Juranek
50da5e31dc DBZ-1931 Allow to specify number of fraction digits for the money type 2021-11-17 09:30:15 +01:00
Vojtech Juranek
6f286fe50c DBZ-1931 Reflect decimal.handling.mode in money conversion 2021-11-17 09:30:15 +01:00
Chris Cranford
e3da449303 DBZ-4276 Expose unavailable value placeholder for PG and Oracle only 2021-11-16 20:29:59 +01:00
Chris Cranford
ca17352221 DBZ-4276 Unify unavailable value placeholder handling 
Deprecate the `toasted.value.placeholder` option in PostgreSQL in favor
of the new `unavailable.value.placeholder` option that is shared by all
relational connectors.
 2021-11-16 20:29:59 +01:00
Kate
44073cf7d8 DBZ-4196 Support schema changes during incremental snapshot 2021-11-16 14:56:06 +01:00
Debezium Builder
f93e901dce [maven-release-plugin] prepare for next development iteration 2021-11-11 09:21:59 +00:00
Debezium Builder
895a16bf19 [maven-release-plugin] prepare release v1.8.0.Alpha2 2021-11-11 09:21:59 +00:00
Kate
992ae537c8 DBZ-4244 Quote signal table name in MySQL connector 2021-11-10 10:18:26 +01:00
Chris Cranford
5ec9381702 DBZ-3710 Unify DefaultValueMapper (Oracle/SQL Server/PostgreSQL) 2021-11-10 09:43:25 +01:00
harveyyue
aa3ae05e22 DBZ-4107 Incremental snapshot doesn't work without primary key 2021-11-05 13:31:26 +01:00
Gunnar Morling
1226c3b2ab DBZ-4029 Misc. clean-up; 
* Exposing heartbeatInterval() on CommonConnectorConfig
* Closing heartbeat connection
* Making test more resilient
* Correct member ordering and visibility
* Doc fixes
 2021-11-05 13:27:09 +01:00
ahodavdekar
91b7ef2c36 DBZ-4029 Adding support for heartbeat action queries to MySQL connector 2021-11-05 13:27:09 +01:00
jiabao.sun
24ead16c24 DBZ-3966 Remove defaultValue and keep defaultValueExpression 2021-11-04 10:47:17 +01:00
jiabao.sun
77c4113697 DBZ-3966 JsonTableChangeSerializer support serialization for defaultValue and enumValues 2021-11-04 10:47:17 +01:00
Gunnar Morling
9a4d0ad363 DBZ-4137 Test clean-up 2021-11-02 16:50:26 +01:00
Andrey Yegorov
b174927a52 DBZ-4137 Postgres defaults handling misses decimal mode configuration and cannot handle numerics with default NULL values 2021-11-02 16:50:26 +01:00
ahodavdekar
1d81b6db19 DBZ-4077 Exposing the transaction metadata topic name as a config option 2021-11-02 06:07:05 +01:00
Anisha Mohanty
9f7ede0e06 DBZ-3626 Renaming master branches to main 2021-10-28 09:33:32 +02:00
Debezium Builder
2026a48302 [maven-release-plugin] prepare for next development iteration 2021-10-27 06:47:27 +00:00
Debezium Builder
97106c1fd9 [maven-release-plugin] prepare release v1.8.0.Alpha1 2021-10-27 06:47:26 +00:00
Gunnar Morling
33764b2adc DBZ-4150 Using set instead of list 2021-10-19 12:32:16 +02:00
Hossein Torabi
6027d762a6 DBZ-4150 Trim numerical defaultValue before converting postgres-connector 2021-10-19 12:27:26 +02:00
Jiri Pechanec
0ccc83db21 DBZ-3860 Provide correct boundaries for multi PK tables 2021-10-14 17:26:50 +02:00
Anisha Mohanty
310a0b6ef3 DBZ-4145 Skip failing test for wal2json 2021-10-14 13:17:08 +02:00
Gunnar Morling
c3eed6051b DBZ-3993 Skipping object size based test on Java 16+ 2021-10-13 10:59:23 +02:00
Sergei Morozov
4a03c5df17 DBZ-2975: Include database name into topic and schema names 
Co-authored-by: Mike Kamornikov <mikekamornikov@gmail.com>
 2021-10-12 15:40:02 +02:00
Gunnar Morling
015b219c58 DBZ-4046 Some testing clean-up; 
* Fixing mixed up order of expected and actual value
* Using more expressive var names
 2021-10-11 06:04:42 +02:00
Judah Rand
5d23b30f67 DBZ-4046 Update isVariableScaleDecimal 
This puts the method in line with
https://github.com/pgjdbc/pgjdbc/issues/2188
where expected value was changed from `131089` to `0`.
However, for numeric arrays the issue remains and so
`131089` remains the correct value for them.
 2021-10-08 10:42:21 +02:00
Jiri Pechanec
d5eafd3ecf DBZ-3919 Add logs to test 2021-10-06 09:36:21 +02:00
Jiri Pechanec
dae384baf4 DBZ-4081 Close leaking connections 2021-10-05 12:52:03 +02:00
Jiri Pechanec
f910fc930b [release] Switch to 1.8 snapshot 2021-10-01 14:50:16 +02:00
Debezium Builder
12b0ae9046 [maven-release-plugin] prepare for next development iteration 2021-10-01 13:26:33 +02:00
Debezium Builder
652162fd85 [maven-release-plugin] prepare release v1.7.0.Final 2021-10-01 13:26:33 +02:00
Jiri Pechanec
6d0760df45 DBZ-4060 Remove race condition 2021-10-01 11:25:14 +02:00
Judah Rand
6ad791bedc DBZ-4060 Add default value mapper for smallserial 2021-10-01 11:25:14 +02:00
Jiri Pechanec
a2c6058404 DBZ-4057 Initialize Postgres schema before incremental snapshot restart 2021-09-30 09:10:49 +02:00
Debezium Builder
bec0b6b67e [maven-release-plugin] prepare for next development iteration 2021-09-23 06:31:25 +00:00
Debezium Builder
cf8da39725 [maven-release-plugin] prepare release v1.7.0.CR2 2021-09-23 06:31:25 +00:00
Naveen Kumar KR
d07643d60f DBZ-3887 PostgresMetricsIT Test fix 
* Remove redundant oneRecordInQueue test
* Rename test method from twoRecordsInQueue -> oneRecordInQueue to reflect the actual assertion done
 2021-09-22 19:12:21 +02:00
Debezium Builder
c15e0b303b [maven-release-plugin] prepare for next development iteration 2021-09-16 08:10:45 +00:00
Debezium Builder
23d9790121 [maven-release-plugin] prepare release v1.7.0.CR1 2021-09-16 08:10:44 +00:00
René Kerner
b92e5c5efe
DBZ-3943 Add ConfigDef unit tests to verify all Debezium Field instances have a proper documentation/description set; 
Also adding missing descriptions.
 2021-09-09 13:35:41 +02:00
Gunnar Morling
7c150de943 DBZ-2525 Don't qualify selected columns with table name; 
It shouldn't be needed as we select from a single table, and it caused the columns to not be found for the Db2 connector.
 2021-08-31 17:39:36 +05:30
Anisha Mohanty
153e0920e1 DBZ-2525 Generifying exclusion of columns from snapshotting in all relational connectors; 
* Using RelationalDatabaseSchema in RelationalSnapshotChangeEventSource for consistency.
* Adding test for table/column names which are keywords
 2021-08-30 12:20:56 +02:00
Gunnar Morling
11ed86a807 DBZ-3889 Reducing logging of serializers 2021-08-30 12:20:55 +02:00
Jiri Pechanec
82b151d0b9 DBZ-3935 Invalid validation of truncate handling mode 2021-08-30 09:47:55 +02:00
Debezium Builder
2ce6bcb2ab [maven-release-plugin] prepare for next development iteration 2021-08-25 05:20:56 -04:00
Debezium Builder
d3a7efe423 [maven-release-plugin] prepare release v1.7.0.Beta1 2021-08-25 05:20:56 -04:00
rkerner
52333596de DBZ-3364 add additional metadata to fields for API spec generator for the Debezium UI 
part of https://issues.redhat.com/browse/DBZ-3364
DBZ-2653 User rollback from parent class
 2021-08-23 17:57:38 +02:00
Gunnar Morling
36af61b789 DBZ-3889 Moving ContainerImageVersions to core for broader re-use and using it in two more places 2021-08-23 12:34:03 +02:00
Gunnar Morling
0fedadd77e DBZ-3889 Avoiding some log noise 2021-08-23 12:34:03 +02:00
Naveen Kumar KR
9527843974 DBZ-3870 Performance Improvements in PostgresChangeRecordEmitter 
1) Reuse parsed TableId to avoid expensive PostgresSchema.parse() call
2) Immediate conditional return in synchronizeTableSchema() call
 2021-08-13 22:11:48 +02:00
Sergei Morozov
a40ad7132d DBZ-2975: Remove partition from offset context 2021-08-11 07:34:37 +02:00
Sergei Morozov
77e2ac8a12 DBZ-2975: Introduce io.debezium.pipeline.spi.Offsets 2021-08-11 07:34:37 +02:00
Sergei Morozov
c7ac5fca52 DBZ-2975: Move Partition to io.debezium.pipeline.spi 2021-08-11 07:34:37 +02:00
Jiri Pechanec
b0e0052a9c DBZ-3828 Update event LSN for commits 2021-08-10 20:41:37 +02:00
Gunnar Morling
85e6d262ea DBZ-3816 Wording changes 2021-08-09 10:08:47 +02:00
Jiri Pechanec
305c15132f DBZ-3816 Workaround for Kafka Connect bug 2021-08-09 10:08:47 +02:00
Gunnar Morling
bf9f7f5596 DBZ-3801 Allocating sequence list with correct size 2021-08-04 10:25:09 +02:00
Brennan Vincent
b03d9d1fa6 DBZ-3801: Address code review comments 2021-08-04 10:25:09 +02:00
Brennan Vincent
975f1690f2 DBZ-3801 Fix calculation of sequence numbers for Postgres 2021-08-04 10:25:09 +02:00
Debezium Builder
d3c4482d06 [maven-release-plugin] prepare for next development iteration 2021-07-30 02:39:08 -04:00
Debezium Builder
35dcfc5426 [maven-release-plugin] prepare release v1.7.0.Alpha1 2021-07-30 02:39:08 -04:00
Kate
9b66ab374a DBZ-3577 Support read-only MySQL incremental snapshots 2021-07-29 07:10:46 +02:00
Jiri Pechanec
b387775448 DBZ-3655 Connection liveness check triggers replication earlier 2021-07-27 11:07:52 -04:00
Jiri Pechanec
f46e0ca9a6 DBZ-3655 Commit check connection query 2021-07-27 15:32:35 +02:00
Gunnar Morling
09ebc833f3 DBZ-3655 Gracefully handling DB shutdown 2021-07-27 10:21:54 +02:00
Chris Cranford
9f5c49246f DBZ-3769 Remove TODO and change log message level 2021-07-27 09:31:51 +02:00
bep7520
9e9c2fc2ab DBZ-3769 Throw sqlexception from postgres message decoder 2021-07-27 09:31:51 +02:00
Naveen Kumar KR
7844b59c2f DBZ-3770 Optimize ObjectMapper initialization by replacing repeated initialization with one time initilization 
- Performance Improvement of approx 5% - 15%
 2021-07-21 14:44:09 -04:00
Chris Cranford
ed31ea9ffb DBZ-3668 Adjust ServiceLoader for SPI change 2021-07-03 16:31:52 +07:00
Gunnar Morling
b84c3c870d DBZ-3668 Extracting SPI package 2021-07-03 16:31:52 +07:00
Chris Cranford
7f7e23b110 DBZ-3668 Add ServiceLoader hook CloudEvents converter 2021-07-03 16:31:52 +07:00
Sergei Morozov
db105baeba DBZ-2975: Add partition awareness to source task components 
Co-authored-by: Mike Kamornikov <mikekamornikov@gmail.com>
 2021-07-03 10:56:34 +02:00
Gunnar Morling
5b76bf1b71 DBZ-3632 Pulling up getJdbcConfig() to RelationalDatabaseConnectorConfig 2021-07-01 15:11:01 +02:00
Jiri Pechanec
ee0bd2f7da DBZ-3685 Postgres keepalive thread uses correct name 2021-07-01 11:40:59 +02:00
Debezium Builder
9d3f6b2a86 [maven-release-plugin] prepare for next development iteration 2021-06-30 03:45:06 -04:00
Debezium Builder
c8c5fc4b56 [maven-release-plugin] prepare release v1.6.0.Final 2021-06-30 03:45:06 -04:00
Debezium Builder
ae053414ac [maven-release-plugin] prepare for next development iteration 2021-06-24 01:50:07 -04:00
Debezium Builder
af3f942a74 [maven-release-plugin] prepare release v1.6.0.CR1 2021-06-24 01:50:07 -04:00
Debezium Builder
48ca58a609 [maven-release-plugin] prepare for next development iteration 2021-06-10 06:03:12 -04:00
Debezium Builder
0961517859 [maven-release-plugin] prepare release v1.6.0.Beta2 2021-06-10 06:03:12 -04:00
Gunnar Morling
a2f927c2db DBZ-3608 Avoiding repeated closing actions 2021-06-10 07:39:49 +02:00
Gunnar Morling
205c63a56a DBZ-3608 Fixing version handling; 
- wrong minor version for Postgres
- minor/patch versions of -1 were not actually ignored in comparison
 2021-06-10 07:39:49 +02:00
Gunnar Morling
dc865cd82d DBZ-2975 Formatting 2021-06-09 16:02:33 +02:00
Gunnar Morling
1ce65fa628 DBZ-2975 Avoiding raw types in some places; 
Also parameterizing offset loader contract, so to avoid casts later on.
 2021-06-09 16:02:33 +02:00
Sergei Morozov
657a6d2542 DBZ-2975: Extract offset context from object states to method signatures 2021-06-09 16:02:33 +02:00
Gunnar Morling
2c5e10b848 DBZ-3596 Using single connection in PgOutputMessageDecoder 2021-06-09 13:41:43 +02:00
Gunnar Morling
4aef8669fb DBZ-3596 Removing unused "exported" flag 2021-06-09 13:41:43 +02:00
Gunnar Morling
7e664b18e5 DBZ-3596 Removing unused field AbstractMessageDecoder#filterBasedOnLsn and code leading to it 2021-06-09 13:41:43 +02:00
Jiri Pechanec
b59f6daac9 DBZ-3550 Use selected test containers deps 2021-06-08 14:49:04 +02:00
Gunnar Morling
41e93e6ce1 DBZ-2790 Avoiding function usage 2021-06-08 12:10:26 +02:00
Eric Slep
8a659ff8ab DBZ-2790 Docs and one final test edge case 2021-06-08 12:10:26 +02:00
Eric Slep
5dde3062b0 DBZ-2790 Handle pgoutput streaming refresh 2021-06-08 12:10:26 +02:00
Eric Slep
13ebeade71 DBZ-2790 Add default values to existing tests 2021-06-08 12:10:26 +02:00
Eric Slep
436b4697ba DBZ-2790 Tests for refresh behaviour, ensure defaults are not cleared on in-mem refresh 2021-06-08 12:10:26 +02:00
Eric Slep
d1917eeff6 DBZ-2790 Handle default values for date & time types 2021-06-08 12:10:26 +02:00
Eric Slep
f88f135c43 DBZ-2790 Add simple placeholders to schema for generated defaults 2021-06-08 12:10:26 +02:00
Eric Slep
de5fe2a520 DBZ-2790 Move TypeRegistry construction back to connection 2021-06-08 12:10:26 +02:00
Kevin Pullin
0462804467 DBZ-2790 Update debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/connection/PostgresDefaultValueConverter.java 
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
 2021-06-08 12:10:26 +02:00
Kevin Pullin
4b62e59400 DBZ-2790 Update debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/connection/PostgresDefaultValueConverter.java 
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
 2021-06-08 12:10:26 +02:00
Kevin Pullin
d2ab8baddf DBZ-2790 Implement default values for postgres 
Using the `SqlServerDefaultValueConverter` class as a starting point,
this commit implements default value support for postgres.
 2021-06-08 12:10:26 +02:00
gvaquez-ubi
03cb907d85 DBZ-3068 Allow several tries on createReplicationStream to get replication slot 2021-06-04 15:11:57 +02:00
Debezium Builder
ead4ee7226 [maven-release-plugin] prepare for next development iteration 2021-05-20 19:07:44 +02:00
Debezium Builder
9e16df7d4b [maven-release-plugin] prepare release v1.6.0.Beta1 2021-05-20 19:07:44 +02:00
Jiri Pechanec
a879b7b3e1 DBZ-3500 Re-use ResultSet to Object code 2021-05-20 08:06:37 +02:00
Jiri Pechanec
a626dc149f DBZ-3500 Provide common test for incremental snapshot testing 2021-05-20 08:06:37 +02:00
Gunnar Morling
a362760429 DBZ-2337 Removing superfluous log statement 2021-05-18 21:47:16 +02:00
Jiri Pechanec
5fbc181b5e DBZ-2337 All snapshots using exported mode 2021-05-18 21:44:08 +02:00
Jiri Pechanec
2a716d1b86 DBZ-3519 Avoid NPE in autoboxing 2021-05-18 10:32:18 +02:00
Debezium Builder
4dd3a146ca [maven-release-plugin] prepare for next development iteration 2021-05-06 02:40:15 -04:00
Debezium Builder
b3c5bb281c [maven-release-plugin] prepare release v1.6.0.Alpha1 2021-05-06 02:40:15 -04:00
Jiri Pechanec
5c722341e2 DBZ-3473 Extract incremental snapshot contract 2021-05-06 05:37:36 +02:00
Jiri Pechanec
7daae11600 DBZ-3473 Postgres has its own schema instance 2021-05-06 05:37:36 +02:00
Jiri Pechanec
39d21806eb DBZ-3473 Metrics enabled 2021-05-06 05:37:36 +02:00
Jiri Pechanec
c53785dde7 DBZ-3473 Incremental Snapshot support for MySQL 2021-05-06 05:37:36 +02:00
Jiri Pechanec
4e95056ddd DBZ-3473 Test stabilization 2021-05-06 05:37:36 +02:00
Jiri Pechanec
49c5a00bbf DBZ-3473 Share read JDBC value between regular and incremental snapshot 2021-05-06 05:37:36 +02:00
Jiri Pechanec
dd2bc6354b DBZ-3473 Pass the main clock form connector 2021-05-06 05:37:36 +02:00
Jiri Pechanec
260c32cf0e DBZ-3473 Resume parallel snapshot after connector restart 2021-05-06 05:37:36 +02:00
Jiri Pechanec
37626c0dfe DBZ-3473 Extract incremental snapshot context 2021-05-06 05:37:36 +02:00
Jiri Pechanec
07371c085b DBZ-3473 Watermark signals must be in separate tx 2021-05-06 05:37:36 +02:00
Jiri Pechanec
a52c536a68 DBZ-3473 First prototype of incremental snapshotting 2021-05-06 05:37:36 +02:00
Gunnar Morling
dc02b896cd DBZ-3449 Moving QA releated plug-ins to separate profile, so to avoid loading them at all 2021-04-28 15:14:05 +02:00
Gunnar Morling
2088c035b1 DBZ-3449 Adding "quick" profile 2021-04-26 06:09:14 +02:00
Hossein Torabi
5d6705c3fe DBZ-2699 implement skipped.operation in core 2021-04-08 11:59:08 +02:00
Debezium Builder
8a88ba9b45 [maven-release-plugin] prepare for next development iteration 2021-04-08 04:04:00 -04:00
Debezium Builder
74152afe74 [maven-release-plugin] prepare release v1.5.0.Final 2021-04-08 04:04:00 -04:00
Gunnar Morling
5425e6c5dc DBZ-3343 Re-using single shared SqlTypeMapper instance 2021-04-06 14:59:55 +02:00
JLDLaughlin
06b0475f17 DBZ-2911 Add additional LSN information to all sources 
Adds a new "sequence" field to all sources via the AbstractSourceInfo
class. Specifically, this field is required to deduplicate records
from PostgreSQL sources in O(1) time. The sequence field is a stringified
list of metadata. For PostgreSQL sources, this will include the last
committed LSN and the current LSN. For other sources, this will be NULL
until implemented. A new integration test was added for PostgreSQL.
 2021-03-26 09:19:17 +01:00
Debezium Builder
6b4d88bb51 [maven-release-plugin] prepare for next development iteration 2021-03-24 06:35:02 -04:00
Debezium Builder
c578715233 [maven-release-plugin] prepare release v1.5.0.CR1 2021-03-24 06:35:02 -04:00
Jiri Pechanec
495be2659b DBZ-2511 Dedicated logger for schema name adjuster 2021-03-18 14:42:34 +01:00
Jiri Pechanec
21f522217c DBZ-3033 Map OID to long value 2021-03-17 18:11:52 +01:00
Gunnar Morling
911b5e9a3b DBZ-3271 Unifying filter handling across relational connectors 2021-03-17 09:54:25 +01:00
Debezium Builder
f57057421b [maven-release-plugin] prepare for next development iteration 2021-03-12 04:38:17 -05:00
Debezium Builder
b57293013a [maven-release-plugin] prepare release v1.5.0.Beta2 2021-03-12 04:38:16 -05:00
Jiri Pechanec
123d9ef4eb DBZ-3213 Refactor Maven structure to deploy support modules 2021-03-11 16:14:15 +01:00
Jiri Pechanec
f23421920b DBZ-2919 Remove race condition 2021-03-10 19:12:34 +01:00
Jiri Pechanec
15c7b11a1f DBZ-3141 Change schema signal tested for Postgres 2021-03-10 19:12:34 +01:00
Jiri Pechanec
ff7cb44801 DBZ-3141 Offsets are propagated to signal actions 2021-03-10 19:12:34 +01:00
Jiri Pechanec
7fdad2ed49 DBZ-3141 Support for signalling table 2021-03-10 19:12:34 +01:00
Jiri Pechanec
9f2cffa21e DBZ-2885 Fix test 2021-03-09 17:32:01 +01:00
Anisha Mohanty
895d704d82 DBZ-2885 Handle misleading error message for filtered publication 2021-03-09 08:39:15 +01:00
Chris Cranford
04849734b3 DBZ-3030 Do not execute initial statements on connector validation 2021-03-05 11:59:22 +01:00
Debezium Builder
b4c260da50 [maven-release-plugin] prepare for next development iteration 2021-02-23 14:04:01 -05:00
Debezium Builder
123a711a6d [maven-release-plugin] prepare release v1.5.0.Beta1 2021-02-23 14:04:01 -05:00
Fahim Farook
194258d87b
DBZ-3164 Log database connection validation failures consistently 2021-02-22 14:52:22 +01:00
Gunnar Morling
62ad3969c1 DBZ-3107 Typo fixes 2021-02-22 09:08:47 +01:00
Jiri Pechanec
a97b86db43 DBZ-3107 Workaround for DBZ-3158 2021-02-22 09:08:47 +01:00
Jiri Pechanec
1edad510ff DBZ-2614 Support for infinite timestamps 2021-02-19 10:44:06 +01:00
Jiri Pechanec
1f79c15a48 DBZ-2616 Fix negative years conversion 2021-02-19 10:44:06 +01:00
Jiri Pechanec
a17e7cee44 DBZ-2865 Regression in warning message cleanup 2021-02-19 10:40:15 +01:00
Yilong Chang
a0b3704832 DBZ-3056 Validate all fields in subclasses. 2021-02-16 12:34:06 +01:00
Yilong Chang
76c7e0d426 DBZ-3056 Extract basic valdiation in parent class. 2021-02-16 12:34:06 +01:00
ani-sha
5f8a22a5df DBZ-2858 Make field descriptions consistent for time values 2021-02-16 09:55:35 +01:00
Kevin Pullin
db5dd54c20 DBZ-3074 Split shouldReceiveChangesForInsertsIndependentOfReplicaIdentity 
Split shouldReceiveChangesForInsertsIndependentOfReplicaIdentity into two tests,
one that works with the PGOUTPUT decoder and one that works with the other decoders
(due to differences between schema update handling).
 2021-02-10 10:58:26 +01:00
Kevin Pullin
2426de849a DBZ-3074 PgOutputMessageDecoder - Fix unordered keys 
Fix issue where `PgOutputMessageDecoder` does not define primary key metadata in key-order.

`PgOutputMessageDecoder.getTablePrimaryKeyColumnNamesFromDatabase()` returns a `Set`, leading to unordered
primary keys in the metadata. Switch the logic to call `JdbcConnection.readPrimaryKeyNames()` which
_does_ correctly return an ordered primary key list.
 2021-02-10 10:58:26 +01:00
Debezium Builder
e08df0a177 [maven-release-plugin] prepare for next development iteration 2021-02-06 01:25:23 -05:00
Debezium Builder
da0411bf70 [maven-release-plugin] prepare release v1.5.0.Alpha1 2021-02-06 01:25:23 -05:00
Naveen Kumar KR
75882f35ac DBZ-2382 Support emitting TRUNCATE events in PG11+ PGOUTPUT plugin 
default - TruncateHandlingMode.SKIP
supported configs - TruncateHandlingMode.SKIP / TruncateHandlingMode.INCLUDE
 2021-02-03 13:53:01 +01:00
ani-sha
d4003c9172 DBZ-2420 Define database configs under RelationalDatabaseConnectorConfig 2021-02-03 08:05:42 +01:00
Debezium Builder
237b4ac0f1 [maven-release-plugin] prepare for next development iteration 2021-01-28 07:38:55 -05:00
Debezium Builder
06b1d0cbad [maven-release-plugin] prepare release v1.4.1.Final 2021-01-28 07:38:55 -05:00
rkerner
6aa13e756a DBZ-2952 Remove Postgres replication slot validation (maybe part 1 to fix user's issues) 2021-01-27 10:31:00 +01:00
Gunnar Morling
aa674842a3 DBZ-2923 Improving test stability 2021-01-14 17:02:04 +01:00
Gunnar Morling
b5b8ef358b DBZ-2764 Misc. clean-up 2021-01-08 13:19:57 +01:00
ani-sha
bd427786e9 DBZ-2764 Centralize Postgres image for Testcontainers 2021-01-08 13:19:57 +01:00
Debezium Builder
3010f99c32 [maven-release-plugin] prepare for next development iteration 2021-01-07 03:38:13 -05:00
Debezium Builder
54534538d9 [maven-release-plugin] prepare release v1.4.0.Final 2021-01-07 03:38:13 -05:00
rkerner
752454e83d DBZ-2800 fix AWS user roles validation 2021-01-07 07:22:37 +01:00
Debezium Builder
b1ab996c3e [maven-release-plugin] prepare for next development iteration 2020-12-16 14:11:31 -05:00
Debezium Builder
8fc864b278 [maven-release-plugin] prepare release v1.4.0.CR1 2020-12-16 14:11:31 -05:00
Debezium Builder
cf921f56b4 [maven-release-plugin] prepare for next development iteration 2020-12-10 00:27:40 -05:00
Debezium Builder
c49a377487 [maven-release-plugin] prepare release v1.4.0.Beta1 2020-12-10 00:27:39 -05:00
Jiri Pechanec
d1a7aae736 DBZ-2772 wal2json needs db write to complete catch-up phase 2020-11-30 10:06:43 +01:00
Debezium Builder
4535549dcd [maven-release-plugin] prepare for next development iteration 2020-11-16 15:18:43 -05:00
Debezium Builder
3c70c5f7d5 [maven-release-plugin] prepare release v1.4.0.Alpha2 2020-11-16 15:18:43 -05:00
Jiri Pechanec
af91d7d2bf DBZ-2662 Increase wait time for metrics 2020-11-16 09:44:52 +01:00
vanhoale
17a424d9ac DBZ-2662 Handle ChangeEventQueue by the size in bytes 2020-11-16 08:39:33 +01:00
rkerner
4faf13d0c9 DBZ-2637 apply feedback 2020-11-16 07:47:25 +01:00
rkerner
aff1df7573 DBZ-2637 fix unit test timeouts 2020-11-16 07:47:25 +01:00
rkerner
77eee964a1 DBZ-2637 unrelated cleanup 2020-11-16 07:47:25 +01:00
rkerner
c59e00c65a DBZ-2637 Enhance Postgres validation checks 
+ add wal_level config check
+ add LOGIN and REPLICATION roles check
+ add replication slot is already in use check
 2020-11-16 07:47:25 +01:00
rkerner
f291b07b6b DBZ-2724 cleanup code and apply PR feedback 2020-11-10 10:44:41 +01:00
rkerner
890bf217dc DBZ-2724 unrelated cleanup 2020-11-10 10:44:41 +01:00
rkerner
0015d628d5 DBZ-2724 Add integration tests for DBZ-2617 (fix Postgres was not able to shutdown with shutdown mode "fast") 
DBZ-2602 Migrate Testcontainers related changes on DebeziumContainer from UI PoC backend to Debezium main repo
 2020-11-10 10:44:41 +01:00
Chris Cranford
4e6b3b1268 DBZ-2704 Fix test failure - PostgresConnectorIT#shouldRegularlyFlushLsnWithTxMonitoring 2020-11-04 06:15:16 +01:00
rkerner
c274c2b39b DBZ-2617 refactor to proper HeartbeatErrorHandler injection 2020-11-03 07:23:02 +01:00
cooksey
c262238183 DBZ-2608 Add a snapshotComplete lifecycle hook to the Snapshotter 2020-10-30 13:34:38 -04:00
Debezium Builder
26883dad92 [maven-release-plugin] prepare for next development iteration 2020-10-22 03:45:59 -04:00
Debezium Builder
9adbc2d67e [maven-release-plugin] prepare release v1.4.0.Alpha1 2020-10-22 03:45:59 -04:00
Jiri Pechanec
ee5353ef6f DBZ-2653 Process commits even with tx monitoring 2020-10-20 10:42:10 +02:00
Kaushik Iyer
7d9e8fdc46 DBZ-2456 fixes after rebase 2020-10-16 10:13:48 +02:00
Kaushik Iyer
e2e6dc0932 DBZ-2456 selective snapshot implementation 2020-10-16 10:13:48 +02:00
Chris Cranford
17039df7ce DBZ-2344 Fix test failure - RecordsStreamProducerIT - shouldReceiveHeartbeatAlsoWhenChangingNonWhitelistedTable 2020-10-06 09:07:42 +02:00
Debezium Builder
95b8ed3877 [maven-release-plugin] prepare for next development iteration 2020-10-01 09:08:51 -04:00
Debezium Builder
19ad73beb2 [maven-release-plugin] prepare release v1.3.0.Final 2020-10-01 09:08:50 -04:00
Jiri Pechanec
12be0d8428 DBZ-2565 Disable test failing for snapshots 2020-10-01 14:27:13 +02:00
Jiri Pechanec
45d683724f DBZ-2565 Fail on unsupported infinite timestamps 2020-10-01 14:27:13 +02:00
Chris Cranford
4fe6813179 DBZ-2435 - Fix test failure - ReplicationConnectionIT#shouldResumeFromLastReceivedLSN 2020-09-30 20:19:50 +02:00
Chris Cranford
155dd44dab DBZ-2544 Fix formatting 2020-09-30 19:42:10 +02:00
Chris Cranford
1fd6fbb1fa DBZ-2544 Fix test failure - PostgresConnectorIT#customSnapshotterSkipsTablesOnRestart 2020-09-30 19:42:10 +02:00
Chris Cranford
bb0b3fd158 DBZ-2468 Fix test failure - PostgresConnectorIT#shouldExecuteOnConnectStatements 2020-09-24 23:06:55 +02:00
Debezium Builder
b146b69154 [maven-release-plugin] prepare for next development iteration 2020-09-24 04:56:42 -04:00
Debezium Builder
c51cfe8eb9 [maven-release-plugin] prepare release v1.3.0.CR1 2020-09-24 04:56:41 -04:00
cooksey
ba074abe4c DBZ-2550 Prevent duplicate events using catch up streaming 
Normally when a connector gracefully shuts down, the connect framework attempts to commit offsets so the latest committed state gets acked on the replication stream.  While the connector is running, the framework periodically commits offsets. Debezium does not manage triggering an offset commit.  When the catch up streaming phase ends, there may be uncommitted state and the connector is unable to determine when the next commit will occur because the commit timing is externally managed. If a commit is not triggered between the end of the catch up streaming phase and the normal streaming phase after the snapshot, the connector may produce some duplicated messages.

Although the replication stream may be out of date, the internal OffsetContext is aware of the latest committed offset. When the snapshot phase recreates a new offset after catch up streaming, the previous offset has access to the latest state. Use the previous offset to forward known state to the new offset.
 2020-09-21 09:28:15 +02:00
Debezium Builder
837669890a [maven-release-plugin] prepare for next development iteration 2020-09-16 12:18:08 -04:00
Debezium Builder
086dbd960b [maven-release-plugin] prepare release v1.3.0.Beta2 2020-09-16 12:18:08 -04:00
Jiri Pechanec
eed321d485
DBZ-2461 LSN must not be flushed after connection close 2020-09-16 11:40:52 +02:00
Chris Cranford
e5c01c6b09 DBZ-137 Fix compilation problems 2020-09-14 12:33:43 +02:00
AndreyIg
77662a0dde DBZ-137 contribution to the core, Oracle related 2020-09-14 12:33:43 +02:00
cooksey
8bcbdb639d DBZ-2094 Allow Postgres snapshotter to set streaming start on resume 
When a connector resumes after previously streaming and takes a
snapshot, through a new method on the snapshotter interface,
shouldStreamEventsStartingFromSnapshot, can choose whether
to resume streaming from the last known streaming position or the
beginning of the snapshot. This is helpful for snapshotters that
may not want to resnapshot every table in the whitelist/blacklist
but not miss event on the tables that are skipped.
 2020-09-09 10:54:11 +02:00
Debezium Builder
9979db648f [maven-release-plugin] prepare for next development iteration 2020-08-28 05:04:26 -04:00
Debezium Builder
5c58b23756 [maven-release-plugin] prepare release v1.3.0.Beta1 2020-08-28 05:04:26 -04:00
Jiri Pechanec
03a11efd8e DBZ-2338 Atomic is not necessary 2020-08-24 21:16:37 +02:00
Jiri Pechanec
cf0db85436 DBZ-2338 Flush LSNs on tx boundaries 2020-08-24 21:16:37 +02:00
Jiri Pechanec
483a483569 DBZ-2338 Log exception; restart keepalive 2020-08-24 21:16:37 +02:00
Jiri Pechanec
f01ab234a4 DBZ-2338 Skip initial statements on reconnect 2020-08-24 21:16:37 +02:00
Jiri Pechanec
4d0ec1f848 DBZ-2397 Test for restart in tx 2020-08-24 21:16:37 +02:00
Jiri Pechanec
aead9a6612 DBZ-2338 Minor corrections 
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
 2020-08-24 21:16:37 +02:00
Jiri Pechanec
b5c2ed17ed DBZ-2338 Support for pgoutput 2020-08-24 21:16:37 +02:00
Jiri Pechanec
39623aa799 DBZ-2338 Base WAL position detection on commit LSNs 2020-08-24 21:16:37 +02:00
Jiri Pechanec
31721c247b DBZ-2200 Introduce LLSN domain type 2020-08-24 21:16:37 +02:00
rkerner
2c5c093587 DBZ-2171 avoid divisive language 2020-08-21 09:29:04 +02:00
Gunnar Morling
e89b65238f DBZ-2423 Increasing visibility of some methods for UI PoC usage 2020-08-12 13:50:42 +02:00
Gunnar Morling
44efd1bc6b DBZ-2423 Propagating port default values 2020-08-12 13:50:42 +02:00
Gunnar Morling
26b51e02cb DBZ-2418 Misc. clean-up 2020-08-12 09:28:35 +02:00
Gunnar Morling
ebe5169a66 DBZ-2418 Delaying retrieval of DB charset; 
This is called only once anyways, so there's no need for storing the info
in a field in PostgresConnection. This allows PostgresConnector#validate()
bring up a more meaningful error message in case of incorrect credentials
and other incorrect connection configuration.
 2020-08-12 09:28:35 +02:00
Debezium Builder
c4c50f5fc6 [maven-release-plugin] prepare for next development iteration 2020-08-06 00:46:18 -04:00
Debezium Builder
94cbeb95fa [maven-release-plugin] prepare release v1.3.0.Alpha1 2020-08-06 00:46:17 -04:00
rkerner
cad6cf62d3 DBZ-2315 fix BIT VARYING handling in Postgres connector 
* removed little endian padding for BIT types in JdbcValueConverters (only used for Postgres yet)
* removed legacy format handling fot BIT related types in JdbcValueConverters#convertBits which were leftovers from PR #1408
* removed unnecessary zero-ing of a newly created byte array to improve performance for huge byte arrays
* updated Postgres connector docs
 2020-08-03 14:58:31 +02:00
Björn Häuser
04254e3fe7 DBZ-2310: Return true when messages are skipped 
See https://issues.redhat.com/browse/DBZ-2310 for more explanation.

Short summary: Whenever LSNs are skipped return back that messages have
been processed, which will cause two things:

* The replication can be advanced properly
* The connector does not fallback to the poll timeout waiting interval

I also think its more correct to say, "yes I have read messages, but
still skipped all of them".
 2020-07-21 15:53:06 +02:00
Debezium Builder
e1ab38de84 [maven-release-plugin] prepare for next development iteration 2020-07-16 04:40:04 -04:00
Debezium Builder
69cf375faa [maven-release-plugin] prepare release v1.2.1.Final 2020-07-16 04:40:04 -04:00
Jiri Pechanec
f06a14f1da DBZ-2288 Rethrow exception in test 2020-07-16 09:51:00 +02:00
Jiri Pechanec
55ba8b8b89 DBZ-2288 Fix copy/paste errors 2020-07-16 09:51:00 +02:00
Jiri Pechanec
ea942a4268 DBZ-2288 Test for pgoutput 2020-07-16 09:51:00 +02:00
Jiri Pechanec
8b0b252eb9 DBZ-2288 Events in exported snapshot no longer filtered by LSN 2020-07-16 09:51:00 +02:00
Jos Huiting
da97be7842 DBZ-2307 - Improve logging when primary keys are missing 2020-07-15 11:18:48 -04:00
Chris Cranford
94f2932e95 DBZ-2271 Fix test failure - RecordsStreamProducerIT#testEmptyChangesProducesHeartbeat 
This fixes a race condition with heartbeat emitted messages with empty DDLs
 2020-07-14 17:25:44 +02:00