kanha gupta
1b17372d2d
DBZ-4902 Replace long durations with Duration
2022-06-27 14:29:51 +02:00
Vojtech Juranek
e15f148deb
DBZ-5070 Don't reset lsn when lastCommitLsn is null
...
Currently we update first `lsn` and then update `lastCommitLsn` in
`PostgresOffsetContext` constructor. However, when `lastCommitLsn`
is for whatever reason `null`, previously updated `lsn` is reset to
`null` as `updateLastCommit()` updates also `lsn`. This can have
unwanted consequences like streaming again records which were already
streamed. To prevent this, update `lsn` in `updateLastCommit()` only
when `lastCommitLsn` is not `null`.
2022-06-22 13:55:17 +02:00
Jiri Pechanec
4fc4c8c374
DBZ-5031 Stop connector on failure to find resume LSN
2022-06-20 12:00:05 +02:00
Oskar Polak
3e288b8385
DBZ-5031 Additional logging when offset flushing is not allowed
2022-06-20 12:00:05 +02:00
Jiri Pechanec
38cd6907ae
DBZ-3942 Load correct last commit LSN from offsets
2022-06-17 14:49:44 +02:00
Jiri Pechanec
04c474bac7
DBZ-5031 Enable LSN flushing only after processed message
2022-06-17 12:39:56 +02:00
Jiri Pechanec
5721dd7ace
DBZ-5031 Emit warning if not exact LSN is found
2022-06-17 12:39:56 +02:00
Jun Zhao
29fbdc03bf
DBZ-5031 Stop skipping message if last received lsn is higher or equal to the startStreamingLsn in WalPositionLocation
2022-06-17 12:39:56 +02:00
Chris Cranford
3fb5bf1339
DBZ-4451 Introduce SchemaChangeEvent factory methods
2022-06-16 09:19:41 +02:00
Chris Cranford
863abcd081
DBZ-4451 Store previous table id in alter schema change events
...
When a table is renamed with the ALTER TABLE statement, the schema
history record not only will reference the current table's unique id
in the "id" metadata, but will also refer to the old table name in
the "previousId" metadata field.
2022-06-16 09:19:41 +02:00
Chris Cranford
20bb2adea5
DBZ-4834 Fix incremental snapshot of table added to include list
2022-06-16 08:39:16 +02:00
Debezium Builder
6fff3a2876
[maven-release-plugin] prepare for next development iteration
2022-06-09 11:49:26 +00:00
Debezium Builder
b2e2e351df
[maven-release-plugin] prepare release v2.0.0.Alpha2
2022-06-09 11:49:26 +00:00
Chris Cranford
660e391851
DBZ-5204 Do not cache connection details in TypeRegistry
.
2022-06-08 09:21:04 +02:00
Vojtech Juranek
4a48eb33df
DBZ-5045 Remove legacy V1 source info
2022-06-08 09:11:00 +02:00
Vojtech Juranek
7b8cb693ed
DBZ-5045 Remove deprecated Postgres TOASTED_VALUE_PLACEHOLDER
2022-06-08 09:11:00 +02:00
Vojtech Juranek
1eacbdc590
DBZ-5045 Remove deprecated Postgres EXPORTED snapshot option
2022-06-08 09:11:00 +02:00
Vojtech Juranek
83dfd9f638
DBZ-5045 Remove deprecated Postgres TRUNCATE_HANDLING_MODE
2022-06-08 09:11:00 +02:00
Vojtech Juranek
b0b3066ed5
DBZ-5045 Remove deprecated COLUMN_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
5031594b75
DBZ-5045 Remove deprecated COLUMN_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
f84893a8b4
DBZ-5045 Remove deprecated SCHEMA_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
261188482f
DBZ-5045 Remove deprecated SCHEMA_WHITELIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
6abbcee6a3
DBZ-5045 Remove deprecated TABLE_BLACKLIST
2022-06-08 09:11:00 +02:00
Vojtech Juranek
db08e894fc
DBZ-5045 Remove deprecated TABLE_WHITELIST
2022-06-08 09:11:00 +02:00
harveyyue
acb743bd34
DBZ-5206 Remove the deprecated monitored tables metrics
2022-06-07 16:13:29 -04:00
Chris Cranford
9d71296b5c
DBZ-4251 Support stopping incremental snapshots
2022-06-02 07:30:07 +02:00
Jiri Pechanec
88b7ae2050
DBZ-5136 Move sourceInfo to common class
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
c64b8af31a
DBZ-5136 reintroduce the CommonOffsetContext class and refactor common functionality into it
2022-06-01 16:56:29 +02:00
Mark Bereznitsky
2d72dae649
DBZ-5136 further refining of the algorithm for determining position in snapshot and IT test fixes
2022-06-01 16:56:29 +02:00
Jiri Pechanec
08e8c3d795
DBZ-5136 Move common class to common package
2022-06-01 16:56:29 +02:00
Andrew Walker
d0759461a4
DBZ-5136: Convert RelationalOffsetContext to CommonOffsetContext
2022-06-01 16:56:29 +02:00
Andrew Walker
b6222d42bf
DBZ-5136: Add first flag and refactor OffsetContext
2022-06-01 16:56:29 +02:00
Andrew Walker
593c2769e6
DBZ-5136: Add last in table flag
2022-06-01 16:56:29 +02:00
Vojtech Juranek
0757c966d0
DBZ-5042 Add DB name logging support into Postgres connector
2022-05-27 14:20:30 -04:00
Chris Cranford
7d8c9486d6
DBZ-5119 Move Heartbeat creation to ConnectorConfig & descendants
2022-05-25 09:59:59 +02:00
Chris Cranford
7ff7826743
DBZ-5119 Pull heartbeat.action.query
support into debezium-core
2022-05-24 12:54:40 +02:00
Chris Cranford
7eee39800f
DBZ-5038 Fix test failures
2022-05-23 11:46:15 +02:00
Chris Cranford
2a4c76cabe
DBZ-5038 Support default values for PostgreSQL enum types
2022-05-23 11:46:15 +02:00
Jiri Pechanec
a11767d05b
DBZ-3921 Remove incorrect comment
2022-05-17 14:56:05 +02:00
troeselereos
836632e5dd
DBZ-3921 fixed logging
...
DBZ-3921: update message
Co-authored-by: Chris Cranford <ccranfor@redhat.com>
2022-05-17 14:56:05 +02:00
roeselert
6b72d22be8
DBZ-3921: fix postgres existing publication is not updated with the new table
2022-05-17 14:56:05 +02:00
Jiri Pechanec
cc90b1cfeb
DBZ-5014 Fix delete tests
2022-05-09 08:53:59 +02:00
Cesar Martinez
61a77c2467
DBZ-5014 Remove unused import
2022-05-09 08:53:59 +02:00
Cesar Martinez
c0ac30593c
DBZ-5014 Uncomment validations
2022-05-09 08:53:59 +02:00
Cesar Martinez
72e51e02c7
DBZ-5014 Remove invalid line
2022-05-09 08:53:59 +02:00
Cesar Martinez
6994c1db56
DBZ-5014 Test case for delete register with infinity date
2022-05-09 08:53:59 +02:00
Cesar Martinez
840979a454
DBZ-5014 Add AbstractRecordsProducerTest.tableIdFromDeleteStmt(String)
2022-05-09 08:53:59 +02:00
Cesar Martinez
46f6db2d50
DBZ-5014 Add infinity date constants (Date & LocalDate
2022-05-09 08:53:59 +02:00
Cesar Martinez
24173cc548
DBZ-5014 Parse infinity date on delete
2022-05-09 08:53:59 +02:00
Cesar Martinez
b1f7fa1a47
DBZ-5014 Add test case for (+/-)infinity date
2022-05-09 08:53:59 +02:00
Jiri Pechanec
6e092d39f2
DBZ-5074 No separate connection for pgoutput decoder
2022-05-06 16:36:28 +02:00
Jiri Pechanec
88dab4d8d3
DBZ-5074 Track connections by name; close heartbeat connection by default
2022-05-06 16:36:28 +02:00
Mark Allanson
0edb86d861
DBZ-5097: Add tests to cover PostgresErrorHandler behaviour
...
The error handler had no unit tests at all, so I thought it might be a good idea
to add some tests to verify the behaviour given there's a bunch of different
ways to determine if something is retryable.
2022-05-06 09:55:06 -04:00
Mark Allanson
dd815f5226
DBZ-5097: Add two new retryable postgresql error messages
...
These two error messages can occur when running inside docker and the
postgres container is either killed (sigtermed) or brought down
using `docker stop`.
In both cases these are retryable as the postgres server can be
restarted shortly thereafter.
2022-05-06 09:55:06 -04:00
Debezium Builder
901825810a
[maven-release-plugin] prepare for next development iteration
2022-04-28 13:51:13 +00:00
Debezium Builder
05dcb45647
[maven-release-plugin] prepare release v2.0.0.Alpha1
2022-04-28 13:51:12 +00:00
Jiri Pechanec
d186ec800f
DBZ-703 Remove support for messages without metadata
2022-04-22 09:57:15 +02:00
Jiri Pechanec
d299c9e707
DBZ-4156 Remove wal2json support
2022-04-22 09:57:15 +02:00
Mohammad Yousuf Minhaj Zia
6f706fc087
DBZ-4948 Added connection has been closed PSQLException as retriable
2022-04-07 12:35:27 +02:00
Debezium Builder
4a6de64c25
[maven-release-plugin] prepare for next development iteration
2022-04-05 14:49:52 +00:00
Debezium Builder
683851fd90
[maven-release-plugin] prepare release v1.9.0.Final
2022-04-05 14:49:51 +00:00
Jiri Pechanec
1355067ef0
DBZ-4814 Use schema not catalog in table id
2022-03-31 15:26:12 +02:00
zalmane
1f64e8f8da
DBZ-4814 exclude spatial_ref_sys
2022-03-30 20:08:38 +02:00
Debezium Builder
ab7fe3b96a
[maven-release-plugin] prepare for next development iteration
2022-03-25 16:45:06 +00:00
Debezium Builder
d19fc234ee
[maven-release-plugin] prepare release v1.9.0.CR1
2022-03-25 16:45:06 +00:00
Sergei Morozov
b258674782
DBZ-3535: Introduce schema.name.adjustment.mode
2022-03-24 09:37:05 +01:00
Sergei Morozov
3a03ff0186
DBZ-3535: Remove unused EventDispatcher constructor parameter
2022-03-24 09:37:05 +01:00
Clément Loiselet
ba33e483ae
DBZ-4861: change log for schema snapshot postgres
...
Update debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/PostgresSnapshotChangeEventSource.java
2022-03-16 09:51:15 +01:00
Gunnar Morling
d20a6f00e7
DBZ-4801 Using JdbcConfiguration in JdbcConnection
2022-03-07 17:53:45 +01:00
Gunnar Morling
c1c322ecf0
DBZ-4801 Simplifying JdbcConnection constructors
2022-03-07 17:53:45 +01:00
Debezium Builder
fece16039c
[maven-release-plugin] prepare for next development iteration
2022-03-03 17:00:08 +00:00
Debezium Builder
e0d8a0d809
[maven-release-plugin] prepare release v1.9.0.Beta1
2022-03-03 17:00:08 +00:00
Sergei Morozov
6d0aa9408c
DBZ-4478: Add Partition parameter to io.debezium.pipeline.source.spi.*Listener interfaces
2022-02-16 11:31:38 +01:00
Jiri Pechanec
54ba2cd598
DBZ-4736 Handle string NULL default values correctly
2022-02-11 12:06:55 +01:00
Jiri Pechanec
b8bafdb4c6
DBZ-2306 Avoid long running TX for publication query
2022-02-10 15:50:37 +01:00
Debezium Builder
7a47f478a4
[maven-release-plugin] prepare for next development iteration
2022-02-09 07:23:10 +00:00
Debezium Builder
b1dd35025e
[maven-release-plugin] prepare release v1.9.0.Alpha2
2022-02-09 07:23:10 +00:00
Andrei Isac
4dee34922e
DBZ-4710: fix replication role check on RDS Postgres
...
According to Debezium documentation[1], on AWS RDS, the `rds_replication` role
is enough to read the wal_log. However, on the connection validation step,
this case is omitted and the connector cannot start. The issue is pressent
starting at least v1.4.x.Final.
[1] https://debezium.io/documentation/reference/1.8/connectors/postgresql.html#postgresql-on-amazon-rds
2022-02-08 08:42:32 +01:00
Chris Cranford
06d682707f
DBZ-4701 Fix PostgreSQL tests DDL execution
2022-02-07 06:21:11 +01:00
Debezium Builder
33e2f5cd0f
[maven-release-plugin] prepare for next development iteration
2022-01-26 19:00:20 +00:00
Debezium Builder
a77b6f8b8e
[maven-release-plugin] prepare release v1.9.0.Alpha1
2022-01-26 19:00:20 +00:00
Chris Cranford
35ef67852c
DBZ-4562 Reduce log verbosity
2022-01-25 18:41:06 +01:00
Jiri Pechanec
58469cd381
DBZ-4592 Make OID not found exception retriable
2022-01-24 14:55:15 +01:00
Chris Cranford
62e2fd48c4
DBZ-4419 Fix test failure
2022-01-22 00:33:21 +01:00
Chris Cranford
383f619ce1
DBZ-4419 Map truncate handling mode internally to skipped ops
2022-01-22 00:33:21 +01:00
Chris Cranford
9b6ade3165
DBZ-4419 Fix field validation test error
2022-01-22 00:33:21 +01:00
Chris Cranford
27fe106912
DBZ-4419 Default skipped.operations
to "t" for PostgreSQL, add tests
2022-01-22 00:33:21 +01:00
Chris Cranford
773debc340
DBZ-4419 Deprecate TruncateHandlingMode, favor skipped.operations
2022-01-22 00:33:21 +01:00
Gunnar Morling
0c62da2aa5
DBZ-4520 Making static collections immutable
2022-01-21 18:03:57 +01:00
Gunnar Morling
9b35294fe8
DBZ-4520 Making collection immutable
2022-01-21 13:39:09 +01:00
Jiri Pechanec
6a89f1ee6a
DBZ-4520 Add backend errors among retriable
2022-01-21 13:26:48 +01:00
Jiri Pechanec
f1b03c176e
DBZ-3953 Mark wal2json plugin as deprecated
2022-01-21 11:21:17 +01:00
Jiri Pechanec
664d989e42
DBZ-4577 Provide custom error message handling
2022-01-21 10:24:19 +01:00
Jiri Pechanec
632a61ee98
DBZ-4517 Enable correct config opts for retrying
2022-01-19 16:17:10 +01:00
Jiri Pechanec
9509e5976a
DBZ-4566 Used topic names for Debezium Engine converters
2022-01-19 11:32:45 +01:00
Jiri Pechanec
8fd3243a86
DBZ-2224 Test logging based on logback
2022-01-18 11:24:21 +01:00
Jiri Pechanec
3f957aa02e
DBZ-4131 Test with PostgreSQL 10 by default
2022-01-14 15:48:00 +01:00
Anisha Mohanty
d6c5ad2e7c
DBZ-4460 Renaming all openapi occurrences to schema
2022-01-07 15:05:34 +01:00
Anisha Mohanty
3d1988cf6f
DBZ-4460 Set default values for format & output directory
2022-01-07 15:05:34 +01:00
Gunnar Morling
a4497cae96
DBZ-3657 Enabling export of domain type array columns via include.unknown.datatypes
2022-01-06 20:23:08 +01:00
sarumont
edbee2e8f6
DBZ-3657: add null check when building array converters
2022-01-06 20:23:08 +01:00
harveyyue
291e8ac221
DBZ-4351 Add create/update/delete event seen metrics for monitor upstream dml operation
2022-01-06 19:03:46 +01:00
harveyyue
792187f669
DBZ-4472 Make connector task partition readability for log
2021-12-20 15:58:14 +01:00
Jiri Pechanec
967432ad08
DBZ-4469 Test should run with any Postgres version
2021-12-17 13:57:53 +01:00
Debezium Builder
67c9663c98
[maven-release-plugin] prepare for next development iteration
2021-12-16 10:11:29 +00:00
Debezium Builder
8d532d895f
[maven-release-plugin] prepare release v1.8.0.Final
2021-12-16 10:11:28 +00:00
Chris Cranford
63731a2a09
DBZ-3940 Unify EventRouter tests for connectors
2021-12-13 13:52:31 +01:00
Jiri Pechanec
740b6b21f6
DBZ-4298 Support non-JDBC logical values in keys for incremental snapshots
...
Incremental snapshot source gets the PK values after they are processed
by value converter. This is usally correct solution. The values are
passed to chunk queries. There are few datatypes like unconstrained
NUMERIC in PostgreSQL that must be converted to a Debezium specific
class to keep and enrich the value. Such class is not known to JDBC
driver. This commit intorduce a marker interface that enables the code
to identify the value not recognized by JDBC and provides a method to
obtain the original value.
2021-12-13 12:29:21 +01:00
Gunnar Morling
0023cb10a5
DBZ-4393 Create a Debezium Schema Generator for Debezium connectors
...
* added an API generator for Debezium connectors and static API definitions for connectors in a separate module
* added Maven plug-in
* added GH workflow for debezium-schema-generator
Co-authored-by: rkerner <rkerner.mobil@gmail.com>
Co-authored-by: Anisha Mohanty <anishamohanty23@gmail.com>
2021-12-10 12:35:09 +01:00
Debezium Builder
f53e6c00df
[maven-release-plugin] prepare for next development iteration
2021-12-09 20:16:04 +00:00
Debezium Builder
826837701a
[maven-release-plugin] prepare release v1.8.0.CR1
2021-12-09 20:16:04 +00:00
Chris Cranford
c095cb7090
DBZ-2792 Remove test
2021-12-07 14:57:23 +01:00
Debezium Builder
b5439b54bf
[maven-release-plugin] prepare for next development iteration
2021-11-30 09:38:47 +00:00
Debezium Builder
125f6e5aa2
[maven-release-plugin] prepare release v1.8.0.Beta1
2021-11-30 09:38:47 +00:00
Chris Cranford
2347b58250
DBZ-4272 Added javadoc
2021-11-30 09:28:44 +01:00
Chris Cranford
0a10d5b484
DBZ-4272 Trigger schema refresh before snapshot; only for PostgreSQL
2021-11-30 09:28:44 +01:00
Chris Cranford
652f41f7fb
DBZ-4272 Add drop column schema change to test
2021-11-30 09:28:44 +01:00
Gunnar Morling
74bb80aae7
DBZ-2363 Removing unused methods
2021-11-29 14:55:49 +01:00
Gunnar Morling
a8cda23ef2
DBZ-4370 Only setting "messages" option on PG 14+;
...
Also avoiding setting stream options repeatedly.
2021-11-29 14:55:49 +01:00
Gunnar Morling
2157256398
DBZ-4364 Awaiting table metadata to become queryable
2021-11-29 09:23:18 +01:00
Gunnar Morling
a041a6bd5c
DBZ-4364 Removing unused methods
2021-11-29 09:23:18 +01:00
Gunnar Morling
5866056a41
DBZ-2363 Using prefix as message key
2021-11-26 18:15:25 +01:00
Gunnar Morling
560ef2babd
DBZ-2363 Misc. adjustments;
...
* Adding Strings::isBlank()
* Simplifying test
* Using long in cases where we know it's present
* Removing some unused code
* Testing binary handling mode
* Avoiding TX id value overflow
* Assigning names to schemas to avoid issues as in DBZ-650
2021-11-26 14:29:15 +01:00
Gunnar Morling
b85a9d2f20
DBZ-2363 Extracting separate test class for logical decoding message tests
2021-11-26 14:29:15 +01:00
Lairen Hightower
450998b49e
DBZ-2363 Replace logicalDecodingMessageHandlingMode with a prefix filtering configuration
2021-11-26 14:29:15 +01:00
Gunnar Morling
fcfc9bc785
DBZ-2363 Misc. adjustments;
...
* Removing unused method
* Unifying update() handling
* Member ordering
* Avoiding NPE in error handler
2021-11-26 14:29:15 +01:00
lairen
a0898ed440
DBZ-2363 Adding support for pg_logical_emit messages in Postgres connector for the pgoutput plugin
2021-11-26 14:29:15 +01:00
Jiri Pechanec
dc3ee488f1
DBZ-4343 Use schema component for incremental snapshot TableId
2021-11-24 10:25:33 +01:00
Andrey Yegorov
b6d2c41fb0
DBZ-4309 Avoid calling 'LoggerFactory.getLogger(getClass())' for each new instance of the RelationalChangeRecordEmitter
2021-11-18 22:37:51 +01:00
Vojtech Juranek
c0a65743f8
DBZ-1042 Allow Postgres connector to use numeric slot names
...
To be able to use numeric slot names, the slot name has to be quoted.
This may be consider as s bug in Postgres driver as Debezium passes the
slot name as string. To workaround this issue, quote the slot name on
the Debezium side.
2021-11-18 10:27:48 +01:00
Vojtech Juranek
1ca917a428
DBZ-1042 Allow numeric slot names
2021-11-18 10:27:48 +01:00
Vojtech Juranek
50da5e31dc
DBZ-1931 Allow to specify number of fraction digits for the money type
2021-11-17 09:30:15 +01:00
Vojtech Juranek
6f286fe50c
DBZ-1931 Reflect decimal.handling.mode in money conversion
2021-11-17 09:30:15 +01:00
Chris Cranford
e3da449303
DBZ-4276 Expose unavailable value placeholder for PG and Oracle only
2021-11-16 20:29:59 +01:00
Chris Cranford
ca17352221
DBZ-4276 Unify unavailable value placeholder handling
...
Deprecate the `toasted.value.placeholder` option in PostgreSQL in favor
of the new `unavailable.value.placeholder` option that is shared by all
relational connectors.
2021-11-16 20:29:59 +01:00
Kate
44073cf7d8
DBZ-4196 Support schema changes during incremental snapshot
2021-11-16 14:56:06 +01:00
Debezium Builder
f93e901dce
[maven-release-plugin] prepare for next development iteration
2021-11-11 09:21:59 +00:00
Debezium Builder
895a16bf19
[maven-release-plugin] prepare release v1.8.0.Alpha2
2021-11-11 09:21:59 +00:00
Kate
992ae537c8
DBZ-4244 Quote signal table name in MySQL connector
2021-11-10 10:18:26 +01:00
Chris Cranford
5ec9381702
DBZ-3710 Unify DefaultValueMapper (Oracle/SQL Server/PostgreSQL)
2021-11-10 09:43:25 +01:00
harveyyue
aa3ae05e22
DBZ-4107 Incremental snapshot doesn't work without primary key
2021-11-05 13:31:26 +01:00
Gunnar Morling
1226c3b2ab
DBZ-4029 Misc. clean-up;
...
* Exposing heartbeatInterval() on CommonConnectorConfig
* Closing heartbeat connection
* Making test more resilient
* Correct member ordering and visibility
* Doc fixes
2021-11-05 13:27:09 +01:00
ahodavdekar
91b7ef2c36
DBZ-4029 Adding support for heartbeat action queries to MySQL connector
2021-11-05 13:27:09 +01:00
jiabao.sun
24ead16c24
DBZ-3966 Remove defaultValue and keep defaultValueExpression
2021-11-04 10:47:17 +01:00
jiabao.sun
77c4113697
DBZ-3966 JsonTableChangeSerializer support serialization for defaultValue and enumValues
2021-11-04 10:47:17 +01:00
Gunnar Morling
9a4d0ad363
DBZ-4137 Test clean-up
2021-11-02 16:50:26 +01:00
Andrey Yegorov
b174927a52
DBZ-4137 Postgres defaults handling misses decimal mode configuration and cannot handle numerics with default NULL values
2021-11-02 16:50:26 +01:00
ahodavdekar
1d81b6db19
DBZ-4077 Exposing the transaction metadata topic name as a config option
2021-11-02 06:07:05 +01:00
Anisha Mohanty
9f7ede0e06
DBZ-3626 Renaming master branches to main
2021-10-28 09:33:32 +02:00
Debezium Builder
2026a48302
[maven-release-plugin] prepare for next development iteration
2021-10-27 06:47:27 +00:00
Debezium Builder
97106c1fd9
[maven-release-plugin] prepare release v1.8.0.Alpha1
2021-10-27 06:47:26 +00:00
Gunnar Morling
33764b2adc
DBZ-4150 Using set instead of list
2021-10-19 12:32:16 +02:00
Hossein Torabi
6027d762a6
DBZ-4150 Trim numerical defaultValue before converting postgres-connector
2021-10-19 12:27:26 +02:00
Jiri Pechanec
0ccc83db21
DBZ-3860 Provide correct boundaries for multi PK tables
2021-10-14 17:26:50 +02:00
Anisha Mohanty
310a0b6ef3
DBZ-4145 Skip failing test for wal2json
2021-10-14 13:17:08 +02:00
Gunnar Morling
c3eed6051b
DBZ-3993 Skipping object size based test on Java 16+
2021-10-13 10:59:23 +02:00
Sergei Morozov
4a03c5df17
DBZ-2975: Include database name into topic and schema names
...
Co-authored-by: Mike Kamornikov <mikekamornikov@gmail.com>
2021-10-12 15:40:02 +02:00
Gunnar Morling
015b219c58
DBZ-4046 Some testing clean-up;
...
* Fixing mixed up order of expected and actual value
* Using more expressive var names
2021-10-11 06:04:42 +02:00
Judah Rand
5d23b30f67
DBZ-4046 Update isVariableScaleDecimal
...
This puts the method in line with
https://github.com/pgjdbc/pgjdbc/issues/2188
where expected value was changed from `131089` to `0`.
However, for numeric arrays the issue remains and so
`131089` remains the correct value for them.
2021-10-08 10:42:21 +02:00
Jiri Pechanec
d5eafd3ecf
DBZ-3919 Add logs to test
2021-10-06 09:36:21 +02:00
Jiri Pechanec
dae384baf4
DBZ-4081 Close leaking connections
2021-10-05 12:52:03 +02:00
Jiri Pechanec
f910fc930b
[release] Switch to 1.8 snapshot
2021-10-01 14:50:16 +02:00
Debezium Builder
12b0ae9046
[maven-release-plugin] prepare for next development iteration
2021-10-01 13:26:33 +02:00
Debezium Builder
652162fd85
[maven-release-plugin] prepare release v1.7.0.Final
2021-10-01 13:26:33 +02:00
Jiri Pechanec
6d0760df45
DBZ-4060 Remove race condition
2021-10-01 11:25:14 +02:00
Judah Rand
6ad791bedc
DBZ-4060 Add default value mapper for smallserial
2021-10-01 11:25:14 +02:00
Jiri Pechanec
a2c6058404
DBZ-4057 Initialize Postgres schema before incremental snapshot restart
2021-09-30 09:10:49 +02:00
Debezium Builder
bec0b6b67e
[maven-release-plugin] prepare for next development iteration
2021-09-23 06:31:25 +00:00
Debezium Builder
cf8da39725
[maven-release-plugin] prepare release v1.7.0.CR2
2021-09-23 06:31:25 +00:00
Naveen Kumar KR
d07643d60f
DBZ-3887 PostgresMetricsIT Test fix
...
* Remove redundant oneRecordInQueue test
* Rename test method from twoRecordsInQueue -> oneRecordInQueue to reflect the actual assertion done
2021-09-22 19:12:21 +02:00
Debezium Builder
c15e0b303b
[maven-release-plugin] prepare for next development iteration
2021-09-16 08:10:45 +00:00
Debezium Builder
23d9790121
[maven-release-plugin] prepare release v1.7.0.CR1
2021-09-16 08:10:44 +00:00
René Kerner
b92e5c5efe
DBZ-3943 Add ConfigDef unit tests to verify all Debezium Field instances have a proper documentation/description set;
...
Also adding missing descriptions.
2021-09-09 13:35:41 +02:00
Gunnar Morling
7c150de943
DBZ-2525 Don't qualify selected columns with table name;
...
It shouldn't be needed as we select from a single table, and it caused the columns to not be found for the Db2 connector.
2021-08-31 17:39:36 +05:30
Anisha Mohanty
153e0920e1
DBZ-2525 Generifying exclusion of columns from snapshotting in all relational connectors;
...
* Using RelationalDatabaseSchema in RelationalSnapshotChangeEventSource for consistency.
* Adding test for table/column names which are keywords
2021-08-30 12:20:56 +02:00
Gunnar Morling
11ed86a807
DBZ-3889 Reducing logging of serializers
2021-08-30 12:20:55 +02:00
Jiri Pechanec
82b151d0b9
DBZ-3935 Invalid validation of truncate handling mode
2021-08-30 09:47:55 +02:00
Debezium Builder
2ce6bcb2ab
[maven-release-plugin] prepare for next development iteration
2021-08-25 05:20:56 -04:00
Debezium Builder
d3a7efe423
[maven-release-plugin] prepare release v1.7.0.Beta1
2021-08-25 05:20:56 -04:00
rkerner
52333596de
DBZ-3364 add additional metadata to fields for API spec generator for the Debezium UI
...
part of https://issues.redhat.com/browse/DBZ-3364
DBZ-2653 User rollback from parent class
2021-08-23 17:57:38 +02:00
Gunnar Morling
36af61b789
DBZ-3889 Moving ContainerImageVersions to core for broader re-use and using it in two more places
2021-08-23 12:34:03 +02:00
Gunnar Morling
0fedadd77e
DBZ-3889 Avoiding some log noise
2021-08-23 12:34:03 +02:00
Naveen Kumar KR
9527843974
DBZ-3870 Performance Improvements in PostgresChangeRecordEmitter
...
1) Reuse parsed TableId to avoid expensive PostgresSchema.parse() call
2) Immediate conditional return in synchronizeTableSchema() call
2021-08-13 22:11:48 +02:00
Sergei Morozov
a40ad7132d
DBZ-2975: Remove partition from offset context
2021-08-11 07:34:37 +02:00
Sergei Morozov
77e2ac8a12
DBZ-2975: Introduce io.debezium.pipeline.spi.Offsets
2021-08-11 07:34:37 +02:00
Sergei Morozov
c7ac5fca52
DBZ-2975: Move Partition to io.debezium.pipeline.spi
2021-08-11 07:34:37 +02:00
Jiri Pechanec
b0e0052a9c
DBZ-3828 Update event LSN for commits
2021-08-10 20:41:37 +02:00
Gunnar Morling
85e6d262ea
DBZ-3816 Wording changes
2021-08-09 10:08:47 +02:00
Jiri Pechanec
305c15132f
DBZ-3816 Workaround for Kafka Connect bug
2021-08-09 10:08:47 +02:00
Gunnar Morling
bf9f7f5596
DBZ-3801 Allocating sequence list with correct size
2021-08-04 10:25:09 +02:00
Brennan Vincent
b03d9d1fa6
DBZ-3801: Address code review comments
2021-08-04 10:25:09 +02:00
Brennan Vincent
975f1690f2
DBZ-3801 Fix calculation of sequence numbers for Postgres
2021-08-04 10:25:09 +02:00
Debezium Builder
d3c4482d06
[maven-release-plugin] prepare for next development iteration
2021-07-30 02:39:08 -04:00
Debezium Builder
35dcfc5426
[maven-release-plugin] prepare release v1.7.0.Alpha1
2021-07-30 02:39:08 -04:00
Kate
9b66ab374a
DBZ-3577 Support read-only MySQL incremental snapshots
2021-07-29 07:10:46 +02:00
Jiri Pechanec
b387775448
DBZ-3655 Connection liveness check triggers replication earlier
2021-07-27 11:07:52 -04:00
Jiri Pechanec
f46e0ca9a6
DBZ-3655 Commit check connection query
2021-07-27 15:32:35 +02:00
Gunnar Morling
09ebc833f3
DBZ-3655 Gracefully handling DB shutdown
2021-07-27 10:21:54 +02:00
Chris Cranford
9f5c49246f
DBZ-3769 Remove TODO and change log message level
2021-07-27 09:31:51 +02:00
bep7520
9e9c2fc2ab
DBZ-3769 Throw sqlexception from postgres message decoder
2021-07-27 09:31:51 +02:00
Naveen Kumar KR
7844b59c2f
DBZ-3770 Optimize ObjectMapper initialization by replacing repeated initialization with one time initilization
...
- Performance Improvement of approx 5% - 15%
2021-07-21 14:44:09 -04:00
Chris Cranford
ed31ea9ffb
DBZ-3668 Adjust ServiceLoader for SPI change
2021-07-03 16:31:52 +07:00
Gunnar Morling
b84c3c870d
DBZ-3668 Extracting SPI package
2021-07-03 16:31:52 +07:00
Chris Cranford
7f7e23b110
DBZ-3668 Add ServiceLoader hook CloudEvents converter
2021-07-03 16:31:52 +07:00
Sergei Morozov
db105baeba
DBZ-2975: Add partition awareness to source task components
...
Co-authored-by: Mike Kamornikov <mikekamornikov@gmail.com>
2021-07-03 10:56:34 +02:00
Gunnar Morling
5b76bf1b71
DBZ-3632 Pulling up getJdbcConfig() to RelationalDatabaseConnectorConfig
2021-07-01 15:11:01 +02:00
Jiri Pechanec
ee0bd2f7da
DBZ-3685 Postgres keepalive thread uses correct name
2021-07-01 11:40:59 +02:00
Debezium Builder
9d3f6b2a86
[maven-release-plugin] prepare for next development iteration
2021-06-30 03:45:06 -04:00
Debezium Builder
c8c5fc4b56
[maven-release-plugin] prepare release v1.6.0.Final
2021-06-30 03:45:06 -04:00
Debezium Builder
ae053414ac
[maven-release-plugin] prepare for next development iteration
2021-06-24 01:50:07 -04:00
Debezium Builder
af3f942a74
[maven-release-plugin] prepare release v1.6.0.CR1
2021-06-24 01:50:07 -04:00
Debezium Builder
48ca58a609
[maven-release-plugin] prepare for next development iteration
2021-06-10 06:03:12 -04:00
Debezium Builder
0961517859
[maven-release-plugin] prepare release v1.6.0.Beta2
2021-06-10 06:03:12 -04:00
Gunnar Morling
a2f927c2db
DBZ-3608 Avoiding repeated closing actions
2021-06-10 07:39:49 +02:00
Gunnar Morling
205c63a56a
DBZ-3608 Fixing version handling;
...
- wrong minor version for Postgres
- minor/patch versions of -1 were not actually ignored in comparison
2021-06-10 07:39:49 +02:00
Gunnar Morling
dc865cd82d
DBZ-2975 Formatting
2021-06-09 16:02:33 +02:00
Gunnar Morling
1ce65fa628
DBZ-2975 Avoiding raw types in some places;
...
Also parameterizing offset loader contract, so to avoid casts later on.
2021-06-09 16:02:33 +02:00
Sergei Morozov
657a6d2542
DBZ-2975: Extract offset context from object states to method signatures
2021-06-09 16:02:33 +02:00
Gunnar Morling
2c5e10b848
DBZ-3596 Using single connection in PgOutputMessageDecoder
2021-06-09 13:41:43 +02:00
Gunnar Morling
4aef8669fb
DBZ-3596 Removing unused "exported" flag
2021-06-09 13:41:43 +02:00
Gunnar Morling
7e664b18e5
DBZ-3596 Removing unused field AbstractMessageDecoder#filterBasedOnLsn and code leading to it
2021-06-09 13:41:43 +02:00
Jiri Pechanec
b59f6daac9
DBZ-3550 Use selected test containers deps
2021-06-08 14:49:04 +02:00
Gunnar Morling
41e93e6ce1
DBZ-2790 Avoiding function usage
2021-06-08 12:10:26 +02:00
Eric Slep
8a659ff8ab
DBZ-2790 Docs and one final test edge case
2021-06-08 12:10:26 +02:00
Eric Slep
5dde3062b0
DBZ-2790 Handle pgoutput streaming refresh
2021-06-08 12:10:26 +02:00
Eric Slep
13ebeade71
DBZ-2790 Add default values to existing tests
2021-06-08 12:10:26 +02:00
Eric Slep
436b4697ba
DBZ-2790 Tests for refresh behaviour, ensure defaults are not cleared on in-mem refresh
2021-06-08 12:10:26 +02:00
Eric Slep
d1917eeff6
DBZ-2790 Handle default values for date & time types
2021-06-08 12:10:26 +02:00
Eric Slep
f88f135c43
DBZ-2790 Add simple placeholders to schema for generated defaults
2021-06-08 12:10:26 +02:00
Eric Slep
de5fe2a520
DBZ-2790 Move TypeRegistry construction back to connection
2021-06-08 12:10:26 +02:00
Kevin Pullin
0462804467
DBZ-2790 Update debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/connection/PostgresDefaultValueConverter.java
...
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
2021-06-08 12:10:26 +02:00
Kevin Pullin
4b62e59400
DBZ-2790 Update debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/connection/PostgresDefaultValueConverter.java
...
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
2021-06-08 12:10:26 +02:00
Kevin Pullin
d2ab8baddf
DBZ-2790 Implement default values for postgres
...
Using the `SqlServerDefaultValueConverter` class as a starting point,
this commit implements default value support for postgres.
2021-06-08 12:10:26 +02:00
gvaquez-ubi
03cb907d85
DBZ-3068 Allow several tries on createReplicationStream to get replication slot
2021-06-04 15:11:57 +02:00
Debezium Builder
ead4ee7226
[maven-release-plugin] prepare for next development iteration
2021-05-20 19:07:44 +02:00
Debezium Builder
9e16df7d4b
[maven-release-plugin] prepare release v1.6.0.Beta1
2021-05-20 19:07:44 +02:00
Jiri Pechanec
a879b7b3e1
DBZ-3500 Re-use ResultSet to Object code
2021-05-20 08:06:37 +02:00
Jiri Pechanec
a626dc149f
DBZ-3500 Provide common test for incremental snapshot testing
2021-05-20 08:06:37 +02:00
Gunnar Morling
a362760429
DBZ-2337 Removing superfluous log statement
2021-05-18 21:47:16 +02:00
Jiri Pechanec
5fbc181b5e
DBZ-2337 All snapshots using exported mode
2021-05-18 21:44:08 +02:00
Jiri Pechanec
2a716d1b86
DBZ-3519 Avoid NPE in autoboxing
2021-05-18 10:32:18 +02:00
Debezium Builder
4dd3a146ca
[maven-release-plugin] prepare for next development iteration
2021-05-06 02:40:15 -04:00
Debezium Builder
b3c5bb281c
[maven-release-plugin] prepare release v1.6.0.Alpha1
2021-05-06 02:40:15 -04:00
Jiri Pechanec
5c722341e2
DBZ-3473 Extract incremental snapshot contract
2021-05-06 05:37:36 +02:00
Jiri Pechanec
7daae11600
DBZ-3473 Postgres has its own schema instance
2021-05-06 05:37:36 +02:00
Jiri Pechanec
39d21806eb
DBZ-3473 Metrics enabled
2021-05-06 05:37:36 +02:00
Jiri Pechanec
c53785dde7
DBZ-3473 Incremental Snapshot support for MySQL
2021-05-06 05:37:36 +02:00
Jiri Pechanec
4e95056ddd
DBZ-3473 Test stabilization
2021-05-06 05:37:36 +02:00
Jiri Pechanec
49c5a00bbf
DBZ-3473 Share read JDBC value between regular and incremental snapshot
2021-05-06 05:37:36 +02:00
Jiri Pechanec
dd2bc6354b
DBZ-3473 Pass the main clock form connector
2021-05-06 05:37:36 +02:00
Jiri Pechanec
260c32cf0e
DBZ-3473 Resume parallel snapshot after connector restart
2021-05-06 05:37:36 +02:00
Jiri Pechanec
37626c0dfe
DBZ-3473 Extract incremental snapshot context
2021-05-06 05:37:36 +02:00
Jiri Pechanec
07371c085b
DBZ-3473 Watermark signals must be in separate tx
2021-05-06 05:37:36 +02:00
Jiri Pechanec
a52c536a68
DBZ-3473 First prototype of incremental snapshotting
2021-05-06 05:37:36 +02:00
Gunnar Morling
dc02b896cd
DBZ-3449 Moving QA releated plug-ins to separate profile, so to avoid loading them at all
2021-04-28 15:14:05 +02:00
Gunnar Morling
2088c035b1
DBZ-3449 Adding "quick" profile
2021-04-26 06:09:14 +02:00
Hossein Torabi
5d6705c3fe
DBZ-2699 implement skipped.operation in core
2021-04-08 11:59:08 +02:00
Debezium Builder
8a88ba9b45
[maven-release-plugin] prepare for next development iteration
2021-04-08 04:04:00 -04:00
Debezium Builder
74152afe74
[maven-release-plugin] prepare release v1.5.0.Final
2021-04-08 04:04:00 -04:00
Gunnar Morling
5425e6c5dc
DBZ-3343 Re-using single shared SqlTypeMapper instance
2021-04-06 14:59:55 +02:00
JLDLaughlin
06b0475f17
DBZ-2911 Add additional LSN information to all sources
...
Adds a new "sequence" field to all sources via the AbstractSourceInfo
class. Specifically, this field is required to deduplicate records
from PostgreSQL sources in O(1) time. The sequence field is a stringified
list of metadata. For PostgreSQL sources, this will include the last
committed LSN and the current LSN. For other sources, this will be NULL
until implemented. A new integration test was added for PostgreSQL.
2021-03-26 09:19:17 +01:00
Debezium Builder
6b4d88bb51
[maven-release-plugin] prepare for next development iteration
2021-03-24 06:35:02 -04:00
Debezium Builder
c578715233
[maven-release-plugin] prepare release v1.5.0.CR1
2021-03-24 06:35:02 -04:00
Jiri Pechanec
495be2659b
DBZ-2511 Dedicated logger for schema name adjuster
2021-03-18 14:42:34 +01:00
Jiri Pechanec
21f522217c
DBZ-3033 Map OID to long value
2021-03-17 18:11:52 +01:00
Gunnar Morling
911b5e9a3b
DBZ-3271 Unifying filter handling across relational connectors
2021-03-17 09:54:25 +01:00
Debezium Builder
f57057421b
[maven-release-plugin] prepare for next development iteration
2021-03-12 04:38:17 -05:00
Debezium Builder
b57293013a
[maven-release-plugin] prepare release v1.5.0.Beta2
2021-03-12 04:38:16 -05:00
Jiri Pechanec
123d9ef4eb
DBZ-3213 Refactor Maven structure to deploy support modules
2021-03-11 16:14:15 +01:00
Jiri Pechanec
f23421920b
DBZ-2919 Remove race condition
2021-03-10 19:12:34 +01:00
Jiri Pechanec
15c7b11a1f
DBZ-3141 Change schema signal tested for Postgres
2021-03-10 19:12:34 +01:00
Jiri Pechanec
ff7cb44801
DBZ-3141 Offsets are propagated to signal actions
2021-03-10 19:12:34 +01:00
Jiri Pechanec
7fdad2ed49
DBZ-3141 Support for signalling table
2021-03-10 19:12:34 +01:00
Jiri Pechanec
9f2cffa21e
DBZ-2885 Fix test
2021-03-09 17:32:01 +01:00
Anisha Mohanty
895d704d82
DBZ-2885 Handle misleading error message for filtered publication
2021-03-09 08:39:15 +01:00
Chris Cranford
04849734b3
DBZ-3030 Do not execute initial statements on connector validation
2021-03-05 11:59:22 +01:00
Debezium Builder
b4c260da50
[maven-release-plugin] prepare for next development iteration
2021-02-23 14:04:01 -05:00
Debezium Builder
123a711a6d
[maven-release-plugin] prepare release v1.5.0.Beta1
2021-02-23 14:04:01 -05:00
Fahim Farook
194258d87b
DBZ-3164 Log database connection validation failures consistently
2021-02-22 14:52:22 +01:00
Gunnar Morling
62ad3969c1
DBZ-3107 Typo fixes
2021-02-22 09:08:47 +01:00
Jiri Pechanec
a97b86db43
DBZ-3107 Workaround for DBZ-3158
2021-02-22 09:08:47 +01:00
Jiri Pechanec
1edad510ff
DBZ-2614 Support for infinite timestamps
2021-02-19 10:44:06 +01:00
Jiri Pechanec
1f79c15a48
DBZ-2616 Fix negative years conversion
2021-02-19 10:44:06 +01:00
Jiri Pechanec
a17e7cee44
DBZ-2865 Regression in warning message cleanup
2021-02-19 10:40:15 +01:00
Yilong Chang
a0b3704832
DBZ-3056 Validate all fields in subclasses.
2021-02-16 12:34:06 +01:00
Yilong Chang
76c7e0d426
DBZ-3056 Extract basic valdiation in parent class.
2021-02-16 12:34:06 +01:00
ani-sha
5f8a22a5df
DBZ-2858 Make field descriptions consistent for time values
2021-02-16 09:55:35 +01:00
Kevin Pullin
db5dd54c20
DBZ-3074 Split shouldReceiveChangesForInsertsIndependentOfReplicaIdentity
...
Split shouldReceiveChangesForInsertsIndependentOfReplicaIdentity into two tests,
one that works with the PGOUTPUT decoder and one that works with the other decoders
(due to differences between schema update handling).
2021-02-10 10:58:26 +01:00
Kevin Pullin
2426de849a
DBZ-3074 PgOutputMessageDecoder - Fix unordered keys
...
Fix issue where `PgOutputMessageDecoder` does not define primary key metadata in key-order.
`PgOutputMessageDecoder.getTablePrimaryKeyColumnNamesFromDatabase()` returns a `Set`, leading to unordered
primary keys in the metadata. Switch the logic to call `JdbcConnection.readPrimaryKeyNames()` which
_does_ correctly return an ordered primary key list.
2021-02-10 10:58:26 +01:00
Debezium Builder
e08df0a177
[maven-release-plugin] prepare for next development iteration
2021-02-06 01:25:23 -05:00
Debezium Builder
da0411bf70
[maven-release-plugin] prepare release v1.5.0.Alpha1
2021-02-06 01:25:23 -05:00
Naveen Kumar KR
75882f35ac
DBZ-2382 Support emitting TRUNCATE events in PG11+ PGOUTPUT plugin
...
default - TruncateHandlingMode.SKIP
supported configs - TruncateHandlingMode.SKIP / TruncateHandlingMode.INCLUDE
2021-02-03 13:53:01 +01:00
ani-sha
d4003c9172
DBZ-2420 Define database configs under RelationalDatabaseConnectorConfig
2021-02-03 08:05:42 +01:00
Debezium Builder
237b4ac0f1
[maven-release-plugin] prepare for next development iteration
2021-01-28 07:38:55 -05:00
Debezium Builder
06b1d0cbad
[maven-release-plugin] prepare release v1.4.1.Final
2021-01-28 07:38:55 -05:00
rkerner
6aa13e756a
DBZ-2952 Remove Postgres replication slot validation (maybe part 1 to fix user's issues)
2021-01-27 10:31:00 +01:00
Gunnar Morling
aa674842a3
DBZ-2923 Improving test stability
2021-01-14 17:02:04 +01:00
Gunnar Morling
b5b8ef358b
DBZ-2764 Misc. clean-up
2021-01-08 13:19:57 +01:00
ani-sha
bd427786e9
DBZ-2764 Centralize Postgres image for Testcontainers
2021-01-08 13:19:57 +01:00
Debezium Builder
3010f99c32
[maven-release-plugin] prepare for next development iteration
2021-01-07 03:38:13 -05:00