alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

DBZ-7917 Document PostgreSQL read-only incremental snapshot.

Browse Source
This commit is contained in:
mfvitale 2024-06-13 11:20:55 +02:00 committed by Jiri Pechanec
parent 06f319d221
commit c0021ca53d
2 changed files with 28 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
debezium-core/src/main/java/io/debezium/pipeline/EventDispatcher.java
View File

@ -426,6 +426,9 @@ public void dispatchHeartbeatEvent(P partition, OffsetContext offset) throws Int
this::enqueueHeartbeat);
}
// Use this method when you want to dispatch the heartbeat also to incremental snapshot.
// Currently, this is used by PostgreSQL for read-only incremental snapshot but doesn't suites well for
// MySQL since the dispatchHeartbeatEvent is called at every received message and not when there is no message from the DB log.
public void dispatchHeartbeatEventAlsoToIncrementalSnapshot(P partition, OffsetContext offset) throws InterruptedException {
heartbeat.heartbeat(
partition.getSourcePartition(),

25
documentation/modules/ROOT/pages/connectors/postgresql.adoc
View File

@ -259,6 +259,25 @@ include::{partialsdir}/modules/all-connectors/proc-stopping-an-incremental-snaps
include::{partialsdir}/modules/all-connectors/proc-stopping-an-incremental-snapshot-kafka.adoc[leveloffset=+1]
ifdef::community[]
[id="postgresql-read-only-incremental-snapshots"]
==== Read-only incremental snapshots
The PostgreSQL connector allows for running incremental snapshots with a read-only connection to the database.
To run an incremental snapshot with read-only access, the connector uses the current in-progress transaction as high and low watermarks.
The state of a chunk's window is updated by comparing the txId of write-ahead-log events or the heartbeats against low and high watermarks.
To switch to a read-only implementation, set the value of the xref:postgres-property-read-only[`read.only`] property to `true`.
.Prerequisites
* PostgreSQL greater than or equal to 13.
==== Ad hoc read-only incremental snapshots
When the PostgreSQL connection is read-only, you can use any of the {link-prefix}:{link-signalling}#sending-signals-to-a-debezium-connector[available signaling channels] without the requirement to use the `source` channel.
endif::community[]
ifdef::community[]
[[connector-custom-snapshot]]
=== Custom snapshotter SPI
@ -3646,6 +3665,12 @@ After the snapshot completes, this entry is removed.
No entry is created for the signal to close the snapshot window.
Set this option to prevent rapid growth of the signaling data collection.
ifdef::community[]
|[[postgres-property-read-only]]<<postgres-property-read-only, `+read.only+`>>
|`false`
|Switch to alternative incremental snapshot watermarks implementation to avoid writes to signal data collection
endif::community[]
|[[postgresql-property-xmin-fetch-interval-ms]]<<postgresql-property-xmin-fetch-interval-ms, `+xmin.fetch.interval.ms+`>>
|`0`
|How often, in milliseconds, the XMIN will be read from the replication slot.