DBZ-8103 Add documentation

This commit is contained in:
Roman Kudryashov 2024-08-08 12:15:25 +03:00 committed by Jiri Pechanec
parent 4d27337ab8
commit b203a71d6b
4 changed files with 127 additions and 2 deletions

View File

@ -39,6 +39,7 @@
** xref:transformations/header-to-value.adoc[HeaderToValue] ** xref:transformations/header-to-value.adoc[HeaderToValue]
** xref:transformations/timezone-converter.adoc[Timezone Converter] ** xref:transformations/timezone-converter.adoc[Timezone Converter]
** xref:transformations/timescaledb.adoc[TimescaleDB Integration] ** xref:transformations/timescaledb.adoc[TimescaleDB Integration]
** xref:transformations/decode-logical-decoding-message-content.adoc[Decode Logical Decoding Message Content]
** xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form] ** xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
* Post Processors * Post Processors
** xref:post-processors/index.adoc[Overview] ** xref:post-processors/index.adoc[Overview]

View File

@ -55,7 +55,7 @@ The following example shows the value of a record before and after the transform
==== ====
Value before the SMT processes the record:: Value before the SMT processes the record::
+ +
[source] [source,json]
---- ----
{ {
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3", "id": "624e6565-99ee-4fdb-9228-653138c3a7b3",
@ -74,7 +74,7 @@ Value before the SMT processes the record::
Value after the SMT processes the record:: Value after the SMT processes the record::
+ +
[source, json] [source,json]
---- ----
{ {
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3", "id": "624e6565-99ee-4fdb-9228-653138c3a7b3",

View File

@ -0,0 +1,121 @@
:page-aliases: configuration/decode-logical-decoding-message-content.adoc
[id="decode-logical-decoding-message-content"]
= Decode Logical Decoding Message Content
:toc:
:toc-placement: macro
:linkattrs:
:icons: font
:source-highlighter: highlight.js
toc::[]
The `DecodeLogicalDecodingMessageContent` SMT converts binary content of a PostgreSQL logical decoding message to a structured form. It can be
used standalone or used in conjunction with another SMT, such as link:/documentation/reference/transformations/outbox-event-router[Outbox Event Router].
[[example-decode-logical-decoding-message-content]]
== Example
To convert binary content of a logical decoding message, configure the `DecodeLogicalDecodingMessageContent` SMT in the Kafka Connect configuration
for a connector:
[source]
----
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
...
"transforms": "decodeLogicalDecodingMessageContent",
"transforms.decodeLogicalDecodingMessageContent.type": "io.debezium.connector.postgresql.transforms.DecodeLogicalDecodingMessageContent",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter.schemas.enable": false,
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
...
----
The following example shows key and value of a record before and after the transformation is applied.
.Effect of applying the `DecodeLogicalDecodingMessageContent` SMT
====
Key before the SMT processes the record::
+
[source,json]
----
{
"prefix": "test-prefix"
}
----
Value before the SMT processes the record::
+
[source,json]
----
{
"op": "m",
"ts_ms": 1723115240065,
"source": {
"version": "3.0.0-SNAPSHOT",
"connector": "postgresql",
"name": "connector-name",
"ts_ms": 1723115239782,
"snapshot": "false",
"db": "source-db",
"sequence": "[\"26997744\",\"26997904\"]",
"ts_us": 1723115239782690,
"ts_ns": 1723115239782690000,
"schema": "",
"table": "",
"txId": 756,
"lsn": 26997904,
"xmin": null
},
"message": {
"prefix": "test-prefix",
"content": "eyJpZCI6IDEsICJpdGVtIjogIkRlYmV6aXVtIGluIEFjdGlvbiIsICJzdGF0dXMiOiAiRU5URVJFRCIsICJxdWFudGl0eSI6IDIsICJ0b3RhbFByaWNlIjogMzkuOTh9"
}
}
----
Key after the SMT processes the record:: null
Value after the SMT processes the record::
+
[source,json]
----
{
"op": "c",
"ts_ms": 1723115415729,
"source": {
"version": "3.0.0-SNAPSHOT",
"connector": "postgresql",
"name": "connector-name",
"ts_ms": 1723115415640,
"snapshot": "false",
"db": "source-db",
"sequence": "[\"26717416\",\"26717576\"]",
"ts_us": 1723115415640161,
"ts_ns": 1723115415640161000,
"schema": "",
"table": "",
"txId": 745,
"lsn": 26717576,
"xmin": null
},
"after": {
"id": 1,
"item": "Debezium in Action",
"status": "ENTERED",
"quantity": 2,
"totalPrice": 39.98
}
}
----
That is, the SMT does the following:
. replaces `op` field value with `c` by which a record will look as an `INSERT` event
. replaces `message` field with `after` field that contains the decoded content of a logical decoding message
. drops a record's key that initially contains prefix of a logical decoding message
Thus, the SMT also makes a record suitable for the further processing by Outbox Event Router.
====

View File

@ -42,6 +42,9 @@ The following SMTs are provided by {prodname}:
|xref:transformations/timescaledb.adoc[TimescaleDB Integration] |xref:transformations/timescaledb.adoc[TimescaleDB Integration]
|Routes and enriches messages that the {prodname} PostgreSQL connector captures from a TimescaleDB. |Routes and enriches messages that the {prodname} PostgreSQL connector captures from a TimescaleDB.
|xref:transformations/decode-logical-decoding-message-content.adoc[Decode Logical Decoding Message Content]
|Converts binary content of a logical decoding message captured by the {prodname} PostgreSQL connector to a structured form.
|xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form] |xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
|Converts values of records deserialized by {prodname} CloudEvents converter to a structure suitable for {prodname} JDBC sink connector. |Converts values of records deserialized by {prodname} CloudEvents converter to a structure suitable for {prodname} JDBC sink connector.