DBZ-8103 Add documentation
This commit is contained in:
parent
4d27337ab8
commit
b203a71d6b
@ -39,6 +39,7 @@
|
|||||||
** xref:transformations/header-to-value.adoc[HeaderToValue]
|
** xref:transformations/header-to-value.adoc[HeaderToValue]
|
||||||
** xref:transformations/timezone-converter.adoc[Timezone Converter]
|
** xref:transformations/timezone-converter.adoc[Timezone Converter]
|
||||||
** xref:transformations/timescaledb.adoc[TimescaleDB Integration]
|
** xref:transformations/timescaledb.adoc[TimescaleDB Integration]
|
||||||
|
** xref:transformations/decode-logical-decoding-message-content.adoc[Decode Logical Decoding Message Content]
|
||||||
** xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
|
** xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
|
||||||
* Post Processors
|
* Post Processors
|
||||||
** xref:post-processors/index.adoc[Overview]
|
** xref:post-processors/index.adoc[Overview]
|
||||||
|
@ -55,7 +55,7 @@ The following example shows the value of a record before and after the transform
|
|||||||
====
|
====
|
||||||
Value before the SMT processes the record::
|
Value before the SMT processes the record::
|
||||||
+
|
+
|
||||||
[source]
|
[source,json]
|
||||||
----
|
----
|
||||||
{
|
{
|
||||||
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3",
|
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3",
|
||||||
@ -74,7 +74,7 @@ Value before the SMT processes the record::
|
|||||||
|
|
||||||
Value after the SMT processes the record::
|
Value after the SMT processes the record::
|
||||||
+
|
+
|
||||||
[source, json]
|
[source,json]
|
||||||
----
|
----
|
||||||
{
|
{
|
||||||
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3",
|
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3",
|
||||||
|
@ -0,0 +1,121 @@
|
|||||||
|
:page-aliases: configuration/decode-logical-decoding-message-content.adoc
|
||||||
|
[id="decode-logical-decoding-message-content"]
|
||||||
|
= Decode Logical Decoding Message Content
|
||||||
|
|
||||||
|
:toc:
|
||||||
|
:toc-placement: macro
|
||||||
|
:linkattrs:
|
||||||
|
:icons: font
|
||||||
|
:source-highlighter: highlight.js
|
||||||
|
|
||||||
|
toc::[]
|
||||||
|
|
||||||
|
The `DecodeLogicalDecodingMessageContent` SMT converts binary content of a PostgreSQL logical decoding message to a structured form. It can be
|
||||||
|
used standalone or used in conjunction with another SMT, such as link:/documentation/reference/transformations/outbox-event-router[Outbox Event Router].
|
||||||
|
|
||||||
|
[[example-decode-logical-decoding-message-content]]
|
||||||
|
== Example
|
||||||
|
|
||||||
|
To convert binary content of a logical decoding message, configure the `DecodeLogicalDecodingMessageContent` SMT in the Kafka Connect configuration
|
||||||
|
for a connector:
|
||||||
|
|
||||||
|
[source]
|
||||||
|
----
|
||||||
|
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
|
||||||
|
...
|
||||||
|
"transforms": "decodeLogicalDecodingMessageContent",
|
||||||
|
"transforms.decodeLogicalDecodingMessageContent.type": "io.debezium.connector.postgresql.transforms.DecodeLogicalDecodingMessageContent",
|
||||||
|
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
|
||||||
|
"key.converter.schemas.enable": false,
|
||||||
|
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
|
||||||
|
"value.converter.schemas.enable": "false",
|
||||||
|
...
|
||||||
|
----
|
||||||
|
|
||||||
|
The following example shows key and value of a record before and after the transformation is applied.
|
||||||
|
|
||||||
|
.Effect of applying the `DecodeLogicalDecodingMessageContent` SMT
|
||||||
|
====
|
||||||
|
Key before the SMT processes the record::
|
||||||
|
+
|
||||||
|
[source,json]
|
||||||
|
----
|
||||||
|
{
|
||||||
|
"prefix": "test-prefix"
|
||||||
|
}
|
||||||
|
----
|
||||||
|
|
||||||
|
Value before the SMT processes the record::
|
||||||
|
+
|
||||||
|
[source,json]
|
||||||
|
----
|
||||||
|
{
|
||||||
|
"op": "m",
|
||||||
|
"ts_ms": 1723115240065,
|
||||||
|
"source": {
|
||||||
|
"version": "3.0.0-SNAPSHOT",
|
||||||
|
"connector": "postgresql",
|
||||||
|
"name": "connector-name",
|
||||||
|
"ts_ms": 1723115239782,
|
||||||
|
"snapshot": "false",
|
||||||
|
"db": "source-db",
|
||||||
|
"sequence": "[\"26997744\",\"26997904\"]",
|
||||||
|
"ts_us": 1723115239782690,
|
||||||
|
"ts_ns": 1723115239782690000,
|
||||||
|
"schema": "",
|
||||||
|
"table": "",
|
||||||
|
"txId": 756,
|
||||||
|
"lsn": 26997904,
|
||||||
|
"xmin": null
|
||||||
|
},
|
||||||
|
"message": {
|
||||||
|
"prefix": "test-prefix",
|
||||||
|
"content": "eyJpZCI6IDEsICJpdGVtIjogIkRlYmV6aXVtIGluIEFjdGlvbiIsICJzdGF0dXMiOiAiRU5URVJFRCIsICJxdWFudGl0eSI6IDIsICJ0b3RhbFByaWNlIjogMzkuOTh9"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
----
|
||||||
|
|
||||||
|
Key after the SMT processes the record:: null
|
||||||
|
|
||||||
|
Value after the SMT processes the record::
|
||||||
|
+
|
||||||
|
[source,json]
|
||||||
|
----
|
||||||
|
{
|
||||||
|
"op": "c",
|
||||||
|
"ts_ms": 1723115415729,
|
||||||
|
"source": {
|
||||||
|
"version": "3.0.0-SNAPSHOT",
|
||||||
|
"connector": "postgresql",
|
||||||
|
"name": "connector-name",
|
||||||
|
"ts_ms": 1723115415640,
|
||||||
|
"snapshot": "false",
|
||||||
|
"db": "source-db",
|
||||||
|
"sequence": "[\"26717416\",\"26717576\"]",
|
||||||
|
"ts_us": 1723115415640161,
|
||||||
|
"ts_ns": 1723115415640161000,
|
||||||
|
"schema": "",
|
||||||
|
"table": "",
|
||||||
|
"txId": 745,
|
||||||
|
"lsn": 26717576,
|
||||||
|
"xmin": null
|
||||||
|
},
|
||||||
|
"after": {
|
||||||
|
"id": 1,
|
||||||
|
"item": "Debezium in Action",
|
||||||
|
"status": "ENTERED",
|
||||||
|
"quantity": 2,
|
||||||
|
"totalPrice": 39.98
|
||||||
|
}
|
||||||
|
}
|
||||||
|
----
|
||||||
|
|
||||||
|
That is, the SMT does the following:
|
||||||
|
|
||||||
|
. replaces `op` field value with `c` by which a record will look as an `INSERT` event
|
||||||
|
. replaces `message` field with `after` field that contains the decoded content of a logical decoding message
|
||||||
|
. drops a record's key that initially contains prefix of a logical decoding message
|
||||||
|
|
||||||
|
Thus, the SMT also makes a record suitable for the further processing by Outbox Event Router.
|
||||||
|
|
||||||
|
====
|
@ -42,6 +42,9 @@ The following SMTs are provided by {prodname}:
|
|||||||
|xref:transformations/timescaledb.adoc[TimescaleDB Integration]
|
|xref:transformations/timescaledb.adoc[TimescaleDB Integration]
|
||||||
|Routes and enriches messages that the {prodname} PostgreSQL connector captures from a TimescaleDB.
|
|Routes and enriches messages that the {prodname} PostgreSQL connector captures from a TimescaleDB.
|
||||||
|
|
||||||
|
|xref:transformations/decode-logical-decoding-message-content.adoc[Decode Logical Decoding Message Content]
|
||||||
|
|Converts binary content of a logical decoding message captured by the {prodname} PostgreSQL connector to a structured form.
|
||||||
|
|
||||||
|xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
|
|xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
|
||||||
|Converts values of records deserialized by {prodname} CloudEvents converter to a structure suitable for {prodname} JDBC sink connector.
|
|Converts values of records deserialized by {prodname} CloudEvents converter to a structure suitable for {prodname} JDBC sink connector.
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user