DBZ-7065 Docs for ConvertCloudEventToSaveableForm
This commit is contained in:
parent
d1c08f154d
commit
f429536a75
@ -38,6 +38,7 @@
|
||||
** xref:transformations/header-to-value.adoc[HeaderToValue]
|
||||
** xref:transformations/timezone-converter.adoc[Timezone Converter]
|
||||
** xref:transformations/timescaledb.adoc[TimescaleDB Integration]
|
||||
** xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
|
||||
* API and SPI
|
||||
** xref:development/engine.adoc[Debezium Engine]
|
||||
** xref:development/converters.adoc[Custom Converters]
|
||||
|
@ -0,0 +1,118 @@
|
||||
:page-aliases: configuration/convert-cloudevent-to-saveable-form.adoc
|
||||
// Category: debezium-using
|
||||
// Type: assembly
|
||||
// ModuleID: convert-cloudevent-to-saveable-form
|
||||
// Title: ConvertCloudEventToSaveableForm
|
||||
[id="convert-cloudevent-to-saveable-form"]
|
||||
= ConvertCloudEventToSaveableForm
|
||||
|
||||
:toc:
|
||||
:toc-placement: macro
|
||||
:linkattrs:
|
||||
:icons: font
|
||||
:source-highlighter: highlight.js
|
||||
|
||||
toc::[]
|
||||
|
||||
The `ConvertCloudEventToSaveableForm` SMT is used in conjunction with `CloudEventsConverter` and `JdbcSinkConnector` that is it makes a
|
||||
CloudEvent suitable for saving to a database.
|
||||
|
||||
It is done using the mapping between a CloudEvent and target database table columns; it specifies which CloudEvent field is
|
||||
associated with which column. A new structure and its schema created by the transform contain fields with the names of target table
|
||||
columns and values from an original CloudEvent. If a CloudEvent contains `data` field as a structure, it is flattened, that is,
|
||||
converted to JSON string.
|
||||
|
||||
To perform the conversion, SMT needs to know serializer type that is used to serialize and deserialize CloudEvents
|
||||
(JSON or Avro). `serializer.type` should have the same value as `value.converter.serializer.type`.
|
||||
|
||||
// Type: concept
|
||||
// Title: Example: Basic configuration of the {prodname} `ConvertCloudEventToSaveableForm` SMT
|
||||
// ModuleID: basic-configuration-of-the-debezium-convert-cloudevent-to-saveable-form-smt
|
||||
[[example-convert-cloudevent-to-saveable-form]]
|
||||
== Example
|
||||
|
||||
To convert a CloudEvent to a form suitable for `JdbcSinkConnector`, configure the `ConvertCloudEventToSaveableForm` SMT in the Kafka Connect configuration
|
||||
for a connector. For example, to convert CloudEvents that were serialized and deserialized with JSON provided that the target database has
|
||||
`id`, `source`, `type`, and `payload` columns, use the following configuration for the SMT:
|
||||
|
||||
[source]
|
||||
----
|
||||
"connector.class": "io.debezium.connector.jdbc.JdbcSinkConnector",
|
||||
...
|
||||
"transforms": "convertCloudEvent",
|
||||
"transforms.convertCloudEvent.type": "io.debezium.connector.jdbc.transforms.ConvertCloudEventToSaveableForm",
|
||||
"transforms.convertCloudEvent.fields.mapping": "id,source,type,data:payload",
|
||||
"transforms.convertCloudEvent.serializer.type": "json",
|
||||
"value.converter": "io.debezium.converters.CloudEventsConverter",
|
||||
"value.converter.serializer.type": "json",
|
||||
"value.converter.data.serializer.type": "json"
|
||||
...
|
||||
----
|
||||
|
||||
The following example shows the value of a record before and after the transformation is applied.
|
||||
|
||||
.Effect of applying the `ConvertCloudEventToSaveableForm` SMT
|
||||
====
|
||||
Value before the SMT processes the record::
|
||||
+
|
||||
[source]
|
||||
----
|
||||
{
|
||||
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3",
|
||||
"source": "/debezium/postgresql/book",
|
||||
"specversion": "1.0",
|
||||
"type": "BookCreated",
|
||||
"time": "2023-11-11T07:11:01.825Z",
|
||||
"datacontenttype": "application/json",
|
||||
"data": {
|
||||
"id": 4,
|
||||
"name": "1984",
|
||||
"publicationYear": 1949
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
Value after the SMT processes the record::
|
||||
+
|
||||
[source, json]
|
||||
----
|
||||
{
|
||||
"id": "624e6565-99ee-4fdb-9228-653138c3a7b3",
|
||||
"source": "/debezium/postgresql/book",
|
||||
"type": "BookCreated",
|
||||
"payload": "{"id": 4, "name": "1984", "publicationYear": 1949}"
|
||||
}
|
||||
----
|
||||
====
|
||||
|
||||
// Type: reference
|
||||
// ModuleID: options-for-configuring-the-cloudevent-to-saveable-form-transformation
|
||||
// Title: Options for configuring the `ConvertCloudEventToSaveableForm` transformation
|
||||
[[cloudevent-to-saveable-form-configuration-options]]
|
||||
== Configuration options
|
||||
|
||||
The following table lists the configuration options that you can use with the `ConvertCloudEventToSaveableForm` SMT.
|
||||
|
||||
.ConvertCloudEventToSaveableForm SMT configuration options
|
||||
[cols="14%a,40%a,10%a, 16%a, 16%a, 10%a"]
|
||||
|===
|
||||
|Property
|
||||
|Description
|
||||
|Type
|
||||
|Default
|
||||
|Valid Values
|
||||
|Importance
|
||||
|
||||
|[[cloudevent-to-saveable-form-fields-mapping]]<<cloudevent-to-saveable-form-fields-mapping, `fields.mapping`>>
|
||||
|A comma-separated list of pairs each of which contains the name of a CloudEvents field followed by the name of a target table column
|
||||
|list
|
||||
|No default value
|
||||
|non-empty list
|
||||
|high
|
||||
|[[cloudevent-to-saveable-form-serializer-type]]<<cloudevent-to-saveable-form-serializer-type, `serializer.type`>>
|
||||
|Serializer type that is used to serialize and deserialize CloudEvents. Should have the same value as `value.converter.serializer.type`.
|
||||
|string
|
||||
|No default value
|
||||
|`json` or `avro`
|
||||
|high
|
||||
|===
|
@ -42,6 +42,9 @@ The following SMTs are provided by {prodname}:
|
||||
|xref:transformations/timescaledb.adoc[TimescaleDB Integration]
|
||||
|Routes and enriches messages that the {prodname} PostgreSQL connector captures from a TimescaleDB.
|
||||
|
||||
|xref:transformations/convert-cloudevent-to-saveable-form.adoc[Convert CloudEvents to Saveable Form]
|
||||
|Converts values of records deserialized by {prodname} CloudEvents converter to a structure suitable for {prodname} JDBC sink connector.
|
||||
|
||||
|===
|
||||
|
||||
By means of xref:transformations/applying-transformations-selectively.adoc[SMT Predicates] you can apply any of the transformations selectively, so that it modifies only that subset of change event messages that share a common characteristic.
|
||||
|
Loading…
Reference in New Issue
Block a user