alexey.rassokhin/tet123
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

DBZ-2501 PostgreSQL event value callouts fixed to render correctly downstream

Browse Source
This commit is contained in:
Tova Cohen 2020-09-09 13:30:22 -04:00 committed by Chris Cranford
parent fabfd22f99
commit 41aa096f35
1 changed files with 47 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

90
documentation/modules/ROOT/pages/connectors/postgresql.adoc
View File

@ -654,7 +654,7 @@ The following example shows the value portion of a change event that the connect
[source,json,options="nowrap",indent=0,subs="+attributes"]
----
{
"schema": { <1>
"schema": { // <1>
"type": "struct",
"fields": [
{
@ -682,7 +682,7 @@ The following example shows the value portion of a change event that the connect
}
],
"optional": true,
"name": "PostgreSQL_server.inventory.customers.Value", <2>
"name": "PostgreSQL_server.inventory.customers.Value", // <2>
"field": "before"
},
{
@ -710,7 +710,7 @@ The following example shows the value portion of a change event that the connect
}
],
"optional": true,
"name": "PostgreSQL_server.inventory.customers.Value", <2>
"name": "PostgreSQL_server.inventory.customers.Value",
"field": "after"
},
{
@ -774,7 +774,7 @@ The following example shows the value portion of a change event that the connect
}
],
"optional": false,
"name": "io.debezium.connector.postgresql.Source", <2>
"name": "io.debezium.connector.postgresql.Source", // <3>
"field": "source"
},
{
@ -789,17 +789,17 @@ The following example shows the value portion of a change event that the connect
}
],
"optional": false,
"name": "PostgreSQL_server.inventory.customers.Envelope" <2>
"name": "PostgreSQL_server.inventory.customers.Envelope" // <4>
},
"payload": { <3>
"before": null, <4>
"after": { <5>
"payload": { // <5>
"before": null, // <6>
"after": { // <7>
"id": 1,
"first_name": "Anne",
"last_name": "Kretchmar",
"email": "annek@noanswer.org"
},
"source": { <6>
"source": { // <8>
"version": "{debezium-version}",
"connector": "postgresql",
"name": "PostgreSQL_server",
@ -812,8 +812,8 @@ The following example shows the value portion of a change event that the connect
"lsn": 24023128,
"xmin": null
},
"op": "c", <7>
"ts_ms": 1559033904863 <8>
"op": "c", // <9>
"ts_ms": 1559033904863 // <10>
}
}
----
@ -830,40 +830,44 @@ The following example shows the value portion of a change event that the connect
|2
|`name`
a|In the `schema` section, each `name` field specifies the schema for a field in the value's payload. In this example:
* `PostgreSQL_server.inventory.customers.Value` is the schema for the payload's `before` and `after` fields. This schema is specific to the `customers` table.
* `io.debezium.connector.postgresql.Source` is the schema for the payload's `source` field. This schema is specific to the PostgreSQL connector. The connector uses it for all events that it generates.
* `PostgreSQL_server.inventory.customers.Envelope` is the schema for the overall structure of the payload, where `PostgreSQL_server` is the connector name, `inventory` is the database, and `customers` is the table.
a|In the `schema` section, each `name` field specifies the schema for a field in the value's payload. +
+
`PostgreSQL_server.inventory.customers.Value` is the schema for the payload's `before` and `after` fields. This schema is specific to the `customers` table. +
+
Names of schemas for `before` and `after` fields are of the form `_logicalName_._tableName_.Value`, which ensures that the schema name is unique in the database. This means that when using the {link-prefix}:{link-avro-serialization}[Avro converter], the resulting Avro schema for each table in each logical source has its own evolution and history.
|3
|`payload`
|The value's actual data. This is the information that the change event is providing.
|`name`
a|`io.debezium.connector.postgresql.Source` is the schema for the payload's `source` field. This schema is specific to the PostgreSQL connector. The connector uses it for all events that it generates.
|4
|`name`
a|`PostgreSQL_server.inventory.customers.Envelope` is the schema for the overall structure of the payload, where `PostgreSQL_server` is the connector name, `inventory` is the database, and `customers` is the table.
|5
|`payload`
|The value's actual data. This is the information that the change event is providing. +
+
It may appear that the JSON representations of the events are much larger than the rows they describe. This is because the JSON representation must include the schema and the payload portions of the message.
However, by using the {link-prefix}:{link-avro-serialization}[Avro converter], you can significantly decrease the size of the messages that the connector streams to Kafka topics.
|4
|6
|`before`
a|An optional field that specifies the state of the row before the event occurred. When the `op` field is `c` for create, as it is in this example, the `before` field is `null` since this change event is for new content. +
+
[NOTE]
====
Whether or not this field is available is dependent on the {link-prefix}:{link-postgresql-connector}#postgresql-replica-identity[`REPLICA IDENTITY`] setting for each table.
====
[NOTE]
====
Whether or not this field is available is dependent on the {link-prefix}:{link-postgresql-connector}#postgresql-replica-identity[`REPLICA IDENTITY`] setting for each table.
====
|5
|7
|`after`
|An optional field that specifies the state of the row after the event occurred. In this example, the `after` field contains the values of the new row's `id`, `first_name`, `last_name`, and `email` columns.
|6
|8
|`source`
a|Mandatory field that describes the source metadata for the event. This field contains information that you can use to compare this event with other events, with regard to the origin of the events, the order in which the events occurred, and whether events were part of the same transaction. The source metadata includes: +
+
a|Mandatory field that describes the source metadata for the event. This field contains information that you can use to compare this event with other events, with regard to the origin of the events, the order in which the events occurred, and whether events were part of the same transaction. The source metadata includes:
* {prodname} version
* Connector type and name
* Database and table that contains the new row
@ -873,7 +877,7 @@ a|Mandatory field that describes the source metadata for the event. This field c
* Offset of the operation in the database log
* Timestamp
|7
|9
|`op`
a|Mandatory string that describes the type of operation that caused the connector to generate the event. In this example, `c` indicates that the operation created a row. Valid values are:
@ -882,7 +886,7 @@ a|Mandatory string that describes the type of operation that caused the connecto
* `d` = delete
* `r` = read (applies to only snapshots)
|8
|10
|`ts_ms`
a|Optional field that displays the time at which the connector processed the event. The time is based on the system clock in the JVM running the Kafka Connect task.
@ -898,16 +902,16 @@ The value of a change event for an update in the sample `customers` table has th
{
"schema": { ... },
"payload": {
"before": { <1>
"before": { // <1>
"id": 1
},
"after": { <2>
"after": { // <2>
"id": 1,
"first_name": "Anne Marie",
"last_name": "Kretchmar",
"email": "annek@noanswer.org"
},
"source": { <3>
"source": { // <3>
"version": "{debezium-version}",
"connector": "postgresql",
"name": "PostgreSQL_server",
@ -920,7 +924,7 @@ The value of a change event for an update in the sample `customers` table has th
"lsn": 24023128,
"xmin": null
},
"op": "u", <4>
"op": "u", // <4>
"ts_ms": 1465584025523
}
}
@ -939,7 +943,7 @@ For an _update_ event to contain the previous values of all columns in the row,
|2
|`after`
| An optional field that specifies the state of the row after the event occurred. In this example, the `first_name` value is now `Anne Marie`.
|An optional field that specifies the state of the row after the event occurred. In this example, the `first_name` value is now `Anne Marie`.
|3
|`source`
@ -985,11 +989,11 @@ The value in a _delete_ change event has the same `schema` portion as _create_ a
{
"schema": { ... },
"payload": {
"before": { <1>
"before": { // <1>
"id": 1
},
"after": null, <2>
"source": { <3>
"after": null, // <2>
"source": { // <3>
"version": "{debezium-version}",
"connector": "postgresql",
"name": "PostgreSQL_server",
@ -1002,8 +1006,8 @@ The value in a _delete_ change event has the same `schema` portion as _create_ a
"lsn": 46523128,
"xmin": null
},
"op": "d", <4>
"ts_ms": 1465581902461 <5>
"op": "d", // <4>
"ts_ms": 1465581902461 // <5>
}
}
----
@ -1021,7 +1025,7 @@ In this example, the `before` field contains only the primary key column because
|2
|`after`
| Optional field that specifies the state of the row after the event occurred. In a _delete_ event value, the `after` field is `null`, signifying that the row no longer exists.
|Optional field that specifies the state of the row after the event occurred. In a _delete_ event value, the `after` field is `null`, signifying that the row no longer exists.
|3
|`source`