From 2c2d11ea04b509adbbdc0c25443fe1832b9d7417 Mon Sep 17 00:00:00 2001 From: Alexander Iskuskov Date: Wed, 8 Apr 2020 03:09:28 +0300 Subject: [PATCH] DBZ-1954 Add docs for mask column and truncate column features --- documentation/modules/ROOT/pages/connectors/db2.adoc | 8 ++++++++ documentation/modules/ROOT/pages/connectors/oracle.adoc | 8 ++++++++ .../modules/ROOT/pages/connectors/postgresql.adoc | 4 ++++ .../modules/ROOT/pages/connectors/sqlserver.adoc | 8 ++++++++ 4 files changed, 28 insertions(+) diff --git a/documentation/modules/ROOT/pages/connectors/db2.adoc b/documentation/modules/ROOT/pages/connectors/db2.adoc index 2dd8786ae..f1c0353c1 100644 --- a/documentation/modules/ROOT/pages/connectors/db2.adoc +++ b/documentation/modules/ROOT/pages/connectors/db2.adoc @@ -1267,6 +1267,14 @@ Note that primary key columns are always included in the event's key, also if bl When `true` the delete operations are represented by a delete event and a subsequent tombstone event. When `false` only a delete event is sent. + Emitting the tombstone event (the default behavior) allows Kafka to completely delete all events pertaining to the given key once the source record got deleted. +|`column.truncate.to._length_.chars` +|_n/a_ +|An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be truncated in the change event message values if the field values are longer than the specified number of characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. + +|`column.mask.with._length_.chars` +|_n/a_ +|An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be replaced in the change event message values with a field value consisting of the specified number of asterisk (`*`) characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer or zero. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. + |`column.propagate.source.type` |_n/a_ |An optional comma-separated list of regular expressions that match the fully-qualified names of columns whose original type and length should be added as a parameter to the corresponding field schemas in the emitted change messages. diff --git a/documentation/modules/ROOT/pages/connectors/oracle.adoc b/documentation/modules/ROOT/pages/connectors/oracle.adoc index 5f1988d57..40a73cdd1 100644 --- a/documentation/modules/ROOT/pages/connectors/oracle.adoc +++ b/documentation/modules/ROOT/pages/connectors/oracle.adoc @@ -1052,6 +1052,14 @@ Emitting the tombstone event (the default behavior) allows Kafka to completely d Each item (regular expression) must match the `:` representing the custom key. + Fully-qualified tables could be defined as `DB_NAME.TABLE_NAME` or `SCHEMA_NAME.TABLE_NAME`, depending on the specific connector. +|`column.truncate.to._length_.chars` +|_n/a_ +|An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be truncated in the change event message values if the field values are longer than the specified number of characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. + +|`column.mask.with._length_.chars` +|_n/a_ +|An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be replaced in the change event message values with a field value consisting of the specified number of asterisk (`*`) characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer or zero. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. + |`column.propagate.source.type` |_n/a_ |An optional comma-separated list of regular expressions that match the fully-qualified names of columns whose original type and length should be added as a parameter to the corresponding field schemas in the emitted change messages. diff --git a/documentation/modules/ROOT/pages/connectors/postgresql.adoc b/documentation/modules/ROOT/pages/connectors/postgresql.adoc index b11c1750a..6905456e9 100644 --- a/documentation/modules/ROOT/pages/connectors/postgresql.adoc +++ b/documentation/modules/ROOT/pages/connectors/postgresql.adoc @@ -1805,6 +1805,10 @@ Only alphanumeric characters and underscores should be used. When `true` the delete operations are represented by a delete event and a subsequent tombstone event. When `false` only a delete event is sent. + Emitting the tombstone event (the default behavior) allows Kafka to completely delete all events pertaining to the given key once the source record got deleted. +|`column.truncate.to._length_.chars` +|_n/a_ +|An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be truncated in the change event message values if the field values are longer than the specified number of characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. + |`column.mask.with._length_.chars` |_n/a_ |An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be replaced in the change event message values with a field value consisting of the specified number of asterisk (`*`) characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer or zero. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. diff --git a/documentation/modules/ROOT/pages/connectors/sqlserver.adoc b/documentation/modules/ROOT/pages/connectors/sqlserver.adoc index 70c9974f4..c0d3cbda1 100644 --- a/documentation/modules/ROOT/pages/connectors/sqlserver.adoc +++ b/documentation/modules/ROOT/pages/connectors/sqlserver.adoc @@ -1247,6 +1247,14 @@ Note that primary key columns are always included in the event's key, also if bl When `true` the delete operations are represented by a delete event and a subsequent tombstone event. When `false` only a delete event is sent. + Emitting the tombstone event (the default behavior) allows Kafka to completely delete all events pertaining to the given key once the source record got deleted. +|`column.truncate.to._length_.chars` +|_n/a_ +|An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be truncated in the change event message values if the field values are longer than the specified number of characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. + +|`column.mask.with._length_.chars` +|_n/a_ +|An optional comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be replaced in the change event message values with a field value consisting of the specified number of asterisk (`*`) characters. Multiple properties with different lengths can be used in a single configuration, although in each the length must be a positive integer or zero. Fully-qualified names for columns are of the form _databaseName_._tableName_._columnName_, or _databaseName_._schemaName_._tableName_._columnName_. + |`column.propagate.source.type` |_n/a_ |An optional comma-separated list of regular expressions that match the fully-qualified names of columns whose original type and length should be added as a parameter to the corresponding field schemas in the emitted change messages.