tet123/documentation/modules/ROOT/pages/features.adoc

50 lines
3.0 KiB
Plaintext
Raw Normal View History

// Category: debezium-using
// Type: concept
// ModuleID: description-of-debezium-features
// Title: Description of Debezium features
[id="debezium-features"]
= {prodname} Features
:linkattrs:
:icons: font
{prodname} is a set of source connectors for Apache Kafka Connect. Each connector
ingests changes from a different database by using that database's features for change data capture (CDC).
Unlike other approaches, such as polling or dual writes,
log-based CDC as implemented by {prodname}:
* Ensures that *all data changes are captured*.
* Produces change events with a *very low delay* while avoiding increased CPU usage required for frequent polling. For example, for MySQL or PostgreSQL, the delay is in the millisecond range.
* Requires *no changes to your data model*, such as a "Last Updated" column.
* Can *capture deletes*.
* Can *capture old record state and additional metadata* such as transaction ID and causing query, depending on the database's capabilities and configuration.
link:https://debezium.io/blog/2018/07/19/advantages-of-log-based-change-data-capture/[Five Advantages of Log-Based Change Data Capture] is a blog post that provides more details.
{prodname} connectors capture data changes with a range of related capabilities and options:
* *Snapshots:* optionally, an initial snapshot of a database's current state can be taken if a connector is started and not all logs still exist. Typically, this is the case when the database has been running for some time and has discarded trannsaction logs that are no longer needed for transaction recovery or replication. There are different modes for performing snapshots. See the documentation for the connector that you are using.
* *Filters:* you can configure the set of captured schemas, tables and columns with include/exclude list filters.
* *Masking:* the values from specific columns can be masked, for example, when they contain sensitive data.
* *Monitoring:* most connectors can be monitored by using JMX.
* Ready-to-use *message transformations* for:
** {link-prefix}:{link-topic-routing}[Message routing]
** {link-prefix}:{link-content-based-routing}[Content-based routing]
ifdef::community[]
** {link-prefix}:{link-event-flattening}[Extraction of new record state for relational connectors] and {link-prefix}:{link-mongodb-event-flattening}[for the MongoDB connector]
endif::community[]
ifdef::product[]
** {link-prefix}:{link-event-flattening}[Extraction of new record state for relational connectors]
endif::product[]
** {link-prefix}:{link-filtering}[Filtering]
** {link-prefix}:{link-outbox-event-router}[Routing of events] from a transactional outbox table
ifdef::community[]
See the {link-prefix}:{link-connectors}[connector documentation] for a list of all supported databases and detailed information about the features and configuration options of each connector.
endif::community[]
ifdef::product[]
The documentation for each connector provides details about the connectors features and configuration options.
endif::product[]