8c10e0ae66
The extension's columns for the outbox table can now be fully customized by using build-time configuration properties to influence a column's database definition, the columns length/precision/scale/nullability as well as specifying a JPA AttributeConverter. While not all theses options apply to every column, such as the identifier, the goal is to allow flexibility.
349 lines
9.6 KiB
Plaintext
349 lines
9.6 KiB
Plaintext
= Outbox Quarkus Extension
|
|
include::../_attributes.adoc[]
|
|
:toc:
|
|
:toc-placement: macro
|
|
:linkattrs:
|
|
:icons: font
|
|
:source-highlighter: highlight.js
|
|
|
|
toc::[]
|
|
|
|
[NOTE]
|
|
====
|
|
This feature is currently in incubating state, i.e. exact semantics, configuration options etc. may change in future revisions, based on the feedback we receive. Please let us know if you encounter any problems while using this extension.
|
|
====
|
|
|
|
== Overview
|
|
|
|
This extension is inspired by the xref::configuration/outbox-event-router.adoc[Outbox Event Router] single message transformation (SMT).
|
|
As discussed in the blog post link:/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/[Reliable Microservices Data Exchange with the Outbox Pattern], microservices often need to exchange information with one another and an excellent way to deal with that is using the Outbox pattern combined with Debezium's Outbox Event Router SMT.
|
|
|
|
The following image shows the overall architecture of this pattern:
|
|
|
|
image:outbox_pattern.png[Outbox Pattern]
|
|
|
|
The Outbox extension's goal is to provide a https://quarkus.io/[Quarkus] application with a reusable, highly configurable component that facilitates the use of the Outbox pattern paired with Debezium's CDC connector pipeline to reliably and asynchronously share data with any consumer of said data.
|
|
|
|
== Getting Started
|
|
|
|
In order to start using the Debezium Outbox Quarkus extension, the extension needs to be added as a part of the Quarkus application as follows:
|
|
[source,xml,subs="verbatim,attributes"]
|
|
----
|
|
<dependency>
|
|
<groupId>io.debezium.quarkus</groupId>
|
|
<artfiactId>debezium-quarkus-outbox</artfiactId>
|
|
<version>{debezium-version}</version>
|
|
</dependency>
|
|
----
|
|
|
|
The extension provides the application with the `io.debezium.outbox.quarkus.ExportedEvent` interface.
|
|
It's expected that an application class will implement this interface and that the event will be emitted using the `javax.enterprise.event.Event` class.
|
|
|
|
[NOTE]
|
|
====
|
|
The `ExportedEvent` interface is parameterized to allow the application to designate the Java types used by several attributes emitted by the event.
|
|
It's important that for a given Quarkus application, *all* implementations of the `ExportedEvent` interface must use the same parameter types or a build failure will be thrown since all events will use the same underlying database table.
|
|
====
|
|
|
|
== Example
|
|
|
|
The following illustrates an implementation of the `ExportedEvent` interface representing an order that has been created:
|
|
|
|
.OrderCreatedEvent.java
|
|
[source,java,indent=0]
|
|
----
|
|
public class OrderCreatedEvent implements ExportedEvent<String, JsonNode> {
|
|
|
|
private static final String TYPE = "Order";
|
|
private static final String EVENT_TYPE = "OrderCreated";
|
|
|
|
private final long orderId;
|
|
private final JsonNode jsonNode;
|
|
private final Instant timestamp;
|
|
|
|
public OrderCreatedEvent(Instant createdAt, Order order) {
|
|
this.orderId = order.getId();
|
|
this.timestamp = createdAt;
|
|
this.jsonNode = convertToJson(order);
|
|
}
|
|
|
|
@Override
|
|
public String getAggregateId() {
|
|
return String.valeuOf(orderId);
|
|
}
|
|
|
|
@Override
|
|
public String getAggregateType() {
|
|
return TYPE;
|
|
}
|
|
|
|
@Override
|
|
public JsonNode getPayload() {
|
|
return jsonNode;
|
|
}
|
|
|
|
@Override
|
|
public String getType() {
|
|
return EVENT_TYPE;
|
|
}
|
|
|
|
@Override
|
|
public Instant getTimestamp() {
|
|
return timestamp;
|
|
}
|
|
}
|
|
----
|
|
|
|
The following example illustrates an `OrderService` that emits the `OrderCreatedEvent`:
|
|
|
|
.OrderService.java
|
|
[source,java,indent=0]
|
|
----
|
|
@ApplicationScoped
|
|
public class OrderService {
|
|
@Inject
|
|
Event<ExportedEvent<?, ?>> event;
|
|
|
|
@Transactional
|
|
public Order addOrder(Order order) {
|
|
order = orderRepository.save(order);
|
|
event.fire(new OrderCreatedEvent(Instant.now(), order));
|
|
return order;
|
|
}
|
|
}
|
|
----
|
|
|
|
When the application code fires the event by calling `Event#fire()`, the Outbox extension will be notified that the event occurred and persists the contents of the event into an outbox event table within the scope of the current transaction.
|
|
The Debezium CDC connector in conjunction with the Outbox Event Router will be monitoring this table and will be responsible for relaying that data using CDC events.
|
|
|
|
To see a full end-to-end demo, the https://github.com/debezium/debezium-examples/tree/master/outbox[Outbox] example illustrates two Quarkus microservice applications using the outbox pattern to share data between them when orders are placed or cancelled.
|
|
|
|
== Configuration
|
|
|
|
The Outbox extension can be configured by setting options in the Quarkus `application.properties` file.
|
|
The extension works out-of-the-box with a default configuration, but this configuration may not be ideal for every situation.
|
|
|
|
=== Build time configuration options
|
|
|
|
[cols="65%a,>15%a,>20%",options="header"]
|
|
|=======================
|
|
|Configuration property
|
|
|Type
|
|
|Default
|
|
|
|
|`quarkus.debezium-outbox.table-name`::
|
|
The table name to be used when creating the outbox table.
|
|
|string
|
|
|OutboxEvent
|
|
|
|
|
|
|`quarkus.debezium-outbox.id.name`::
|
|
The column name for the event id column.
|
|
|string
|
|
|`id`
|
|
|
|
|`quarkus.debezium-outbox.id.column-definition`::
|
|
The database-specific column definition for the event id column.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-id.name`::
|
|
The column name for the event key column.
|
|
|string
|
|
|`aggregateid`
|
|
|
|
|`quarkus.debezium-outbox.aggregate-id.column-definition`::
|
|
The database-specific column definition for the aggregate id.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-id.length`::
|
|
The length of the event key column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-id.scale`::
|
|
The scale of the event key column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-id.precision`::
|
|
The precision of the event key column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-id.converter`::
|
|
The JPA AttributeConverter for the event key column.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-id.nullable`::
|
|
Whether the event key column is nullable.
|
|
|boolean
|
|
|false
|
|
|
|
|`quarkus.debezium-outbox.aggregate-type-name`::
|
|
The column name for the event aggregate type column.
|
|
|string
|
|
|`aggregatetype`
|
|
|
|
|`quarkus.debezium-outbox.aggregate-type.column-definition`::
|
|
The database-specific column definition for the aggregate type.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-type.length`::
|
|
The length of the event aggregate type column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-type.scale`::
|
|
The scale of the event aggregate type column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-type.precision`::
|
|
The precision of the event aggregate type column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-type.converter`::
|
|
The JPA AttributeConverter for the event aggregate type column.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.aggregate-type.nullable`::
|
|
Whether the event aggregate type column is nullable.
|
|
|boolean
|
|
|false
|
|
|
|
|`quarkus.debezium-outbox.type.name`::
|
|
The column name for the event type column.
|
|
|string
|
|
|`type`
|
|
|
|
|`quarkus.debezium-outbox.type.column-definition`::
|
|
The database-specific column definition for the event type.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.type.length`::
|
|
The length of the event type column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.type.scale`::
|
|
The scale of the event type column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.type.precision`::
|
|
The precision of the event type column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.type.converter`::
|
|
The JPA AttributeConverter for the event type column.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.type.nullable`::
|
|
Whether the event type column is nullable.
|
|
|boolean
|
|
|false
|
|
|
|
|`quarkus.debezium-outbox.timestamp.name`::
|
|
The column name for the event timestamp column.
|
|
|string
|
|
|`timestamp`
|
|
|
|
|`quarkus.debezium-outbox.timestamp.column-definition`::
|
|
The database-specific column definition for the event timestamp.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.timestamp.length`::
|
|
The length of the event timestamp column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.timestamp.scale`::
|
|
The scale of the event timestamp column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.timestamp.precision`::
|
|
The precision of the event timestamp column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.timestamp.converter`::
|
|
The JPA AttributeConverter for the event timestamp column.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.timestamp.nullable`::
|
|
Whether the event timestamp column is nullable.
|
|
|boolean
|
|
|false
|
|
|
|
|`quarkus.debezium-outbox.payload.name`::
|
|
The column name for the event payload column.
|
|
|string
|
|
|`payload`
|
|
|
|
|`quarkus.debezium-outbox.payload.column-definition`::
|
|
The database-specific column definition for the event payload.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.payload.length`::
|
|
The length of the event payload column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.payload.scale`::
|
|
The scale of the event payload column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.payload.precision`::
|
|
The precision of the event payload column.
|
|
|integer
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.payload.converter`::
|
|
The JPA AttributeConverter for the event payload column.
|
|
|string
|
|
|
|
|
|
|
|`quarkus.debezium-outbox.payload.nullable`::
|
|
Whether the event payload column is nullable.
|
|
|boolean
|
|
|true
|
|
|
|
|=======================
|
|
|
|
[NOTE]
|
|
====
|
|
The build time configuration defaults will work with the Outbox Event Router SMT out of the box.
|
|
When not using the default values, be sure that the SMT configuration matches.
|
|
====
|
|
|
|
=== Runtime configuration options
|
|
|
|
[cols="65%a,>15%a,>20%",options="header"]
|
|
|=======================
|
|
|Configuration property
|
|
|Type
|
|
|Default
|
|
|
|
|`quarkus.debezium-outbox.remove-after-insert`::
|
|
Whether the outbox entry is removed after having been inserted. +
|
|
+
|
|
_The removal of the entry does not impact the Debezium connector from being able to emit CDC events.
|
|
This is used as a way to keep the table's underlying storage from growing over time._
|
|
|boolean
|
|
|true
|
|
|
|
|======================= |