tet123/documentation/modules/ROOT/pages/integrations/outbox.adoc

[id="outbox-quarkus-extension"]
= Outbox Quarkus Extension

:toc:
:toc-placement: macro
:linkattrs:
:icons: font
:source-highlighter: highlight.js

toc::[]

== Overview

This extension is inspired by the xref:transformations/outbox-event-router.adoc[Outbox Event Router] single message transformation (SMT).
As discussed in the blog post link:/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/[Reliable Microservices Data Exchange with the Outbox Pattern], microservices often need to exchange information with one another and an excellent way to deal with that is using the Outbox pattern combined with {prodname}'s Outbox Event Router SMT.

The following image shows the overall architecture of this pattern:

image:outbox_pattern.png[Outbox Pattern]

The Outbox extension's goal is to provide a https://quarkus.io/[Quarkus] application with a reusable, highly configurable component that facilitates the use of the Outbox pattern paired with {prodname}'s CDC connector pipeline to reliably and asynchronously share data with any consumer of said data.

== Getting Started

In order to start using the {prodname} Outbox Quarkus extension, the extension needs to be added as a part of the Quarkus application as follows:
[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>io.debezium</groupId>
  <artifactId>debezium-quarkus-outbox</artifactId>
  <version>{debezium-version}</version>
</dependency>
----

The extension provides the application with the `io.debezium.outbox.quarkus.ExportedEvent` interface.
It's expected that an application class will implement this interface and that the event will be emitted using the `javax.enterprise.event.Event` class.

[NOTE]
====
The `ExportedEvent` interface is parameterized to allow the application to designate the Java types used by several attributes emitted by the event.
It's important that for a given Quarkus application, *all* implementations of the `ExportedEvent` interface must use the same parameter types or a build failure will be thrown since all events will use the same underlying database table.
====

== Example

The following example illustrates an implementation of the `ExportedEvent` interface representing an order that has been created:

.OrderCreatedEvent.java
[source,java,indent=0]
----
public class OrderCreatedEvent implements ExportedEvent<String, JsonNode> {

    private static final String TYPE = "Order";
    private static final String EVENT_TYPE = "OrderCreated";

    private final long orderId;
    private final JsonNode jsonNode;
    private final Instant timestamp;

    public OrderCreatedEvent(Instant createdAt, Order order) {
        this.orderId = order.getId();
        this.timestamp = createdAt;
        this.jsonNode = convertToJson(order);
    }

    @Override
    public String getAggregateId() {
        return String.valueOf(orderId);
    }

    @Override
    public String getAggregateType() {
        return TYPE;
    }

    @Override
    public JsonNode getPayload() {
        return jsonNode;
    }

    @Override
    public String getType() {
        return EVENT_TYPE;
    }

    @Override
    public Instant getTimestamp() {
        return timestamp;
    }

    @Override
    public Map<String, Object> getAdditionalFieldValues() {
        // no additional fields
        return Collections.emptyMap();
    }
}
----

The following example illustrates an `OrderService` that emits the `OrderCreatedEvent`:

.OrderService.java
[source,java,indent=0]
----
@ApplicationScoped
public class OrderService {

    @Inject
    OrderRepository orderRepository;

    @Inject
    Event<ExportedEvent<?, ?>> event;

    @Transactional
    public Order addOrder(Order order) {
        order = orderRepository.save(order);
        event.fire(new OrderCreatedEvent(Instant.now(), order));
        return order;
    }
}
----

When the application code fires the event by calling `Event#fire()`, the Outbox extension will be notified that the event occurred and persists the contents of the event into an outbox event table within the scope of the current transaction.
The {prodname} CDC connector in conjunction with the Outbox Event Router will be monitoring this table and will be responsible for relaying that data using CDC events.

To see a full end-to-end demo, the https://github.com/debezium/debezium-examples/tree/main/outbox[Outbox] example illustrates two Quarkus microservice applications using the outbox pattern to share data between them when orders are placed or cancelled.

[id=reactive-outbox]
== Reactive Variant

If your application uses reactive datasources, or Hibernate Reactive, you must use a slightly different configuration to add the extension to your application.

For example, use the following configuration to  import the reactive variant of the extension:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>io.debezium</groupId>
  <artifactId>debezium-quarkus-outbox-reactive</artifactId>
  <version>{debezium-version}</version>
</dependency>
----

The reactive extension provides the application with the same `io.debezium.outbox.quarkus.ExportedEvent` interface as the non-reactive variant, but it also provides the `DebeziumOutboxHandler` class.
Injecting the `DebeziumOutboxHandler` bean into an application provides a method for quickly persisting an `ExportedEvent` to the outbox table.
The following example shows an invocation that uses the same `ExportedEvent` as in the earlier example:

.OrderService.java
[source,java,indent=0]
----
@ApplicationScoped
public class OrderService {

    @Inject
    DebeziumOutboxHandler debeziumOutboxHandler;

    @Inject
    OrderRepository orderRepository;

    @Inject
    Event<ExportedEvent<?, ?>> event;

    @ReactiveTransactional
    public Uni<Order> addOrder(Order order) {
        return orderRepository.persistAndFlush(order)
        .call(debeziumOutboxHandler.persistToOutbox(new OrderCreatedEvent(Instant.now(), order)));

    }
}
----

[NOTE]
====
The `persistToOutbox` method returns a Uni; therefore, it must be observed by a subscriber ito receive the result.
For more information about using the Mutiny library to build reactive applications, see the link:https://quarkus.io/guides/mutiny-primer[Quarkus Mutiny primer].
====



[id=additional-field-mappings]
== Additional Field mappings

The {prodname} Outbox SMT can be configured to read additional fields and emit those field values either as event headers, or as part of the event value.

In order to pass additional field mappings to be saved by the Quarkus Outbox extension,
the configuration property `quarkus.debezium-outbox.additional-fields` must be specified in the `application.properties`.
This configuration property is a comma-separated list of additional field definitions that will be added to the Outbox entity mapping and passed by the application's implementation of the `ExportedEvent` interface.

Each entry in this comma-separated list must follow this format:
[source]
----
<field-name>:<field-java-type>[:<field-column-definition>[:<field-jpa-attribute-converter>]]
----

The pattern indicates that the field's name and java-type are required while the column definition and JPA attribute converter are optional.
However, please note that if you wish to specify a JPA attribute converter then the column definition must be specified.

The following example shows how to define an additional field called `customer_name` that is represented in Java as a `String` and which should be stored in the outbox table as a `VARCHAR(100)` column.
This example also shows a JPA Attribute converter defined that forces the storage of the string to upper-case.

.application.properties
[source,properties,indent=0]
----
quarkus.debezium-outbox.additional-fields=customer_name:string:varchar(100):example.UpperCase
----

Once the field(s) are configured in the application's `.properties` file, the application's code needs to provide the corresponding values through its exported events.
In order to do this, the application class that extends the `ExportedEvent` needs to override the method called `getAdditionalFieldValues()` and return a `Map` of the additional field names and values.

In the following example, we show how to specify the `customer_name` field with a value of `Acme Goods`.
Using our `OrderCreatedEvent` from the example section above, we've extended the event:

.OrderCreatedEvent.java
[source,java,indent=0]
----
public class OrderCreatedEvent implements ExportedEvent<String, JsonNode> {
    ...
    @Override
    public Map<String, Object> getAdditionalFieldValues() {
        return Collections.singletonMap("customer_name", "Acme Goods");
    }
}
----

[NOTE]
====
Additional field mappings do allow specifying a JPA attribute converter per field.

In this example, we defined `example.UpperCase` that will convert any supplied string-value to upper-case prior to insertion.
A JPA attribute converter allows decoupling this type of behavior from the call site, allowing reuse of a common behavior.
====

With the configuration in the application's `.properties` file and updating of `OrderCreateedEvent` to provide these additional fields and values,
the {prodname} Outbox SMT now can access these additional field values and place them in the emitted event.

== Configuration

The Outbox extension can be configured by setting options in the Quarkus `application.properties` file.
The extension works out-of-the-box with a default configuration, but this configuration may not be ideal for every situation.

=== Build time configuration options

[cols="65%a,>12%a,>23%"]
|===
|Configuration property
|Type
|Default

|[[quarkus-debezium-outbox-table-name]]<<quarkus-debezium-outbox-table-name,`+quarkus.debezium-outbox.table-name+`>>::
The table name to be used when creating the outbox table.
|string
|OutboxEvent


|[[quarkus-debezium-outbox-id-name]]<<quarkus-debezium-outbox-id-name,`+quarkus.debezium-outbox.id.name+`>>::
The column name for the event id column. +
for example, `uuid`
|string
|`id`

|[[quarkus-debezium-outbox-id-column-definition]]<<quarkus-debezium-outbox-id-column-definition,`+quarkus.debezium-outbox.id.column-definition+`>>::
The database-specific column definition for the event id column. +
for example, `uuid not null`
|string
|`UUID NOT NULL`

|[[quarkus-debezium-outbox-aggregate-id-name]]<<quarkus-debezium-outbox-aggregate-id-name,`+quarkus.debezium-outbox.aggregate-id.name+`>>::
The column name for the event key column.
|string
|`aggregateid`

|[[quarkus-debezium-outbox-aggregate-id-column-definition]]<<quarkus-debezium-outbox-aggregate-id-column-definition,`+quarkus.debezium-outbox.aggregate-id.column-definition+`>>::
The database-specific column definition for the aggregate id. +
for example, `varchar(50) not null`
|string
|`VARCHAR(255) NOT NULL`

|[[quarkus-debezium-outbox-aggregate-id-converter]]<<quarkus-debezium-outbox-aggregate-id-converter,`+quarkus.debezium-outbox.aggregate-id.converter+`>>::
The JPA AttributeConverter for the event key column. +
for example, `com.company.TheAttributeConverter`
|string
|

|[[quarkus-debezium-outbox-aggregate-type-name]]<<quarkus-debezium-outbox-aggregate-type-name,`+quarkus.debezium-outbox.aggregate-type.name+`>>::
The column name for the event aggregate type column.
|string
|`aggregatetype`

|[[quarkus-debezium-outbox-aggregate-type-column-definition]]<<quarkus-debezium-outbox-aggregate-type-column-definition,`+quarkus.debezium-outbox.aggregate-type.column-definition+`>>::
The database-specific column definition for the aggregate type. +
for example, `varchar(15) not null`
|string
|`VARCHAR(255) NOT NULL`

|[[quarkus-debezium-outbox-aggregate-type-converter]]<<quarkus-debezium-outbox-aggregate-type-converter,`+quarkus.debezium-outbox.aggregate-type.converter+`>>::
The JPA AttributeConverter for the event aggregate type column. +
for example, `com.company.TheAttributeConverter`
|string
|

|[[quarkus-debezium-outbox-type-name]]<<quarkus-debezium-outbox-type-name,`+quarkus.debezium-outbox.type.name+`>>::
The column name for the event type column.
|string
|`type`

|[[quarkus-debezium-outbox-type-column-definition]]<<quarkus-debezium-outbox-type-column-definition,`+quarkus.debezium-outbox.type.column-definition+`>>::
The database-specific column definition for the event type. +
for example, `varchar(50) not null`
|string
|`VARCHAR(255) NOT NULL`

|[[quarkus-debezium-outbox-type-converter]]<<quarkus-debezium-outbox-type-converter,`+quarkus.debezium-outbox.type.converter+`>>::
The JPA AttributeConverter for the event type column. +
for example, `com.company.TheAttributeConverter`
|string
|

|[[quarkus-debezium-outbox-timestamp-name]]<<quarkus-debezium-outbox-timestamp-name,`+quarkus.debezium-outbox.timestamp.name+`>>::
The column name for the event timestamp column.
|string
|`timestamp`

|[[quarkus-debezium-outbox-timestamp-column-definition]]<<quarkus-debezium-outbox-timestamp-column-definition,`+quarkus.debezium-outbox.timestamp.column-definition+`>>::
The database-specific column definition for the event timestamp. +
for example, `timestamp not null`
|string
|`TIMESTAMP NOT NULL`

|[[quarkus-debezium-outbox-timestamp-converter]]<<quarkus-debezium-outbox-timestamp-converter,`+quarkus.debezium-outbox.timestamp.converter+`>>::
The JPA AttributeConverter for the event timestamp column. +
for example, `com.company.TheAttributeConverter`
|string
|

|[[quarkus-debezium-outbox-payload-name]]<<quarkus-debezium-outbox-payload-name,`+quarkus.debezium-outbox.payload.name+`>>::
The column name for the event payload column.
|string
|`payload`

|[[quarkus-debezium-outbox-payload-column-definition]]<<quarkus-debezium-outbox-payload-column-definition,`+quarkus.debezium-outbox.payload.column-definition+`>>::
The database-specific column definition for the event payload. +
for example, `text not null`
|string
|`VARCHAR(8000)`

|[[quarkus-debezium-outbox-payload-converter]]<<quarkus-debezium-outbox-payload-converter,`+quarkus.debezium-outbox.payload.converter+`>>::
The JPA AttributeConverter for the event payload column. +
for example, `com.company.TheAttributeConverter`
|string
|

|[[quarkus-debezium-outbox-payload-type]]<<quarkus-debezium-outbox-payload-type,`+quarkus.debezium-outbox.payload.type+`>>::
A fully-qualified class name of a Hibernate https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#basic-custom-type[user type] implementation. +
for example, `io.company.types.JsonNodeBinaryType`
|string
|

|[[quarkus-debezium-outbox-tracing-span-name]]<<quarkus-debezium-outbox-tracing-span-name,`+quarkus.debezium-outbox.tracing-span.name+`>>::
The column name for the tracing span context column.
|string
|`tracingspancontext`

|[[quarkus-debezium-outbox-tracing-span-column-definition]]<<quarkus-debezium-outbox-tracing-span-column-definition,`+quarkus.debezium-outbox.tracingspancontext.column-definition+`>>::
The database-specific column definition for the tracing span context column. +
for example, `text not null`
|string
|`VARCHAR(256)`

|[[quarkus-debezium-outbox-additional-fields]]<<quarkus-debezium-outbox-additional-fields,`+quarkus.debezium-outbox.additional-fields+`>>::
A comma-separated list of additional field mappings that will be persisted in the outbox table. +
 +
See xref:#additional-field-mappings[additional field mappings] for details on format and usage.
|string
|

|===

[NOTE]
====
The build time configuration defaults will work with the Outbox Event Router SMT out of the box.
When not using the default values, be sure that the SMT configuration matches.
====

=== Runtime configuration options

[cols="65%a,>15%a,>20%"]
|===
|Configuration property
|Type
|Default

|[[quarkus-debezium-outbox-remove-after-insert]]<<quarkus-debezium-outbox-remove-after-insert,`+quarkus.debezium-outbox.remove-after-insert+`>>::
Whether the outbox entry is removed after having been inserted. +
+
_The removal of the entry does not impact the {prodname} connector from being able to emit CDC events.
This is used as a way to keep the table's underlying storage from growing over time._
|boolean
|true

|===

=== Distributed tracing

The extension has support for the distributed tracing.
See link:/documentation/reference/integrations/tracing[tracing documentation] for more details.