There is a corner case where it's possible the Oracle connector may query
the Oracle metadata tables quicker than the ARC process can generate an
archive log history record in V$ARCHIVED_LOG, and this can lead to a race
condition where we may incorrectly advance the connector forward to start
mining a group of logs when a log sequence gap exists in the log ranges.
For users who use the online_catalog strategy, there are some checks that
LogMiner does automatically which it skips, and one is with log sequence
gaps. This fix enforces that check by Debezium even for users who may use
the faster online_catalog mode so that no logs are omitted and events
could be missed.
There was a possible situation where if a long transaction consisted of
updating and inserting into the same table with identical keys with a
given sequence that the commit handler would merge several events for a
table without LOB columns, resulting in a difference in expected events
in the Kafka topic vs what was seen in LogMiner.
This config will be re-used by possible other implementations of
DebeiumEngine API in the embedded package. As DebeziumEngine API
can have completely different implementations and thus also config,
the class is called `EmbeddedEngineConfig` as it's assumed to be used
only by embedded engine "family" of implementations.
To keep backward compatibility, the config options are extracted into
an interface and `EmbeddedEngine` implements this interface, thus
allowing to use these options in custom classes without any need for the
code changes.
It is recommended by Infinispan that specific calls that return a collection
of elements should be treated as a closable object so that any and all the
potential resources associated with the operation are closed.
Add a new internal `log.mining.schema_changes.username.exclude.list` to allow users
to customize the default behavior for excluding the SYS and SYSTEM usernames from
DDL changes.
In some corner cases, users may have unusually large SQL statements that
need to be buffered due to the number of columns paired with the data in
those columns. Previously we capped this to 4000*10 or 40kb primarily to
address situations with LOB operations that could lead to OOM scenarios.
The new code rather logs a warning when exceeding 100kb and hard faults
ony when the connector sees Integer.MAX_VALUE number of SQL lines for a
single SQL buffer.
Pending transactions with a START_SCN of 0 are considered transactions
that have started before the oldest available archive log and these
will be ignored as the entire transaction cannot be mined.
