jchipmunk
3ec0b3be81
DBZ-918 Moving connector identifier from SourceInfo to Module class
2018-10-04 09:00:13 +02:00
jchipmunk
05ba99bb0a
DBZ-918 Adding Debezium connector field to source info
...
This will allow consumers to recognize the Debezium connector used for creating a given message, helping them to adjust their behavior for a variety of connectors.
2018-10-04 09:00:13 +02:00
Gunnar Morling
984081dd1a
DBZ-926 Adding flushed LSN to log message
2018-10-03 09:47:38 +02:00
Gunnar Morling
08ec4f48d0
DBZ-911 Style adjustments; wording fix
2018-10-01 12:49:56 +02:00
Ian Axelrod
427249887b
DBZ-911 Simplifying schemaChanged() shortcircuit logic
...
Break up initial conditional and comment missing column portion to
clarify the role of skipRefreshSchemaOnMissingToastableData.
2018-10-01 12:49:48 +02:00
Ian Axelrod
9f083d0f1b
DBZ-911 Simplifying getToastableColumnsForTableId() impl.
...
No need to wrap ArrayList instance in unmodifiableList. Can just use
Collections.emptyList.
2018-10-01 12:49:36 +02:00
Ian Axelrod
ad433c5747
DBZ-911 Don't refresh schema on absent TOAST cols
...
Improves PostgreSQL RecordsStreamProducer performance for processing up
dates to a table with TOASTable columns, where those updates do not
affect said columns. Prior to this fix, these updates would trigger a
refresh of the in-memory table schema. In the worst case, this means a
query for every update. This puts significant load on the database
server and adds tens of milliseconds to the processing of each update
record.
The fix requires a new configuration option, called
'schema.refresh.mode'. This option has values 'columns_diff' (the
default) and 'columns_diff_exclude_unchanged_toast'. 'columns_diff'
maintains the pre-fix behavior. 'columns_diff_exclude_unchanged_toast'
activates the fix.
The fix must be toggleable because it decreases the consistency guarantees
Debezium provides, since in-memory table schemas may not stay
synchronized with their remote counterparts. With type metadata included
in the replication message, inconsistencies are limited to the unchanged
toast columns.
2018-10-01 11:39:32 +02:00
Gunnar Morling
6f7bdbe5e1
DBZ-898 Using 'Json' semantic type when encoding HSTORE column values as string
2018-09-25 12:22:20 +02:00
Gunnar Morling
7150e387c7
DBZ-898 Misc. clean-up:
...
* Using database encoding for string conversion
* Not making hstore schemas optional by default
* Using Jackson instead of GSon for JSON serialization
* Removing superfluous method and log messages
* Adjusting to naming and style conventions
2018-09-24 13:36:28 +02:00
SyedMuhammadSufyian
623cc98cb0
Change log levels
2018-09-22 20:15:07 +02:00
SyedMuhammadSufyian
346f2b3761
Handling HStore type , hstore.handling.mode:map|json , default is map.Added support for protobuff
2018-09-22 20:07:13 +02:00
SyedMuhammadSufyian
38f4a0e65b
HStore Data type support , configuration options are map|json , default is map
2018-09-22 05:54:45 +02:00
Gunnar Morling
f9ede861ba
DBZ-912 Upgrading to Postgres driver 42.2.5
2018-09-19 15:26:44 +02:00
Gunnar Morling
39a47fabfa
DBZ-899 Using custom statement to obtain SQL type info in order to avoid N+1 issues with PgDatabaseMetaData#getTypeInfo()
2018-09-19 09:51:38 +02:00
Gunnar Morling
3331d4c1a9
DBZ-899 Removing unused field;
...
Also inlining method only used in one place.
2018-09-19 09:51:38 +02:00
Jiri Pechanec
59fbcca725
DBZ-907 Log refreshing schema due to different column count
2018-09-12 19:07:01 +02:00
Jiri Pechanec
e20c183d5d
DBZ-895 Make tcpKeepalive enabled by default
2018-09-11 09:34:21 +02:00
Gunnar Morling
e1bbbd7992
DBZ-865 DBZ-878 Replacing invalid topic name characters with "_";
2018-09-05 12:56:53 +02:00
Gunnar Morling
bd692b8b84
DBZ-862 Extracting values for controlling retry attempts into constants
2018-09-03 14:50:54 +02:00
Jiri Pechanec
c6e20d4967
DBZ-862 Fix test interdependency problem
2018-09-03 14:40:57 +02:00
Jiri Pechanec
1ff53e35ec
DBZ-862 Fix Travis timing issue
2018-09-03 14:40:57 +02:00
Jiri Pechanec
4b9141e1fd
DBZ-862 Handle concurrent transaction blocking taking snapshot
2018-09-03 14:40:57 +02:00
Jiri Pechanec
e65fb26ae9
DBZ-887 Close connection even for non-started producer
2018-08-31 13:02:23 +02:00
Gunnar Morling
ae879cf475
DBZ-859 Adjustments after rebase; typo fix
2018-08-23 12:00:44 +02:00
Jiri Pechanec
cd6dc09e72
DBZ-859 Heartbeat is sent after PostgreSQL snapshot
2018-08-23 11:44:53 +02:00
oripwk
7d958a1928
DBZ-866 PR fixes
2018-08-22 17:09:26 +02:00
oripwk
d82372a6b2
DBZ-866 Add schema, database and table to Postgres connector
2018-08-22 17:09:26 +02:00
Jiri Pechanec
d4e64a61cb
DBZ-705 Allow to configure postgres config file location
2018-08-13 20:21:41 +02:00
Jenkins user
e00dad127f
[maven-release-plugin] prepare for next development iteration
2018-07-26 08:00:12 +00:00
Jenkins user
16bfd5c700
[maven-release-plugin] prepare release v0.9.0.Alpha1
2018-07-26 08:00:12 +00:00
Gunnar Morling
3061c1dc0b
DBZ-815 Passing offset to Heartbeat#heartbeat();
...
That's simpler to grasp than the approach of passing a supplier lambda to the constructor.
Also it allows to pass on the offset via local variables instead of instance fields in some cases.
2018-07-25 14:49:52 +02:00
Gunnar Morling
8e1450bc71
DBZ-800 Adding test for receiving heartbeat events with Postgres
2018-07-20 13:37:54 +02:00
Gunnar Morling
4abc471bcb
DBZ-800 Emitting heartbeat events in Postgres streaming loop
2018-07-20 13:37:54 +02:00
Gunnar Morling
20db9299c5
DBZ-627 Unifying TopicSelector implementations
2018-07-18 12:36:07 +02:00
Gunnar Morling
1523f230ca
DBZ-793 Using TableFilter consistently;
...
Only the MySQL connector's Filters class still uses Predicate for the time being, so to avoid to much merging trouble with DBZ-175.
2018-07-11 13:16:44 +02:00
Jenkins user
f9b8d830a8
[maven-release-plugin] prepare for next development iteration
2018-07-11 07:36:30 +00:00
Jenkins user
290ded678f
[maven-release-plugin] prepare release v0.8.0.Final
2018-07-11 07:36:29 +00:00
Jenkins user
033db6659d
[maven-release-plugin] prepare for next development iteration
2018-07-04 07:07:44 +00:00
Jenkins user
696f35f2c0
[maven-release-plugin] prepare release v0.8.0.CR1
2018-07-04 07:07:44 +00:00
Jiri Pechanec
c2f2f686f0
DBZ-644 Disable test making problems in Travis CI
2018-07-03 07:03:59 +02:00
Jiri Pechanec
646a66672c
DBZ-644 Close unused replication connection
2018-07-03 07:03:59 +02:00
Gunnar Morling
dcd6b01134
DBZ-644 Simplifying RelationalDatabaseSchema constructor;
...
Also retrieving logical name via CommonConnectorConfig#getLogicalName() in more places
2018-07-03 07:03:59 +02:00
Gunnar Morling
b0980b994c
DBZ-644 Adding test for Postgres
2018-07-03 07:03:59 +02:00
Gunnar Morling
8744488c8a
DBZ-644 Unifying retrieval of ColumnMappers in RelationalDatabaseSchema
2018-07-03 07:03:59 +02:00
Jiri Pechanec
0182eecd06
DBZ-778 Make the tests resilient against race condition
2018-07-02 11:52:23 +02:00
Gunnar Morling
a7be3847e3
DBZ-773 Removing some dead code and unused methods
2018-06-29 09:27:57 +02:00
Gunnar Morling
c4c8cbc3ab
DBZ-773 Moving management of Tables to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
bbfbdf6fab
DBZ-773 Moving management of table schemas to RelationalDatabaseSchema
2018-06-29 09:09:10 +02:00
Gunnar Morling
0600ccf213
DBZ-751 Incorporating review remarks;
...
* Expanding test
* Extracting constant for parameter key in tests
2018-06-27 14:48:59 +02:00
Gunnar Morling
a292b05a96
DBZ-751 Propagating DECIMAL column precision to Avro schemas
2018-06-27 14:48:59 +02:00
Jiri Pechanec
ca6b1c9168
DBZ-743 Empty password for PostgreSQL and MySQL
2018-06-27 11:18:19 +02:00
Gunnar Morling
f9c1cfc134
DBZ-762 Adding support for CITEXT array columns
2018-06-26 11:40:19 +02:00
Gunnar Morling
daeeaa74cd
DBZ-762 Formatting
2018-06-26 11:40:19 +02:00
Olavi Mustanoja
c401a98166
DBZ-762 Added support for CITEXT column type
2018-06-26 11:40:19 +02:00
Gunnar Morling
bf7a5018ca
Setting POM version back to 0.8.0-SNAPSHOT
2018-06-22 12:21:03 +02:00
Jenkins user
db42e4657a
[maven-release-plugin] prepare for next development iteration
2018-06-21 14:07:45 +00:00
Jenkins user
c4b8ecaa99
[maven-release-plugin] prepare release v0.8.0.Beta1
2018-06-21 14:07:45 +00:00
Gunnar Morling
987e94cfbb
DBZ-578 Removing unnecessary "server.zone.offset" option;
...
This reverts parts of the previous work done for DBZ-578.
TIMESTAMP columns (logical date + time without time zone)
are always interpreted using UTC again.
2018-06-20 15:58:19 +02:00
Gunnar Morling
b83f1493a2
DBZ-20 Avoiding Optional as method parameter
2018-06-20 13:05:37 +02:00
Gunnar Morling
dce1aa29f1
DBZ-20 Oracle precision supports negative numbers
2018-06-20 13:05:37 +02:00
Jiri Pechanec
89694ffbbf
DBZ-20 Support for year to month interval
2018-06-20 13:05:37 +02:00
Gunnar Morling
d7e196a18e
DBZ-20 Initial import of Oracle connector based on XStream
2018-06-20 13:05:37 +02:00
Gunnar Morling
9eb4b90ec9
DBZ-191 Adding record validations
2018-06-15 20:23:17 +02:00
Jiri Pechanec
1e678d9959
DBZ-679 Make streaming decoding optional
2018-06-15 08:35:50 +02:00
Jiri Pechanec
7ee1f98f57
DBZ-679 Support decoding of out-of-order fragments
2018-06-15 08:35:50 +02:00
Jiri Pechanec
eadf65cd27
DBZ-578 Build failing on Jenkins CI
2018-06-13 14:43:29 +02:00
Gunnar Morling
a9d3d0c95d
DBZ-711 Using correct SSL mode names
2018-06-13 10:12:14 +02:00
Jiri Pechanec
29f8891b4f
DBZ-578 Process timestamps without timezones as in server timezone
2018-06-11 15:27:08 +02:00
Jiri Pechanec
4111ddbcaf
DBZ-709 Add test for timestamp with precision
2018-06-05 15:51:35 +02:00
Jiri Pechanec
176dcbbc0b
DBZ-709 PostgreSQL uses scale for time precision not length
2018-06-05 15:51:35 +02:00
Jiri Pechanec
525d073fea
DBZ-681 Time with precision for wal2json plug-in
2018-06-04 14:50:16 +02:00
Gunnar Morling
8d7806e71a
DBZ-693 Expanding test; updating connector option description
2018-05-28 12:35:47 +02:00
Jiri Pechanec
45bc7b1536
DBZ-693 On connect statements supported for PostgreSQL
2018-05-28 12:35:47 +02:00
Jiri Pechanec
e2603d392e
DBZ-699 Handle all exceptions
2018-05-24 13:50:16 +02:00
Jiri Pechanec
e533b75cbf
DBZ-699 Close JDBC connection when initialization fails
2018-05-24 09:49:42 +02:00
Jiri Pechanec
fa2691ffd1
DBZ-701 Support for arrays with large OID
2018-05-23 22:13:21 +02:00
Jiri Pechanec
f844054f50
DBZ-696 Support nanoseconds in timestamp data
2018-05-22 11:33:50 +02:00
Jiri Pechanec
09b28c0913
DBZ-694 Support milliseconds for pre-epoch dates
2018-05-22 11:22:06 +02:00
Jiri Pechanec
37cfe622fc
DBZ-697 Support for large type OIDs
2018-05-22 10:39:14 +02:00
Jiri Pechanec
2706858cf4
DBZ-676 Support testing against 8bit enc database
2018-04-06 18:26:45 +02:00
Gunnar Morling
9a29063ef4
DBZ-673 Converting tx id to long in one more place
2018-03-27 10:04:43 +02:00
listman
aa4bf5d367
DBZ-673 txId must be long in wal2jon decoder
2018-03-27 00:31:03 -05:00
Gunnar Morling
bdb1069b0d
DBZ-638 Logging message on TRACE level instead of DEBUG
2018-03-23 16:38:05 +01:00
Jiri Pechanec
363921a7b1
DBZ-638 Chunk processing operates on byte array
2018-03-23 14:41:59 +01:00
Jiri Pechanec
4051a4a439
DBZ-638 Improved variable naming
2018-03-23 14:41:59 +01:00
Jiri Pechanec
ce5196dc3e
DBZ-638 Keep non-streaming decoding as a fallback
2018-03-23 14:41:59 +01:00
Jiri Pechanec
24965dd076
DBZ-638 Process tx as stream not as batch
2018-03-23 14:41:59 +01:00
Jiri Pechanec
0743467102
DBZ-671 Date/time parsing covers more exotic formats
2018-03-23 08:55:24 +01:00
Jenkins user
f4e151b23a
[maven-release-plugin] prepare for next development iteration
2018-03-20 08:14:19 +00:00
Jenkins user
93b3252332
[maven-release-plugin] prepare release v0.7.5
2018-03-20 08:14:19 +00:00
Jiri Pechanec
40a527b6ea
DBZ-592 Make snapshot/init-sync false instead of null
2018-03-19 13:19:34 +01:00
Jenkins user
daf27207be
[maven-release-plugin] prepare for next development iteration
2018-03-07 08:31:07 +00:00
Jenkins user
9c73774928
[maven-release-plugin] prepare release v0.7.4
2018-03-07 08:31:07 +00:00
Gunnar Morling
1d5d6b6f3a
DBZ-635 Schema for geo-spatial array columns shouldn't be optional by default
2018-03-07 07:17:43 +01:00
Gunnar Morling
ff524695bc
DBZ-635 Schema for NUMERIC columns shouldn't be optional by default
2018-03-07 07:17:43 +01:00
Gunnar Morling
6412e9e06e
DBZ-605 Schema for BYTEA columns shouldn't be optional by default
2018-03-07 07:17:43 +01:00
Jiri Pechanec
327085164e
DBZ-611 Special values supported for double
2018-03-06 16:55:24 +01:00
Jiri Pechanec
424aefbbb4
DBZ-611 Decimal-> String supported for MySQL
2018-03-06 16:55:24 +01:00
Gunnar Morling
6e29b82358
DBZ-351 Avoiding call to new BigDecimal() for NaN
2018-02-27 16:36:21 +01:00
Gunnar Morling
5786d44200
DBZ-351 Avoiding call to getBigDecimal() for NaN
2018-02-27 16:36:21 +01:00
Gunnar Morling
d040ec4427
DBZ-351, DBZ-606 Misc. clean-up;
...
* Renaming DebeziumDecimal to SpecialValueDecimal
* Simplifying logic in PostgresValueConverter
2018-02-27 16:36:21 +01:00
Jiri Pechanec
2534696b21
DBZ-606 Exception thrown for special values when mode handling is not string
2018-02-27 16:36:21 +01:00
Jiri Pechanec
5827a7ac1b
DBZ-611 Decimals can be encoded as strings
2018-02-27 16:36:21 +01:00
Jiri Pechanec
c71707afc9
DBZ-351 wal2json usses unlimited precision too
2018-02-27 16:36:21 +01:00
Jiri Pechanec
0c0ac72568
DBZ-606 Type for handling special BigDecimals
2018-02-27 16:36:21 +01:00
Jiri Pechanec
ceaa7f8efe
DBZ-351 Numeric is passed with unlimited precision
2018-02-27 16:36:21 +01:00
Jiri Pechanec
4623c1d8c4
DBZ-606 Numeric NaN represented as null
2018-02-27 16:36:21 +01:00
Gunnar Morling
c900c7fc92
DBZ-623 Formatting and indentation
2018-02-22 21:16:00 +01:00
Jiri Pechanec
01e5f8da8c
DBZ-623 Extract parsing code to a separate method
2018-02-22 21:12:05 +01:00
Jiri Pechanec
80c74a9d7c
DBZ-623 Fix NPE for invlid restored LSNs
2018-02-22 21:12:05 +01:00
Gunnar Morling
eef1ad7c2a
DBZ-630 Refactoring around MongoDB task context;
...
* Renaming ConnectorTaskContext to CdcSourceTaskContext
* Renaming ReplicationContext to MongoDbTaskContext
* Making relationship from MongoDbTaskContext to ConnectionContext has-a instead of is-a
2018-02-21 12:04:41 +01:00
Gunnar Morling
bb12e521f8
DBZ-630 Pulling up getClock() to ConnectorTaskContext
2018-02-21 12:04:41 +01:00
Gunnar Morling
8264c20cf6
DBZ-630 Unifying common start-up logic across connectors
2018-02-21 12:04:41 +01:00
Jiri Pechanec
4e74e0a4ac
Revert default Postgres decoder plugin for testing
2018-02-20 10:29:06 +01:00
Jiri Pechanec
ad181493da
DBZ-626 Make caller method naming consistent
2018-02-20 09:26:47 +01:00
Gunnar Morling
7fe45db831
DBZ-626 Renaming AvroValidator to SchemaNameAdjuster;
...
Making apparent that this functionality doesn't solely validate but returns a new name actually
2018-02-20 09:26:47 +01:00
Gunnar Morling
cddd5fcc8a
DBZ-626 Passing schema validator as AvroValidator instance instead of Function
2018-02-20 09:26:47 +01:00
Jiri Pechanec
40a5422998
DBZ-605 Support for BYTEA datatype
2018-02-20 08:55:56 +01:00
Jiri Pechanec
6e00924ee9
DBZ-625 Fix parsing of float without decimals
2018-02-20 08:47:43 +01:00
Gunnar Morling
8369f10d52
DBZ-628 Adding MAX_QUEUE_SIZE, MAX_BATCH_SIZE and POLL_INTERVAL_MS to CommonConnectorConfig;
...
Also using these options in PG connector
2018-02-20 06:34:42 +01:00
Gunnar Morling
ab11f48843
DBZ-620 Creating envelope schemas only once per table type
2018-02-19 12:18:37 +01:00
Jenkins user
6d0cd88e12
[maven-release-plugin] prepare for next development iteration
2018-02-15 04:15:34 +00:00
Jenkins user
7d1e1a989e
[maven-release-plugin] prepare release v0.7.3
2018-02-15 04:15:34 +00:00
Gunnar Morling
cc64a7a616
DBZ-618 Removing unused method and field
2018-02-14 16:53:52 +01:00
Gunnar Morling
f54ed51897
DBZ-618 Removing superfluous method parameter
2018-02-14 16:43:52 +01:00
Jiri Pechanec
d0198e3e26
DBZ-618 Incorrect type retrieved by stream producer for time/date columns
2018-02-14 13:06:50 +01:00
Gunnar Morling
0f726b5f4e
DBZ-616 Making PostgresType#equals() and hashCode() symmetric; logging improvements
2018-02-14 10:54:39 +01:00
Jiri Pechanec
2a08a718ba
DBZ-616 Get default length and scale for types
2018-02-14 10:48:13 +01:00
Gunnar Morling
bda12b59ee
DBZ-571 Misc. improvements;
...
* More consistent variable names
* Making type registry truly immutable
* Avoiding call to instance method from constructor
2018-02-13 16:15:10 +01:00
Jiri Pechanec
5a0c7c07b0
DBZ-609 DBZ-571 Refactor Postgres type handling
...
TypeRegistry introduced for Postgres connector
JDBC column does not have a special componentType
JDBC column provide a database specific type id
OID is a primary type identifier to be used in Postgres connector code - dropping JDB/OID dichotomy
2018-02-13 14:54:42 +01:00
Gunnar Morling
2cd1a43db8
DBZ-220 Removing println() call
2018-02-13 12:31:36 +01:00
Jiri Pechanec
682cabb84a
DBZ-220 Generate heartbeat events in binlog thread
2018-02-13 12:24:51 +01:00
Gunnar Morling
f3ea0b5683
DBZ-582 Supporting TOMBSTONES_ON_DELETE option with Postgres connector
2018-02-13 10:19:18 +01:00
Gunnar Morling
a77c8d733a
DBZ-580, DBZ-586 Making enqueue() and thus snapshots in Postgres interruptable;
...
* ChangeEventQueue#enqueue() checks the interrupted state of the calling
thread now, raising an InterruptedException in case the interrupted flag
has been set (because the producer's thread executor has been stopped)
* RecordSnapshotProducer has been adjusted to check for the interrupted
regularly, aborting if it has been set
2018-02-11 10:05:06 +01:00
Gunnar Morling
45a9847d42
DBZ-580 Initial implementation of dedicated change message queue:
...
* To be used across connectors, unifying the enqueuing/polling logic and handling of related config options
* Using it for Postgres connector
2018-02-11 10:05:06 +01:00
Jiri Pechanec
6150955a47
DBZ-590 Detect non-available offset
2018-02-05 12:20:47 +01:00
Gunnar Morling
712824e43f
DBZ-593 Passing version explicitly from SourceInfos to base class instead of relying specific packaging structure
2018-02-02 13:16:43 +01:00
Jiri Pechanec
7ebee94169
DBZ-593 Add Debezium version to source in envelope
2018-02-02 13:16:43 +01:00
Jiri Pechanec
9b592204ac
DBZ-587 Centralize and unify thread management
2018-02-01 10:04:20 +01:00
Jiri Pechanec
2c74dd74a3
DBZ-577 Numeric arrays correctly working
2018-01-29 15:14:11 +01:00
Jenkins user
04624341f5
[maven-release-plugin] prepare for next development iteration
2018-01-25 09:39:44 +00:00
Jenkins user
898f6884e1
[maven-release-plugin] prepare release v0.7.2
2018-01-25 09:39:44 +00:00
Robert Coup
e3fbf165ed
DBZ-507 Temporarily disabling geometry/geography array support in postgres until array support is improved.
...
Test tidyups.
2018-01-24 16:51:54 +01:00
Robert Coup
f9d90a482f
DBZ-507 Expanding support for geometry types;
...
* Adding support for PostGIS geometry types
* Adding support for GEOMETRY, POLYGON and more in MySQL
* For all newly supported types, changes are represented using two new schema types Geometry and Geography, containing the WKB (binary geo data) ans srid (coord system identifier)
* The existing Point type also contains the new (optional) srid field
2018-01-24 16:51:49 +01:00
Jiri Pechanec
5ff2cf4dd8
DBZ-558 Return correct array OID
2018-01-23 19:34:39 +01:00
Jiri Pechanec
fe16050cdb
DBZ-521 SourceRecord is now associated with metadata
2018-01-19 21:18:55 +01:00
Gunnar Morling
6f6ce33958
DBZ-521 Misc. improvements and fixes;
...
* Actually assing last processed LSN to commit()
* passing LSN to commit() instead of entire SourceRecord
* Avoiding exception in case of wal2json batch with exactly one element
* Using "Lsn" instead of "LSN" in method names
* JavaDoc clarifications
2018-01-19 21:18:55 +01:00
Jiri Pechanec
0b5348339a
DBZ-521 Record LSN for the last message committed to Connect
2018-01-19 21:18:55 +01:00
Jiri Pechanec
c3edf8ee9c
DBZ-504 Type changes detected for constraints
2018-01-19 14:45:08 +01:00
Gunnar Morling
0c4190c493
DBZ-516 Using Duration instead of long in a few more places
2018-01-18 14:13:58 +01:00
Jiri Pechanec
24bdcaf059
DBZ-516 Return control to Connect periodically
2018-01-18 14:13:58 +01:00
Gunnar Morling
5c88431c07
DBZ-494 Making tests more lenient towards specific List implementations;
...
also fixing a few typos.
2018-01-15 10:40:50 +01:00
Jiri Pechanec
5756f6dc40
DBZ-494 Array elements goes through value conversion
2018-01-15 10:24:54 +01:00
Jiri Pechanec
a629591f1f
DBZ-506 Support for char and varchar arrays
2018-01-05 13:52:19 +01:00
Jiri Pechanec
ac5ed3bd92
DBZ-510 snapshot.select.statement.overrides for Postgres
2018-01-05 04:15:36 +01:00
jchipmunk
8df63e894d
DBZ-519 NullPointerException happened for PAUSED task
...
PostgresConnectorTask throws java.lang.NullPointerException during commit for PAUSED connector after the restart of Kafka Connect application
2018-01-02 10:54:21 +01:00
Jiri Pechanec
24e305ad57
DBZ-525 Timestamp supports optional microseconds part
2018-01-02 08:52:27 +01:00
Jenkins user
6bb34b42f9
[maven-release-plugin] prepare for next development iteration
2017-12-20 07:15:12 +00:00
Jenkins user
16dcd4c980
[maven-release-plugin] prepare release v0.7.1
2017-12-20 07:15:12 +00:00
Jiri Pechanec
42d103294e
DBZ-518 Remove a leftover event
2017-12-20 06:06:41 +01:00
Jiri Pechanec
ecf6a86594
DBZ-517 Manually override RDS environment
2017-12-20 06:05:59 +01:00
Jiri Pechanec
ffce17484e
DBZ-512 Need to use OID with the old Postgres connector version
2017-12-20 06:04:40 +01:00
Jenkins user
5e09932cb9
[maven-release-plugin] prepare for next development iteration
2017-12-15 05:10:23 +00:00
Jenkins user
6c1d61e03b
[maven-release-plugin] prepare release v0.7.0
2017-12-15 05:10:23 +00:00
Gunnar Morling
4e8cedd094
DBZ-379 Postgres connector minimizes use of JDBC metadata
2017-12-13 12:20:37 +01:00
Jiri Pechanec
d4daf639f7
DBZ-498 Unknown types can be handled as binary data
2017-12-11 13:44:25 +01:00
Jiri Pechanec
104f1bc466
DBZ-501 Avoid race condition in stop of thread executor and task submission
2017-12-06 13:21:15 +01:00
rkerner
c7ac481c43
[DBZ-342] fix broken MySQL data type "TIME" handling
2017-11-29 20:34:12 +01:00
Gunnar Morling
4309164f27
DBZ-453 Improving description of configuration option; also
...
* some JavaDoc adjustments
* removing some dead code
2017-11-29 15:14:19 +01:00
Gunnar Morling
5dae38bfab
DBZ-453 Keeping existing behavior of omitting fields of unsupported column types from change messages;
...
Also removing confusing log message as those fields are not propagated as binary data actually.
2017-11-29 15:14:19 +01:00
Robert Coup
65a8113cb7
Deal with non-builtin PG types.
...
Fix DBZ-454 issue with mapping to the wrong OID.
Return byte[] as the standard approach
2017-11-29 15:14:19 +01:00
Robert Coup
71d6a1bcb6
DBZ-453 add support for newer wal2json with verbose type specifiers.
2017-11-29 15:14:19 +01:00
Jiri Pechanec
b60ff5e863
DBZ-478 Fix null array handling for wal2json
2017-11-23 14:53:46 +01:00
Gunnar Morling
833e81a3a4
DBZ-478 Avoiding NPE in case of unsupported array column types for Postgres;
...
Instead, just the warning about the unsupported type will be logged.
2017-11-22 09:43:10 +01:00
Gunnar Morling
b1cb58f7cc
DBZ-478 Correctly handling null values for array-typed columns
2017-11-22 09:43:10 +01:00
Jiri Pechanec
3fae495bc7
DBZ-259 Return correct value for null Duration
2017-11-14 13:04:25 +01:00
Gunnar Morling
5fbe742be8
DBZ-285 Specifying scope of dependencies in the individual POMs for the sake of comprehensibility
2017-11-10 16:48:32 +01:00
Scofield Xu
0697ad7cd9
DBZ-424 Can't support PostgreSQL 10
2017-11-03 15:05:23 +01:00
Gunnar Morling
38bda6625a
DBZ-416 Removing unneccessary configuration of build helper plug-in;
...
It would have been needed for the Postres module only anyways. But it seems the generator plug-in is adding that source path automatically to the compilation already, so it's not needed at all.
2017-10-26 15:23:02 +02:00
Jiri Pechanec
faf3e92f6c
DBZ-404 Postgres container is detected as started after initialization
2017-10-20 08:45:13 +02:00
Ben Williams
a3b4fedd5f
DBZ-363 Add support for BIGINT UNSIGNED handling for MySQL
2017-10-18 10:20:03 +02:00
Jiri Pechanec
cf856f1a06
DBZ-258 Added test for date/time parser
2017-10-18 09:21:22 +02:00
Gunnar Morling
9e563c8333
DBZ-258 Avoiding some data copying by directly accessing underlying message structure from column implementations
2017-10-18 09:21:22 +02:00
Gunnar Morling
73189892b3
DBZ-258 Misc. improvements while reviewing the change;
...
* Removing superfluous config option
* Making loggers static
* JavaDoc fixes
* Extracting hexStringToByteArray() to helper and adding test
* Removing superfluous super() invocations
2017-10-18 09:21:22 +02:00
Jiri Pechanec
a78ef1a1d3
DBZ-258 Fix test when messages are coming in batch
2017-10-18 09:21:22 +02:00
Jiri Pechanec
fa5090a71e
DBZ-258 timestamptz can contain optional usec part
2017-10-18 09:21:22 +02:00
Jiri Pechanec
e47b4cb81c
DBZ-258 Changes after first review
2017-10-18 09:21:22 +02:00
Jiri Pechanec
0bc8129961
DBZ-258 Support for wal2json plugin
2017-10-18 09:21:22 +02:00
Jiri Pechanec
271f87afda
DBZ-380 Task is failed on exception in RecordsStreamProducer
2017-09-29 10:54:27 +02:00
Jenkins user
75937711fa
[maven-release-plugin] prepare for next development iteration
2017-09-21 04:42:02 +00:00
Jenkins user
a89b9332e4
[maven-release-plugin] prepare release v0.6.0
2017-09-21 04:42:02 +00:00
Gunnar Morling
79fefa3cce
DBZ-356 Making sure connection closing exception isn't swallowed
2017-09-19 12:08:56 +02:00
Gunnar Morling
abaf233066
DBZ-356 Closing typeResolverConnection upon RecordsStreamProducer#close()
2017-09-19 12:08:56 +02:00
Jiri Pechanec
70672525ad
DBZ-356 Close leaked JDBC connection
2017-09-19 12:08:56 +02:00
Jiri Pechanec
604c2cba7f
DBZ-279 Consume SQL warnings
2017-09-19 08:53:06 +02:00
Jiri Pechanec
ba3d7d762b
DBZ-318 Support for a Decimale with variable scale schema
2017-09-19 08:39:07 +02:00
Jiri Pechanec
b53b4f67d2
DBZ-319 Views are now ignored in snapshots
2017-09-15 11:37:38 +02:00
Jenkins user
214696ef0c
[maven-release-plugin] prepare for next development iteration
2017-08-17 11:51:05 +00:00
Jenkins user
c867e6fea6
[maven-release-plugin] prepare release v0.5.2
2017-08-17 11:51:05 +00:00
Steven Siahetiong
6c34aea9d1
DBZ-337 Added ability to treat Postgres NUMERIC as double
2017-08-17 09:55:50 +02:00
Gunnar Morling
9d2993f005
DBZ-298 Moving two methods only used in tests to test itself
2017-08-17 09:08:17 +02:00
Gunnar Morling
1665026c8a
DBZ-298 Adding support for quoted identifiers for schemas, tables and
...
columns; expanding test
2017-08-17 09:08:17 +02:00
Emrul Islam
b9be392cd9
DBZ-298 Adding test case for quoted table and column names in Postgres
2017-08-17 09:03:16 +02:00
Gunnar Morling
564d90dc09
DBZ-298 Formatting
2017-08-17 09:03:16 +02:00
Emrul Islam
bb79b2f60f
DBZ-297 Initial support for some Postgres array types (one-dimensional only)
2017-07-27 10:04:13 +02:00
Gunnar Morling
bddf715324
DBZ-287 Avoiding double conversion; changing test to use a larger scale
2017-07-25 14:34:21 +02:00
Matteo Capitanio
7fb8829cb1
DBZ-287 Corrected scale representation for Postgres NUMERIC/DECIMAL
2017-07-25 14:14:40 +02:00
Jiri Pechanec
1f2061c2fc
DBZ-286 Connection keepAlive support
2017-07-14 15:17:10 +02:00
Jiri Pechanec
c2d6d16803
DBZ-301 Providing a hint for test extensions
2017-07-13 12:46:25 +02:00
Gunnar Morling
a8d1817c22
[maven-release-plugin] prepare for next development iteration
2017-06-09 16:14:31 +00:00
Gunnar Morling
3f512aace7
[maven-release-plugin] prepare release v0.5.1
2017-06-09 16:14:31 +00:00
Duncan Sands
19a4b0a012
DBZ-282 Correct the text describing the default values to match the actual defaults
2017-06-09 10:50:45 +02:00
Jiri Pechanec
168914c9fb
DBZ-281 Fail task when connection to db is lost
2017-06-09 10:27:19 +02:00
Gunnar Morling
aa4f102ff0
DBZ-280 Expanding test
2017-06-08 22:17:29 +02:00
arosenber
616ba945bf
DBZ-280 - Explicitly handle the tstzrange type to directly push the postgres string representation of the column type
2017-06-07 10:08:02 -04:00
Gunnar Morling
a88527df7d
DBZ-250 Making status update interval configurable;
...
Using the default of PG (10 seconds) helps to reduce CPU utilization, but it causes our tests to run much longer,
as the connector will only stop after the next status update.
So we are using the default in general now, only for the tests the default is overridden with a much smaller value.
2017-06-01 14:52:33 +02:00
Duncan Sands
d7539e03d2
DBZ-250 Fix confusion between int with special value 0 and Integer with special value
...
null, which resulted in high CPU usage due to ceaselessly polling postgres
(createReplicationConnection uses zero with the intent of disabling polling).
2017-06-01 14:52:28 +02:00
Gunnar Morling
f17ced3b4a
DBZ-262 Letting configuration enum types implement EnumeratedValue
2017-05-23 21:26:22 +02:00
Randall Hauch
787959c4d0
DBZ-232 Removed the database and table recommenders
...
It’s not clear how valuable these recommenders actually are. First, it’s not clear about the expected semantics: can the user use values that don’t appear in the recommended values? Second, the recommenders that return large numbers of values can be slow and can result in very large REST API responses.
Debezium was using recommenders to return the database and table/collection names, but these lists can be very large for large databases. Rather than cap the number of recommended values and have the recommender return a subset of all potential values, we will instead remove the recommenders altogether.
2017-05-19 09:24:07 +02:00
Randall Hauch
e365fc2af9
Merge pull request #225 from jpechane/ssl-factory-only
...
[DBZ-244] Allow unvalidated SSL connections
2017-05-10 16:19:49 -05:00
Jiri Pechanec
58c048950b
Fixed import
2017-05-09 12:01:26 +02:00
Jiri Pechanec
ec915c559c
[DBZ-246] Correctly handle enumeratedvale type
2017-05-09 11:24:28 +02:00
Jiri Pechanec
e8fc6d1a8c
Allow unvalidated SSL connections
2017-05-09 08:02:56 +02:00
Gunnar Morling
80903b0622
DBZ-237 Using regular Postgres default port in tests
2017-05-05 22:25:17 +02:00
Gunnar Morling
64958d11db
DBZ-238 Moving EnumeratedValue to its own file
2017-05-05 21:55:46 +02:00
Brendan Maguire
928c9cb5f0
DBZ-238: Fix Postgres database.sslmode = "require" bug
...
* Also added EnumeratedValue interface for Configuration so getValue is used instead of toString when getting the config value
2017-05-05 21:55:45 +02:00
Randall Hauch
709cd8f3fe
[maven-release-plugin] prepare for next development iteration
2017-03-27 11:28:12 -05:00
Randall Hauch
2bc3d45954
[maven-release-plugin] prepare release v0.5.0
2017-03-27 11:28:11 -05:00
Randall Hauch
81f62b6961
DBZ-205 Corrected MySQL connector to handle 2-digit years
...
MySQL has special handling of 2-digit years that it deems are ambiguous, such as the year value `17` that is actually treated as `2017`. Apparently the 2-digit values are stored in MySQL and the interpretation is performed when the data is extracted, so therefore the connector needs to also perform this adjustment of the year values. This commit uses the JDK’s `TemporalAdjuster` interface and passes this down to the requisite temporal-related datatype handling code. The MySQL connector then provides its own `TemporalAdjuster` implementation that adjusts the year values via the excellend JDK `Temporal` methods.
A row in one of the MySQL test databases was changed to use a 2-digit year of `16` while the test method still checks that the year is still 2016`, verifying that the year value is properly adjusted.
2017-03-27 10:58:21 -05:00
Randall Hauch
430d756062
[maven-release-plugin] prepare for next development iteration
2017-03-17 15:41:58 -05:00
Randall Hauch
536cbf6300
[maven-release-plugin] prepare release v0.4.1
2017-03-17 15:41:57 -05:00
Randall Hauch
8c60c29883
[maven-release-plugin] prepare for next development iteration
2017-02-07 14:22:12 -06:00
Randall Hauch
20134286e9
[maven-release-plugin] prepare release v0.4.0
2017-02-07 14:22:11 -06:00
Randall Hauch
f0db8d1b1f
DBZ-179 Corrected JavaDoc in PostgreSQL connector
...
Corrected the JavaDoc and removed trailing spaces in the PostgreSQL connector code.
2017-01-20 11:51:17 -06:00
Randall Hauch
e11f242b00
DBZ-179 Moved generated source for Protobuf
...
The project requires that all JavaDoc for public methods exist and are valid (e.g., have all @param, @return and @throws to match the signature). However, the generated Java source for Protobuf contain numerous JavaDoc errors relative to these settings. This causes lots of errors inside Eclipse (and probably other IDEs), but ignoring/disabling the JavaDoc errors leads to improper JavaDoc (fixed in next commit). By moving the generated Protobuf source code to a separate directory (e.g., `generated-sources`), the IDEs will automatically discover the directory and the user can ignore any compiler and JavaDoc errors/warnings for those files while keeping the more strict JavaDoc checking enabled for the rest of the code.
2017-01-20 11:50:08 -06:00
Horia Chiorean
c024d0789b
DBZ-177 Changes the way the PostgreSQL connector loads the JDBC driver, to use the connector's classloader
2017-01-18 17:59:20 +02:00
Randall Hauch
eeb4eafacf
DBZ-3 Changed the PostgreSQL connector’s use of Docker containers
...
The PostgreSQL connetor was not able to build locally, since the Maven build would wait forever trying to talk to the TCP port for PostgreSQL before starting the integration tests. Even when I corrected the `wait` specification to use the localhost (rather than the direct container address), the build successfully connected to Postgres when it started the first time but before it shutdown to adjust the configuration, and thus the tests failed as the server was shutdown. The build now looks for a specific log message which is unique and output by the container after the second startup, and this seems to work great (at least locally).
2017-01-13 16:44:30 -06:00
Horia Chiorean
c6b594e100
DBZ-3 Adds the assembly profile to the connector's POM
2017-01-04 11:41:52 +02:00
Horia Chiorean
ae85656851
DBZ-3 Fixes topic naming to include the name of the server
2016-12-30 09:42:48 +02:00
Horia Chiorean
d681c11eee
DBZ-3 Adds README and NOTES documentation files for the postgres connector module
2016-12-27 14:44:34 +02:00
Horia Chiorean
6159524618
DBZ-3 Removes the sources for the Postgres JDBC driver and replaces them with a SNAPSHOT release which contains the streaming support
2016-12-27 14:44:33 +02:00
Horia Chiorean
a300d3e1cf
DBZ-3 Changes the configuration of the Docker Maven plugin to only use alias naming when necessary and moves the PG connector ahead of the Mongo connector in the build
2016-12-27 14:44:33 +02:00
Horia Chiorean
737614a555
DBZ-3 Implements a connector for streaming changes from a Postgres database
...
The version of the DB server required for this to work is at least 9.4. To be able to stream logical changes, the code relies on enhancements to the JDBC driver which are not yet public. Therefore, the current codebase includes the sources for the JDBC driver.
The commit also updates the general DBZ build system for:
* custom checkstyle package exclusions - required by the Postgres driver the protobuf code for now
* adds support for debugging Surefire and Failsafe
2016-12-27 14:44:32 +02:00
Horia Chiorean
23e3f59fa1
DBZ-3 Implements a connector for streaming changes from a Postgres database
...
The version of the DB server required for this to work is at least 9.4
The commit also updates the general DBZ build system for:
* custom checkstyle package exclusions - required by the Postgres driver the protobuf code for now
* adds support for debugging Surefire and Failsafe
2016-12-27 14:44:32 +02:00
Horia Chiorean
f13208a229
DBZ-3 Adds a local copy of the PG driver which contains the logical replication support
2016-12-27 14:44:31 +02:00
Horia Chiorean
8e14f150db
DBZ-3 Adds the structure for a Postgres connector which uses a Debezium Postgres docker image that has the decoderbufs plugin enabled to read WAL changes
2016-12-27 14:44:29 +02:00
Randall Hauch
fa4ae33ba2
Removed unused modules
2016-03-17 16:13:50 -05:00
Randall Hauch
046fc83850
DBZ-23 Simplified PosgreSQL Connector's use of Docker plugin
2016-02-25 10:24:52 -06:00
Randall Hauch
1d46e59048
DBZ-17 Minor changes to the POMs
2016-02-18 13:58:29 -06:00
Christian Posta
c730685a01
add option to run without integration tests
2016-02-15 16:26:32 -07:00