Commit Graph

7214 Commits

Author SHA1 Message Date
Martin Medek
3c7f26b611 DBZ-5165 fixing comments from code review 2022-08-04 19:04:56 +02:00
Martin Medek
301ecd4cbb DBZ-5165 checkstyle 2022-08-04 19:04:56 +02:00
Martin Medek
a0b414e2a8 DBZ-5165 cleanup of forsaken changes 2022-08-04 19:04:56 +02:00
Martin Medek
8eeab0802c DBZ-5165 refactoring the jenkins jobs, removing need for secret name specification 2022-08-04 19:04:56 +02:00
Martin Medek
7216569677 DBZ-5165 Fixed the copy secret action, refactor 2022-08-04 19:04:56 +02:00
Martin Medek
26d793bae4 DBZ-5165 Removed the java changes enabling using secret name instead of path 2022-08-04 19:04:56 +02:00
Martin Medek
b91ff7f7ef DBZ-5165 Changed the secret logic to avoid extensive changes in java 2022-08-04 19:04:56 +02:00
Martin Medek
a7125a840c DBZ-5165 Added downstream dockerfile, fixing the downstream prepare logic 2022-08-04 19:04:56 +02:00
Martin Medek
1f8f06b4c0 DBZ-5165 Fixed the secret copy action, removed the temporary logging 2022-08-04 19:04:56 +02:00
Martin Medek
f25536c99b DBZ-5165 Added test logging for the secret load, commented the waiting section in OCP system pipeline job, added secret name param to pom.xml 2022-08-04 19:04:56 +02:00
Martin Medek
ae3afef012 DBZ-5165 Refactoring the image preparation jobs, put back the git pull and recompile step in docker for quicker java development 2022-08-04 19:04:56 +02:00
Martin Medek
6dd4109649 DBZ-5165 added option to use existing pull secret from namespace instead of the one baked inside image. 2022-08-04 19:04:56 +02:00
Martin Medek
1bde16db8c DBZ-5165 reworked the public/local service distinction, refactor of OcpClient 2022-08-04 19:04:56 +02:00
Martin Medek
d6d4901e2b DBZ-5165 added waiting for the job finish and printing logs. Fixed apicurio descriptor name for product build 2022-08-04 19:04:56 +02:00
Martin Medek
1739ee9cf5 DBZ-5165 fixing the product build in testsuite 2022-08-04 19:04:56 +02:00
Martin Medek
240700a725 DBZ-5165 trying job deployment without specifying secret name 2022-08-04 19:04:56 +02:00
Martin Medek
87abe699f4 DBZ-5165 bugfixing, added missing newlines to EOF 2022-08-04 19:04:56 +02:00
Martin Medek
3f0c6ea577 DBZ-5165 added temporary fix for secret name/secret filename non-alignment 2022-08-04 19:04:56 +02:00
Martin Medek
35bc00f642 DBZ-5165 created job-dsl for ocp system pipeline, renamed deployment parameters 2022-08-04 19:04:56 +02:00
Martin Medek
b7cbbb3bb2 DBZ-5165 refactoring the testsuite scripts 2022-08-04 19:04:56 +02:00
Martin Medek
7a265104bc DBZ-5165 added apicurio step and a jenkins job to build the testsuite image 2022-08-04 19:04:56 +02:00
Martin Medek
6ee07d6a65 DBZ-5165 added oracle project system property to pom, refactor. Added docker tag to DockerRhelMongoConnectorIT 2022-08-04 19:04:56 +02:00
Martin Medek
116e2cfc5b DBZ-5165 modified all db fixtures to not use loadbalancers when running in ocp 2022-08-04 19:04:56 +02:00
Martin Medek
57eeb46c20 DBZ-5165 removed usage of postgresql service inside ocp, changed deployment.yml to run postgresql tests, codestyle 2022-08-04 19:04:56 +02:00
Martin Medek
2bad919b9d DBZ-5165 added Postgresql loadBalancer (not)creation logic 2022-08-04 19:04:56 +02:00
Martin Medek
543582e054 DBZ-5165 fixed deployment.yml formatting 2022-08-04 19:04:56 +02:00
Martin Medek
89812a614f DBZ-5165 moved the incremental build into testsuite script, fixed connect image version in deployment 2022-08-04 19:04:56 +02:00
Martin Medek
385d49fc9c DBZ-5165 changed git pull preparations in Dockerfile 2022-08-04 19:04:56 +02:00
Martin Medek
3d313a621f DBZ-5165 changed logging for isRunningFromOcp conditions 2022-08-04 19:04:56 +02:00
Martin Medek
5614712628 DBZ-5165 Refactored mysql test to not create mysql loadBalancer when running inside openshift, refactored the docker testsuite for running inside openshift. Moved project checkout and compilation to Dockerfile, so that it's not always done from scratch 2022-08-04 19:04:56 +02:00
harveyyue
2481172309 DBZ-5471 Missing the regex properties validation before start connector of DefaultRegexTopicNamingStrategy 2022-08-04 10:49:29 -04:00
jcechace
ea7219a2c8 DBZ-5465 Ability to detect AMQ Streams operator installed via OLM 2022-08-03 07:44:18 +02:00
Chris Cranford
90a4c0059a
DBZ-5434 Fix test failure 2022-08-01 14:36:09 -04:00
harveyyue
7c93c597f7 DBZ-5434 Translation from mongodb document to kafka connect schema fails when nested arrays contain no elements 2022-08-01 10:07:28 -04:00
Chris Cranford
0ee30e2115
DBZ-4727 Adjust chunk size for pause test 2022-08-01 10:04:42 -04:00
Jiri Novotny
a330b7da66 DBZ-5466 Change default base image of node snapshot preparation job 2022-08-01 15:50:03 +02:00
Chris Cranford
41a9324044 DBZ-5441 Gracefully skip Oracle non-relational tables 2022-07-29 12:56:27 -04:00
Vojtech Juranek
d39e98fc9f DBZ-4727 Document pause and resume incremental snapshot 2022-07-29 12:41:58 -04:00
Vojtech Juranek
2c505c08f5 DBZ-4727 Add Kafka based pause/resume signals
Add pause and resume snapshot signals. See previous commit for more
details. These signal do the same, but are used by read-only MySQL
where sending signals is done by sending Kafka messages.
2022-07-29 12:41:58 -04:00
Vojtech Juranek
3968b6b75f DBZ-4727 Intorduce KafkaSignal interface
In the follow-up commit signal for pause and resume snapshot will be
added. Intorduce `KafkaSignal` interface which would server as general
kafka-based signal.
2022-07-29 12:41:58 -04:00
Vojtech Juranek
c7d8353d7b DBZ-4727 Add snapshot paused JXM metrics
Add two JXM objects:
* `snapshotPaused` - detemines if the incremental snapshot is paused
  or not
* `snapshotPausedDurationInSeconds` - overall time when the incremental
  snapshot was paused. The time adds up - if the snapshot was paused
  e.g. two times, the `snapshotPausedDurationInSeconds` is the sum of
  these two paused times.
2022-07-29 12:41:58 -04:00
Vojtech Juranek
ed504173e1 DBZ-4727 Implement incremental snaphost pause/resume
Intorduce two new signals:
* `pause-snapshot` - pauses running incremental snapshot
* `resume-snapshot` - resumes paused incremental snapshot

If the incremental snapshot is running and pause signal is sent,
currently processed chunk is finished and then no other chunk is read.
Snapshot state is kept in running state. Once snapshot is resume,
chunk is reverted to position which was last sent to the broker and
process of reading chunks and emitting `read` events continues until
all tables are read and sent.

This patch handles only table-based signals. MySQL Kafka-based signals
will be addressed later.
2022-07-29 12:41:58 -04:00
Vojtech Juranek
65bfeeccea DBZ-4727 Make wait for snapshot event generic 2022-07-29 12:41:58 -04:00
Bob Roldan
13c9e517ef DBZ-5458 Remove conditionals surrounding Oracle ad hoc/incremental refs 2022-07-29 08:53:01 -04:00
Bob Roldan
9f0d0000fc DBZ-5422 Update based on review comments 2022-07-28 17:38:41 -04:00
Bob Roldan
2d9cbbc6d8 DBZ-5422 Further edits; add links to connector docs 2022-07-28 17:38:41 -04:00
Bob Roldan
94d9363ff1 DBZ-5422 Language update 2022-07-28 17:38:41 -04:00
Bob Roldan
6de5321c1c DBZ-5422 Update link format and multiple edits 2022-07-28 17:38:41 -04:00
Vojtech Juranek
e70ceeeacd DBZ-5307 Fix adding events from WireMock
In existing test we added all events in the loop which can lead into
adding one event multiple times and therefore failing the tests.
When test run alone, this probably never happen or it was very rare.
Now, when test runs with Debezium server already running, it fails more
often. Fix it by removing already added events from WireMock.
2022-07-28 14:19:13 -04:00
Vojtech Juranek
37d6d3685b DBZ-5307 Update HTTP consumer documentation 2022-07-28 14:19:13 -04:00