Merge pull request #58 from rhauch/dbz-37
DBZ-37 Changed build to support running integration tests against multiple MySQL configurations
This commit is contained in:
Randall Hauch
commit
cc11f23cd0
@ -20,7 +20,7 @@ after_install:
|
||||
- docker ps -a
|
||||
|
||||
script:
|
||||
- mvn clean install -DskipLongRunningTests=false
|
||||
- mvn clean install -Passembly
|
||||
|
||||
notifications:
|
||||
webhooks:
|
||||
|
||||
@ -22,7 +22,7 @@ A *unit test* is a JUnit test class named `*Test.java` or `Test*.java` that neve
|
||||
|
||||
An *integration test* is a JUnit test class named `*IT.java` or `IT*.java` that uses a MySQL database server running in a custom Docker container based upon the [mysql/mysql-server:5.7](https://hub.docker.com/r/mysql/mysql-server/) Docker image maintained by the MySQL team. The build will automatically start the MySQL container before the integration tests are run and automatically stop and remove it after all of the integration tests complete (regardless of whether they suceed or fail). All databases used in the integration tests are defined and populated using `*.sql` files and `*.sh` scripts in the `src/test/docker/init` directory, which are copied into the Docker image and run in lexicographical order by MySQL upon startup. Multiple test methods within a single integration test class can reuse the same database, but generally each integration test class should use its own dedicated database(s).
|
||||
|
||||
Running `mvn install` will compile all code and run the unit tests. If there are any compile problems or any of the unit tests fail, the build will stop immediately. Otherwise, the command will continue to create the module's artifacts, create the Docker image with MySQL and custom scripts, start the Docker container, run the integration tests, stop the container (even if there are integration test failures), and run checkstyle on the code. If there are still no problems, the build will then install the module's artifacts into the local Maven repository.
|
||||
Running `mvn install` will compile all code and run the unit and integration tests. If there are any compile problems or any of the unit tests fail, the build will stop immediately. Otherwise, the command will continue to create the module's artifacts, create the Docker image with MySQL and custom scripts, start the Docker container, run the integration tests, stop the container (even if there are integration test failures), and run checkstyle on the code. If there are still no problems, the build will then install the module's artifacts into the local Maven repository.
|
||||
|
||||
You should always default to using `mvn install`, especially prior to committing changes to Git. However, there are a few situations where you may want to run a different Maven command.
|
||||
|
||||
@ -56,14 +56,10 @@ For example, you can define these properties by passing these arguments to the J
|
||||
|
||||
-Ddatabase.dbname=<DATABASE_NAME> -Ddatabase.hostname=<DOCKER_HOST> -Ddatabase.port=3306 -Ddatabase.user=mysqluser -Ddatabase.password=mysqlpw
|
||||
|
||||
When you are finished running the integration tests from your IDE, you have to stop and remove the Docker container (conveniently named "database") before you can run the next build:
|
||||
When you are finished running the integration tests from your IDE, you have to stop and remove the Docker container before you can run the next build:
|
||||
|
||||
$ mvn docker:stop
|
||||
|
||||
or using Docker directly:
|
||||
|
||||
$ docker stop database
|
||||
$ docker rm database
|
||||
|
||||
Please note that when running the MySQL database Docker container, the output is written to the Maven build output and includes several lines with `[Warning] Using a password on the command line interface can be insecure.` You can ignore these warnings, since we don't need a secure database server for our transient database testing.
|
||||
|
||||
@ -75,14 +71,23 @@ Sometimes you may want to inspect the state of the database(s) after one or more
|
||||
|
||||
### Stopping the Docker container
|
||||
|
||||
This instructs Maven to run the normal Maven lifecycle through `integration-test`, and to stop before the `post-integration-test` phase when the Docker container is normally shut down and removed. Be aware that you will need to manually stop and remove the container (conveniently named "database") before running the build again:
|
||||
This instructs Maven to run the normal Maven lifecycle through `integration-test`, and to stop before the `post-integration-test` phase when the Docker container is normally shut down and removed. Be aware that you will need to manually stop and remove the container before running the build again:
|
||||
|
||||
$ mvn docker:stop
|
||||
|
||||
or using Docker directly:
|
||||
### Using MySQL with GTIDs
|
||||
|
||||
$ docker stop database
|
||||
$ docker rm database
|
||||
By default the build will run a MySQL server instance that is not configured to use GTIDs. However, we've provided a Maven profile that will instead run all the same integration tests against a MySQL instance that does use GTIDs. Simply use the `gtid-mysql` profile on each of your normal Maven commands. For example, to run a build:
|
||||
|
||||
$ mvn clean install -Pgtid-mysql
|
||||
|
||||
or to manually start the Docker container and keep it running:
|
||||
|
||||
$ mvn docker:start -Pgtid-mysql
|
||||
|
||||
or to stop and remove the Docker container:
|
||||
|
||||
$ mvn docker:stop -Pgtid-mysql
|
||||
|
||||
### Using an alternative MySQL Server
|
||||
|
||||
@ -102,3 +107,8 @@ or to stop and remove the Docker container:
|
||||
|
||||
$ mvn docker:start -Palt-mysql
|
||||
|
||||
### Testing all MySQL configurations
|
||||
|
||||
In Debezium builds, the `assembly` profile is used when issuing a release or in our continuous integration builds. In addition to the normal steps, it also creates several additional artifacts (including the connector plugin's ZIP and TAR archives) and runs the whole
|
||||
integration test suite once for _each_ of the MySQL configurations. If you want to make sure that your changes work on all MySQL configurations, add `-Passembly` to your Maven commands.
|
||||
|
||||
|
||||
@ -79,12 +79,15 @@
|
||||
work on all platforms. We'll set some of these as system properties during integration testing.
|
||||
-->
|
||||
<database.port>3306</database.port>
|
||||
<database.gtid.port>3306</database.gtid.port>
|
||||
<database.alt.port>3306</database.alt.port>
|
||||
<database.user>mysqluser</database.user>
|
||||
<database.password>mysqlpw</database.password>
|
||||
<!--
|
||||
By default, we should use the docker image maintained by the MySQL team. This property is changed with different profiles.
|
||||
However, we run one container with GTIDs and one without.
|
||||
-->
|
||||
<docker.image>debezium/mysql-server-test-databases</docker.image>
|
||||
<docker.image>debezium/mysql-server-test-database</docker.image>
|
||||
<docker.skip>false</docker.skip>
|
||||
</properties>
|
||||
<build>
|
||||
@ -100,7 +103,7 @@
|
||||
<images>
|
||||
<image>
|
||||
<!-- A Docker image using a partial MySQL installation maintained by MySQL team. -->
|
||||
<name>debezium/mysql-server-test-databases</name>
|
||||
<name>debezium/mysql-server-test-database</name>
|
||||
<alias>database</alias>
|
||||
<run>
|
||||
<namingStrategy>alias</namingStrategy>
|
||||
@ -147,8 +150,8 @@
|
||||
</image>
|
||||
<image>
|
||||
<!-- A Docker image using a partial MySQL installation maintained by MySQL team. -->
|
||||
<name>debezium/mysql-server-gtids-test-databases</name>
|
||||
<alias>database</alias>
|
||||
<name>debezium/mysql-server-gtids-test-database</name>
|
||||
<alias>database-gtids</alias>
|
||||
<run>
|
||||
<namingStrategy>alias</namingStrategy>
|
||||
<env>
|
||||
@ -158,12 +161,12 @@
|
||||
<MYSQL_PASSWORD>${database.password}</MYSQL_PASSWORD>
|
||||
</env>
|
||||
<ports>
|
||||
<port>${database.port}:3306</port>
|
||||
<port>${database.gtid.port}:3306</port>
|
||||
</ports>
|
||||
<log>
|
||||
<prefix>mysql</prefix>
|
||||
<prefix>mysql-gtids</prefix>
|
||||
<enabled>true</enabled>
|
||||
<color>yellow</color>
|
||||
<color>cyan</color>
|
||||
</log>
|
||||
<wait>
|
||||
<log>MySQL init process done. Ready for start up.</log>
|
||||
@ -194,8 +197,8 @@
|
||||
</image>
|
||||
<image>
|
||||
<!-- A Docker image using a complete MySQL installation maintained by Docker team. -->
|
||||
<name>debezium/mysql-test--databases</name>
|
||||
<alias>database</alias>
|
||||
<name>debezium/mysql-test-alt-database</name>
|
||||
<alias>database-alt</alias>
|
||||
<run>
|
||||
<namingStrategy>alias</namingStrategy>
|
||||
<env>
|
||||
@ -205,15 +208,15 @@
|
||||
<MYSQL_PASSWORD>${database.password}</MYSQL_PASSWORD>
|
||||
</env>
|
||||
<ports>
|
||||
<port>${database.port}:3306</port>
|
||||
<port>${database.alt.port}:3306</port>
|
||||
</ports>
|
||||
<log>
|
||||
<prefix>alt-mysql</prefix>
|
||||
<enabled>true</enabled>
|
||||
<color>yellow</color>
|
||||
<color>magenta</color>
|
||||
</log>
|
||||
<wait>
|
||||
<log>Version: \'5\.7\.11-log\' socket: \'/var/run/mysqld/mysqld\.sock\' port: 3306 MySQL Community Server \(GPL\)</log>
|
||||
<log>socket: \'/var/run/mysqld/mysqld\.sock\' port: 3306 MySQL Community Server \(GPL\)</log>
|
||||
<time>30000</time>
|
||||
</wait>
|
||||
</run>
|
||||
@ -325,11 +328,27 @@
|
||||
Define several useful profiles
|
||||
-->
|
||||
<profiles>
|
||||
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
This assembly profile is used during official builds. In addition to
|
||||
compiling, and running the unit and integration tests like the non-assembly
|
||||
profiles, this provfile creates additional (like the connector plugin archives),
|
||||
starts up all three Docker containers (normal MySQL, MySQL+GTIDs, and alt-MySQL)
|
||||
and runs the integration tests against each of them.
|
||||
|
||||
To use, specify "-Passembly" on the Maven command line.
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
|
||||
<profile>
|
||||
<id>assembly</id>
|
||||
<activation>
|
||||
<activeByDefault>false</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<!-- Run multiple images at the same time, but use different ports for all MySQL servers -->
|
||||
<docker.image>debezium/mysql-server-test-database,debezium/mysql-server-gtids-test-database,debezium/mysql-test-alt-database</docker.image>
|
||||
<database.port>3306</database.port>
|
||||
<database.gtid.port>4306</database.gtid.port>
|
||||
<database.alt.port>5306</database.alt.port>
|
||||
</properties>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
@ -359,6 +378,62 @@
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<!-- Override the failsafe plugin to run the integration tests twice -->
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-failsafe-plugin</artifactId>
|
||||
<executions>
|
||||
<!-- First run the integration tests with the non-GTID server -->
|
||||
<execution>
|
||||
<id>integration-test-mysql</id>
|
||||
<goals>
|
||||
<goal>integration-test</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
<!-- Then run the integration tests with the GTID server on different port -->
|
||||
<execution>
|
||||
<id>integration-test-mysql-gtids</id>
|
||||
<goals>
|
||||
<goal>integration-test</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<systemPropertyVariables>
|
||||
<database.port>${database.gtid.port}</database.port>
|
||||
</systemPropertyVariables>
|
||||
</configuration>
|
||||
</execution>
|
||||
<!-- Then run the integration tests with the Docker's MySQL server (without GTIDs) on different port -->
|
||||
<execution>
|
||||
<id>integration-test-alt-mysql</id>
|
||||
<goals>
|
||||
<goal>integration-test</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<systemPropertyVariables>
|
||||
<database.port>${database.alt.port}</database.port>
|
||||
</systemPropertyVariables>
|
||||
</configuration>
|
||||
</execution>
|
||||
<execution>
|
||||
<id>verify</id>
|
||||
<goals>
|
||||
<goal>verify</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
<configuration>
|
||||
<skipTests>${skipITs}</skipTests>
|
||||
<enableAssertions>true</enableAssertions>
|
||||
<systemPropertyVariables>
|
||||
<!-- Make these available to the tests via system properties -->
|
||||
<database.hostname>${docker.host.address}</database.hostname>
|
||||
<database.port>${database.port}</database.port>
|
||||
<database.user>${database.user}</database.user>
|
||||
<database.password>${database.password}</database.password>
|
||||
<skipLongRunningTests>false</skipLongRunningTests>
|
||||
</systemPropertyVariables>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</profile>
|
||||
@ -392,8 +467,7 @@
|
||||
</property>
|
||||
</activation>
|
||||
<properties>
|
||||
<docker.image>debezium/mysql-test--databases</docker.image>
|
||||
<docker.skip>false</docker.skip>
|
||||
<docker.image>debezium/mysql-test-alt-database</docker.image>
|
||||
</properties>
|
||||
</profile>
|
||||
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@ -409,8 +483,7 @@
|
||||
</property>
|
||||
</activation>
|
||||
<properties>
|
||||
<docker.image>debezium/mysql-server-gtids-test-databases</docker.image>
|
||||
<docker.skip>false</docker.skip>
|
||||
<docker.image>debezium/mysql-server-gtids-test-database</docker.image>
|
||||
</properties>
|
||||
</profile>
|
||||
</profiles>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user