DBZ-6874 Drop MySQL 5.7, test MySQL 8.0 & adjust documentation

.github/workflows/debezium-workflow.yml

@@ -291,11 +291,10 @@ jobs:
           -Dmaven.wagon.http.pool=false
           -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
 
-  ## Add this as until 5.7 will be EOL in October 2023
+  build_mysql_8_0:
-  build_mysql_5_7:
     needs: [ check_style, file_changes ]
     if: ${{ needs.file_changes.outputs.common-changed == 'true' || needs.file_changes.outputs.mysql-changed == 'true' || needs.file_changes.outputs.mysql-ddl-parser-changed == 'true' || needs.file_changes.outputs.schema-generator-changed == 'true' }}
-    name: "MySQL 5.7"
+    name: "MySQL 8.0"
     runs-on: ubuntu-latest
     steps:
       - name: Checkout Action
@@ -318,8 +317,7 @@ jobs:
       - name: Build Debezium Connector MySQL
         run: >
           ./mvnw clean install -B -pl debezium-connector-mysql -am -Passembly
-          -Dmysql.server.image.source=mysql/mysql-server
+          -Dversion.mysql.server=8.0
-          -Dversion.mysql.server=5.7
           -Dcheckstyle.skip=true
           -Dformat.skip=true
           -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn

debezium-connector-mysql/NOTES.md

@@ -4,7 +4,7 @@ The [mysql](https://hub.docker.com/r/_/mysql/) images are maintained by Docker,
 
 ## Using MySQL Server
 
-As mentioned in the [README.md]() file, our Maven build can be used to start a container using either one of these images. The `container-registry.oracle.com/mysql/community-server:8.1` image is used:
+As mentioned in the [README.md]() file, our Maven build can be used to start a container using either one of these images. The `container-registry.oracle.com/mysql/community-server:8.2` image is used:
 
     $ mvn docker:start
 
@@ -31,9 +31,9 @@ or
 
 Although using the Maven command is far simpler, the Maven command for the `alt-server` profile really just runs (via the Jolokia Maven plugin) a Docker command to start the container, so it's equivalent to:
 
-    $ docker run -it --name database -p 3306:3306 -v $(pwd)/src/test/docker/alt-server:/etc/mysql/conf.d -v $(pwd)/src/test/docker/init:/docker-entrypoint-initdb.d -e MYSQL_DATABASE=mysql -e MYSQL_ROOT_PASSWORD=debezium-rocks -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw mysql:5.7
+    $ docker run -it --name database -p 3306:3306 -v $(pwd)/src/test/docker/alt-server:/etc/mysql/conf.d -v $(pwd)/src/test/docker/init:/docker-entrypoint-initdb.d -e MYSQL_DATABASE=mysql -e MYSQL_ROOT_PASSWORD=debezium-rocks -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw mysql:8.2
 
-This will use the `mysql:5.7` image to start a new container named `database` where the MySQL instance uses the combined startup settings from `/etc/mysql/my.cnf` (defined in the Docker image) and the same local configuration file we used in the integration test MySQL container, `src/test/docker/mysql.cnf` (mounted into the container at `/etc/mysql/conf.d/mysql.cnf`). The settings from the latter file take precedence.
+This will use the `mysql:8.2` image to start a new container named `database` where the MySQL instance uses the combined startup settings from `/etc/mysql/my.cnf` (defined in the Docker image) and the same local configuration file we used in the integration test MySQL container, `src/test/docker/mysql.cnf` (mounted into the container at `/etc/mysql/conf.d/mysql.cnf`). The settings from the latter file take precedence.
 
 The second volume mount, namely `-v src/test/docker/init:/docker-entrypoint-initdb.d`, makes available all of our existing scripts inside the `src/test/docker/init` directory so that they are run upon server initialization.
 
@@ -43,4 +43,4 @@ The command also defines the same `mysql` database and uses the same username an
 
 The following command can be used to manually start up a Docker container to run the MySQL command line client:
 
-    $ docker run -it --link database:mysql --rm mysql:5.7 sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"'
+    $ docker run -it --link database:mysql --rm mysql:8.2 sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"'

debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/MySqlHistoryRecordComparator.java

@@ -75,7 +75,7 @@ protected boolean isPositionAtOrBefore(Document recorded, Document desired) {
             // recorded position is older since GTIDs are often enabled but rarely disabled. And if they are disabled,
             // it is likely that the desired position would not include GTIDs as we would be trying to read the binlog of a
             // server that no longer has GTIDs. And if they are enabled, disabled, and re-enabled, per
-            // https://dev.mysql.com/doc/refman/5.7/en/replication-gtids-failover.html all properly configured slaves that
+            // https://dev.mysql.com/doc/refman/8.2/en/replication-gtids-failover.html all properly configured slaves that
             // use GTIDs should always have the complete set of GTIDs copied from the master, in which case
             // again we know that recorded not having GTIDs is before the desired position ...
             return true;

debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/MySqlSnapshotChangeEventSource.java

@@ -178,9 +178,9 @@ protected void lockTablesForSchemaSnapshot(ChangeEventSourceContext sourceContex
         // from the same MVCC snapshot. Thus each plain (non-locking) SELECT statements within the same transaction are
         // consistent also with respect to each other.
         //
-        // See: https://dev.mysql.com/doc/refman/5.7/en/set-transaction.html
+        // See: https://dev.mysql.com/doc/refman/8.2/en/set-transaction.html
-        // See: https://dev.mysql.com/doc/refman/5.7/en/innodb-transaction-isolation-levels.html
+        // See: https://dev.mysql.com/doc/refman/8.2/en/innodb-transaction-isolation-levels.html
-        // See: https://dev.mysql.com/doc/refman/5.7/en/innodb-consistent-read.html
+        // See: https://dev.mysql.com/doc/refman/8.2/en/innodb-consistent-read.html
         connection.connection().setTransactionIsolation(Connection.TRANSACTION_REPEATABLE_READ);
         connection.executeWithoutCommitting("SET SESSION lock_wait_timeout=" + connectorConfig.snapshotLockTimeout().getSeconds());
         try {
@@ -225,7 +225,7 @@ protected void releaseSchemaSnapshotLocks(RelationalSnapshotContext<MySqlPartiti
                 // using 'FLUSH TABLE <tableName> WITH READ LOCK'. However, if we were to do this, the 'UNLOCK TABLES'
                 // would implicitly commit our active transaction, and this would break our consistent snapshot logic.
                 // Therefore, we cannot unlock the tables here!
-                // https://dev.mysql.com/doc/refman/5.7/en/flush.html
+                // https://dev.mysql.com/doc/refman/8.2/en/flush.html
                 LOGGER.warn("Tables were locked explicitly, but to get a consistent snapshot we cannot release the locks until we've read all tables.");
             }
         }

debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/MySqlSystemVariables.java

@@ -38,7 +38,7 @@ public int priority() {
 
     /**
      * The system variable name for the name of the character set that the server uses by default.
-     * See http://dev.mysql.com/doc/refman/5.7/en/server-options.html#option_mysqld_character-set-server
+     * See http://dev.mysql.com/doc/refman/8.2/en/server-options.html#option_mysqld_character-set-server
      */
     public static final String CHARSET_NAME_SERVER = "character_set_server";
 
@@ -52,7 +52,7 @@ public int priority() {
 
     /**
      * The system variable name to see if the MySQL tables are stored and looked-up in case sensitive way.
-     * See https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_lower_case_table_names
+     * See https://dev.mysql.com/doc/refman/8.2/en/server-system-variables.html#sysvar_lower_case_table_names
      */
     public static final String LOWER_CASE_TABLE_NAMES = "lower_case_table_names";
 

debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/MySqlUnsignedIntegerConverter.java

@@ -17,7 +17,7 @@
 public class MySqlUnsignedIntegerConverter {
     /**
      * Maximum values for Unsigned Integer Types. Needed in order to calculate actual value of an Unsigned Integer Types from binlog value.
-     * Reference to {@see https://dev.mysql.com/doc/refman/5.7/en/integer-types.html}
+     * Reference to {@see https://dev.mysql.com/doc/refman/8.2/en/integer-types.html}
      */
     private static final short TINYINT_MAX_VALUE = 255;
     private static final int SMALLINT_MAX_VALUE = 65535;

debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/MySqlValueConverters.java

@@ -56,7 +56,7 @@
  * <p>
  * This class always uses UTC for the default time zone when converting values without timezone information to values that require
  * timezones. This is because MySQL {@code TIMESTAMP} values are always
- * <a href="https://dev.mysql.com/doc/refman/5.7/en/datetime.html">stored in UTC</a> (unlike {@code DATETIME} values) and
+ * <a href="https://dev.mysql.com/doc/refman/8.2/en/datetime.html">stored in UTC</a> (unlike {@code DATETIME} values) and
  * are replicated in this form. Meanwhile, the MySQL Binlog Client library will {@link AbstractRowsEventDataDeserializer
  * deserialize} these as {@link java.sql.Timestamp} values that have no timezone and, therefore, are presumed to be in UTC.
  * When the column is properly marked with a {@link Types#TIMESTAMP_WITH_TIMEZONE} type, the converters will need to convert
@@ -91,7 +91,7 @@ public interface ParsingErrorHandler {
     private static final Pattern TIMESTAMP_FIELD_PATTERN = Pattern.compile("([0-9]*)-([0-9]*)-([0-9]*) .*");
 
     /**
-     * A utility method that adjusts <a href="https://dev.mysql.com/doc/refman/5.7/en/two-digit-years.html">ambiguous</a> 2-digit
+     * A utility method that adjusts <a href="https://dev.mysql.com/doc/refman/8.2/en/two-digit-years.html">ambiguous</a> 2-digit
      * year values of DATETIME, DATE, and TIMESTAMP types using these MySQL-specific rules:
      * <ul>
      * <li>Year values in the range 00-69 are converted to 2000-2069.</li>

debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/RowDeserializers.java

@@ -285,7 +285,7 @@ protected static Serializable deserializeVarString(int meta, ByteArrayInputStrea
     /**
      * Converts a MySQL {@code DATE} value to a {@link LocalDate}.
      * <p>
-     * This method treats all <a href="http://dev.mysql.com/doc/refman/5.7/en/date-and-time-types.html">zero values</a>
+     * This method treats all <a href="http://dev.mysql.com/doc/refman/8.2/en/date-and-time-types.html">zero values</a>
      * for {@code DATE} columns as NULL, since they cannot be accurately represented as valid {@link LocalDate} objects.
      *
      * @param inputStream the binary stream containing the raw binlog event data for the value
@@ -381,7 +381,7 @@ protected static Serializable deserializeTimeV2(int meta, ByteArrayInputStream i
     /**
      * Converts a MySQL {@code DATETIME} value <em>without fractional seconds</em> to a {@link LocalDateTime}.
      * <p>
-     * This method treats all <a href="http://dev.mysql.com/doc/refman/5.7/en/date-and-time-types.html">zero values</a>
+     * This method treats all <a href="http://dev.mysql.com/doc/refman/8.2/en/date-and-time-types.html">zero values</a>
      * for {@code DATETIME} columns as NULL, since they cannot be accurately represented as valid {@link LocalDateTime} objects.
      *
      * @param inputStream the binary stream containing the raw binlog event data for the value
@@ -414,7 +414,7 @@ protected static Serializable deserializeDatetime(ByteArrayInputStream inputStre
     /**
      * Converts a MySQL {@code DATETIME} value <em>with fractional seconds</em> to a {@link LocalDateTime}.
      * <p>
-     * This method treats all <a href="http://dev.mysql.com/doc/refman/5.7/en/date-and-time-types.html">zero values</a>
+     * This method treats all <a href="http://dev.mysql.com/doc/refman/8.2/en/date-and-time-types.html">zero values</a>
      * for {@code DATETIME} columns as NULL, since they cannot be accurately represented as valid {@link LocalDateTime} objects.
      *
      * @param meta the {@code meta} value containing the fractional second precision, or {@code fsp}

debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/antlr/listener/SetStatementParserListener.java

@@ -91,7 +91,7 @@ else if (variableClauseContext.LOCAL() != null) {
     public void enterSetCharset(MySqlParser.SetCharsetContext ctx) {
         String charsetName = ctx.charsetName() != null ? parser.withoutQuotes(ctx.charsetName()) : parser.currentDatabaseCharset();
         // Sets variables according to documentation at
-        // https://dev.mysql.com/doc/refman/5.7/en/set-character-set.html
+        // https://dev.mysql.com/doc/refman/8.2/en/set-character-set.html
         // Using default scope for these variables, because this type of set statement you cannot specify
         // the scope manually
         parser.systemVariables().setVariable(MySqlSystemVariables.MySqlScope.SESSION, MySqlSystemVariables.CHARSET_NAME_CLIENT, charsetName);
@@ -105,7 +105,7 @@ public void enterSetCharset(MySqlParser.SetCharsetContext ctx) {
     public void enterSetNames(MySqlParser.SetNamesContext ctx) {
         String charsetName = ctx.charsetName() != null ? parser.withoutQuotes(ctx.charsetName()) : parser.currentDatabaseCharset();
         // Sets variables according to documentation at
-        // https://dev.mysql.com/doc/refman/5.7/en/set-names.html
+        // https://dev.mysql.com/doc/refman/8.2/en/set-names.html
         // Using default scope for these variables, because this type of set statement you cannot specify
         // the scope manually
         parser.systemVariables().setVariable(MySqlSystemVariables.MySqlScope.SESSION, MySqlSystemVariables.CHARSET_NAME_CLIENT, charsetName);

debezium-connector-mysql/src/test/docker/server-gtids/my.cnf

@@ -1,5 +1,5 @@
 # For advice on how to change settings please see
-# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
+# http://dev.mysql.com/doc/refman/8.2/en/server-configuration-defaults.html
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 5.5 and cross-version common configurations
@@ -88,6 +88,7 @@ gtid_mode                 = on
 enforce_gtid_consistency  = on
 
 default_authentication_plugin = mysql_native_password
+binlog_expire_logs_seconds    = 259200
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 8.2 specific configurations

debezium-connector-mysql/src/test/docker/server-replica/my.cnf

@@ -1,5 +1,5 @@
 # For advice on how to change settings please see
-# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
+# http://dev.mysql.com/doc/refman/8.2/en/server-configuration-defaults.html
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 5.5 and cross-version common configurations
@@ -73,6 +73,7 @@ gtid_mode                 = on
 enforce_gtid_consistency  = on
 
 default_authentication_plugin = mysql_native_password
+binlog_expire_logs_seconds    = 259200
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 8.2 specific configurations

debezium-connector-mysql/src/test/docker/server-ssl/my.cnf

@@ -1,5 +1,5 @@
 # For advice on how to change settings please see
-# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
+# http://dev.mysql.com/doc/refman/8.2/en/server-configuration-defaults.html
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 5.5 and cross-version common configurations
@@ -64,6 +64,7 @@ expire_logs_days              = 1
 # --------------------------------------------------------------------------------------------
 [mysqld-8.0]
 default_authentication_plugin = mysql_native_password
+binlog_expire_logs_seconds    = 86400
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 8.2 specific configurations

debezium-connector-mysql/src/test/docker/server/my.cnf

@@ -1,5 +1,5 @@
 # For advice on how to change settings please see
-# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
+# http://dev.mysql.com/doc/refman/8.2/en/server-configuration-defaults.html
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 5.5 and cross-version common configurations
@@ -59,6 +59,7 @@ expire_logs_days              = 1
 # --------------------------------------------------------------------------------------------
 [mysqld-8.0]
 default_authentication_plugin = mysql_native_password
+binlog_expire_logs_seconds  = 86400
 
 # --------------------------------------------------------------------------------------------
 # This section specifies 8.2 specific configurations

debezium-connector-mysql/src/test/resources/docker/conf/mysql.cnf

@@ -1,5 +1,5 @@
 # For advice on how to change settings please see
-# https://dev.mysql.com/doc/refman/8.0/en/server-configuration-defaults.html
+# https://dev.mysql.com/doc/refman/8.2/en/server-configuration-defaults.html
 
 [mysqld]
 #
@@ -46,3 +46,4 @@ binlog_format               = row
 binlog_row_metadata = FULL
 
 default_authentication_plugin = mysql_native_password
+binlog_expire_logs_seconds = 86400

documentation/modules/ROOT/pages/connectors/mysql.adoc

@@ -3239,7 +3239,7 @@ a|Controls whether and how long the connector holds the global MySQL read lock,
  +
 `minimal` - the connector holds the global read lock for only the initial portion of the snapshot during which the connector reads the database schemas and other metadata. The remaining work in a snapshot involves selecting all rows from each table. The connector can do this in a consistent fashion by using a REPEATABLE READ transaction. This is the case even when the global read lock is no longer held and other MySQL clients are updating the database. +
  +
-`minimal_percona` - the connector holds link:https://www.percona.com/doc/percona-server/5.7/management/backup_locks.html[the global backup lock] for only the initial portion of the snapshot during which the connector reads the database schemas and other metadata. The remaining work in a snapshot involves selecting all rows from each table. The connector can do this in a consistent fashion by using a REPEATABLE READ transaction. This is the case even when the global backup lock is no longer held and other MySQL clients are updating the database. This mode does not flush tables to disk, is not blocked by long-running reads, and is available only in Percona Server. +
+`minimal_percona` - the connector holds link:https://www.percona.com/doc/percona-server/8.0/management/backup_locks.html[the global backup lock] for only the initial portion of the snapshot during which the connector reads the database schemas and other metadata. The remaining work in a snapshot involves selecting all rows from each table. The connector can do this in a consistent fashion by using a REPEATABLE READ transaction. This is the case even when the global backup lock is no longer held and other MySQL clients are updating the database. This mode does not flush tables to disk, is not blocked by long-running reads, and is available only in Percona Server. +
  +
 `extended` - blocks all writes for the duration of the snapshot. Use this setting if there are clients that are submitting operations that MySQL excludes from REPEATABLE READ semantics. +
  +

documentation/modules/ROOT/pages/operations/kubernetes.adoc

@@ -397,7 +397,7 @@ Connect to the MySQL database:
 
 [source,subs="attributes",options="nowrap"]
 ----
-$ kubectl run -n debezium-example -it --rm --image=mysql:8.0 --restart=Never --env MYSQL_ROOT_PASSWORD=debezium mysqlterm -- mysql -hmysql -P3306 -uroot -pdebezium
+$ kubectl run -n debezium-example -it --rm --image=mysql:8.2 --restart=Never --env MYSQL_ROOT_PASSWORD=debezium mysqlterm -- mysql -hmysql -P3306 -uroot -pdebezium
 ----
 
 Do some changes in the `customers` table:

documentation/modules/ROOT/pages/operations/openshift.adoc

@@ -375,7 +375,7 @@ Connect to the MySQL database:
 
 [source,subs="attributes",options="nowrap"]
 ----
-$ oc run -n debezium-example -it --rm --image=mysql:8.0 --restart=Never --env MYSQL_ROOT_PASSWORD=debezium mysqlterm -- mysql -hmysql -P3306 -uroot -pdebezium
+$ oc run -n debezium-example -it --rm --image=mysql:8.2 --restart=Never --env MYSQL_ROOT_PASSWORD=debezium mysqlterm -- mysql -hmysql -P3306 -uroot -pdebezium
 ----
 
 Do some changes in the `customers` table:

documentation/modules/ROOT/partials/modules/tutorial/proc-starting-mysql-command-line-client.adoc

@@ -13,12 +13,12 @@ After starting MySQL, you start a MySQL command line client so that you access t
 . Open a new terminal, and use it to start the MySQL command line client in a container.
 +
 --
-This command runs a new container using the https://hub.docker.com/r/_/mysql/[mysql:8.0] image,
+This command runs a new container using the https://hub.docker.com/r/_/mysql/[mysql:8.2] image,
 and defines a shell command to run the MySQL command line client with the correct options:
 
 [source,shell,options="nowrap"]
 ----
-$ docker run -it --rm --name mysqlterm --link mysql --rm mysql:8.0 sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"'
+$ docker run -it --rm --name mysqlterm --link mysql --rm mysql:8.2 sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"'
 ----
 
 `-it`:: The container is interactive,
@@ -34,7 +34,7 @@ ifdef::community[]
 If you use Podman, run the following command:
 [source,shell,options="nowrap",subs="+attributes"]
 ----
-$ sudo podman run -it --rm --name mysqlterm --pod dbz --rm mysql:5.7 sh -c 'exec mysql -h 0.0.0.0 -uroot -pdebezium'
+$ sudo podman run -it --rm --name mysqlterm --pod dbz --rm mysql:8.2 sh -c 'exec mysql -h 0.0.0.0 -uroot -pdebezium'
 ----
 ====
 endif::community[]

documentation/modules/ROOT/partials/modules/tutorial/proc-starting-mysql-database.adoc

@@ -15,7 +15,7 @@ In this procedure, you will start a MySQL server with an example database.
 . Open a new terminal, and use it to start a new container that runs a MySQL database server preconfigured with an `inventory` database.
 +
 --
-This command runs a new container using version {debezium-docker-label} of the `quay.io/debezium/example-mysql` image, which is https://github.com/debezium/container-images/blob/main/examples/mysql/{debezium-docker-label}/Dockerfile[based on] the https://hub.docker.com/r/_/mysql/[mysql:8.0] image.
+This command runs a new container using version {debezium-docker-label} of the `quay.io/debezium/example-mysql` image, which is https://github.com/debezium/container-images/blob/main/examples/mysql/{debezium-docker-label}/Dockerfile[based on] the https://hub.docker.com/r/_/mysql/[mysql:8.2] image.
 It also defines and populates a sample `inventory` database:
 
 [source,shell,options="nowrap",subs="attributes"]