DBZ-5227 Specify use of JAR artifact for script engine language
This commit is contained in:
Bob Roldan
Jiri Pechanec
parent
25728f8897
commit
cd1caf682d
@ -13,6 +13,7 @@ asciidoc:
|
||||
debezium-kafka-version: '3.1.0'
|
||||
debezium-docker-label: '1.9'
|
||||
DockerKafkaConnect: registry.redhat.io/amq7/amq-streams-kafka-28-rhel8:1.8.0
|
||||
groovy-version: '3.0.11'
|
||||
install-version: '1.9'
|
||||
assemblies: '../assemblies'
|
||||
modules: '../../modules'
|
||||
|
||||
@ -43,9 +43,16 @@ For example:
|
||||
The content-based routing SMT supports scripting languages that integrate with https://jcp.org/en/jsr/detail?id=223[JSR 223] (Scripting for the Java(TM) Platform).
|
||||
|
||||
{prodname} does not come with any implementations of the JSR 223 API.
|
||||
To use an expression language with {prodname}, you must download the JSR 223 script engine implementation for the language, and add to your {prodname} connector plug-in directories, along any other JAR files used by the language implementation.
|
||||
To use an expression language with {prodname}, you must download the JSR 223 script engine implementation for the language.
|
||||
ifdef::community[]
|
||||
For example, for Groovy 3, you can download its JSR 223 implementation from https://groovy-lang.org/.
|
||||
The JSR 223 implementation for GraalVM JavaScript is available at https://github.com/graalvm/graaljs.
|
||||
The JSR223 implementation for GraalVM JavaScript is available at https://github.com/graalvm/graaljs.
|
||||
After you obtain the script engine files, you add them to your {prodname} connector plug-in directories, along any other JAR files used by the language implementation.
|
||||
endif::community[]
|
||||
ifdef::product[]
|
||||
Depending on the method that you use to deploy {prodname}, you can automatically download the required artifacts from Maven Central,
|
||||
or you can manually download the artifacts, and then add them to your {prodname} connector plug-in directories, along any other JAR files used by the language implementation.
|
||||
endif::product[]
|
||||
|
||||
// Type: procedure
|
||||
// Title: Setting up the {prodname} content-based-routing SMT
|
||||
@ -55,8 +62,14 @@ The JSR 223 implementation for GraalVM JavaScript is available at https://github
|
||||
|
||||
For security reasons, the content-based routing SMT is not included with the {prodname} connector archives.
|
||||
Instead, it is provided in a separate artifact, `debezium-scripting-{debezium-version}.tar.gz`.
|
||||
To use the content-based routing SMT with a {prodname} connector plug-in, you must explicitly add the SMT artifact to your Kafka Connect environment.
|
||||
|
||||
ifdef::product[]
|
||||
If you deploy the {prodname} connector by building a custom Kafka Connect container image from a Dockerfile, to use the filter SMT, you must explicitly add the SMT artifact to your Kafka Connect environment.
|
||||
When you use {StreamsName} to deploy the connector, it can download the required artifacts automatically based on configuration parameters that you specify in the Kafka Connect custom resource.
|
||||
endif::product[]
|
||||
ifdef::community[]
|
||||
To use the content-based routing SMT with a {prodname} connector plug-in, you must explicitly add the SMT artifact to your Kafka Connect environment.
|
||||
endif::community[]
|
||||
IMPORTANT: After the routing SMT is present in a Kafka Connect instance, any user who is allowed to add a connector to the instance can run scripting expressions.
|
||||
To ensure that scripting expressions can be run only by authorized users, be sure to secure the Kafka Connect instance and its configuration interface before you add the routing SMT.
|
||||
|
||||
@ -70,6 +83,9 @@ With https://zookeeper.apache.org[Zookeeper], http://kafka.apache.org/[Kafka], {
|
||||
endif::community[]
|
||||
|
||||
ifdef::product[]
|
||||
The following procedure applies if you build your Kafka Connect container image from a Dockerfile.
|
||||
If you use {StreamsName} to create the Kafka Connect image, follow the instructions in the deployment topic for your connector.
|
||||
|
||||
.Procedure
|
||||
. From a browser, open the link:{DebeziumDownload}, and download the {prodname} scripting SMT archive (`debezium-scripting-{debezium-version}.tar.gz`).
|
||||
. Extract the contents of the archive into the {prodname} plug-in directories of your Kafka Connect environment.
|
||||
|
||||
@ -41,8 +41,16 @@ For example:
|
||||
The filter SMT supports scripting languages that integrate with https://jcp.org/en/jsr/detail?id=223[JSR 223] (Scripting for the Java(TM) Platform).
|
||||
|
||||
{prodname} does not come with any implementations of the JSR 223 API.
|
||||
To use an expression language with {prodname}, you must download the JSR 223 script engine implementation for the language, and add to your {prodname} connector plug-in directories, along any other JAR files used by the language implementation.
|
||||
For example, for Groovy 3, you can download its JSR 223 implementation from https://groovy-lang.org/. The JSR223 implementation for GraalVM JavaScript is available at https://github.com/graalvm/graaljs.
|
||||
To use an expression language with {prodname}, you must download the JSR 223 script engine implementation for the language.
|
||||
ifdef::community[]
|
||||
For example, for Groovy 3, you can download its JSR 223 implementation from https://groovy-lang.org/.
|
||||
The JSR223 implementation for GraalVM JavaScript is available at https://github.com/graalvm/graaljs.
|
||||
After you obtain the script engine files, you add them to your {prodname} connector plug-in directories, along any other JAR files used by the language implementation.
|
||||
endif::community[]
|
||||
ifdef::product[]
|
||||
Depending on the method that you use to deploy {prodname}, you can automatically download the required artifacts from Maven Central,
|
||||
or you can manually download the artifacts, and then add them to your {prodname} connector plug-in directories, along any other JAR files used by the language implementation.
|
||||
endif::product[]
|
||||
|
||||
// Type: procedure
|
||||
// Title: Setting up the {prodname} filter SMT
|
||||
@ -52,8 +60,14 @@ For example, for Groovy 3, you can download its JSR 223 implementation from http
|
||||
|
||||
For security reasons, the filter SMT is not included with the {prodname} connector archives.
|
||||
Instead, it is provided in a separate artifact, `debezium-scripting-{debezium-version}.tar.gz`.
|
||||
To use the filter SMT with a {prodname} connector plug-in, you must explicitly add the SMT artifact to your Kafka Connect environment.
|
||||
|
||||
ifdef::product[]
|
||||
If you deploy the {prodname} connector by building a custom Kafka Connect container image from a Dockerfile, to use the filter SMT, you must explicitly download the SMT archive and deploy the files alongside the connector plug-in.
|
||||
When you use {StreamsName} to deploy the connector, it can download the required artifacts automatically based on configuration parameters that you specify in the Kafka Connect custom resource.
|
||||
endif::product[]
|
||||
ifdef::community[]
|
||||
To use the content-based routing SMT with a {prodname} connector plug-in, you must explicitly add the SMT artifact to your Kafka Connect environment.
|
||||
endif::community[]
|
||||
IMPORTANT: After the filter SMT is present in a Kafka Connect instance, any user who is allowed to add a connector to the instance can run scripting expressions.
|
||||
To ensure that scripting expressions can be run only by authorized users, be sure to secure the Kafka Connect instance and its configuration interface before you add the filter SMT.
|
||||
|
||||
@ -67,6 +81,9 @@ With https://zookeeper.apache.org[Zookeeper], http://kafka.apache.org/[Kafka], {
|
||||
endif::community[]
|
||||
|
||||
ifdef::product[]
|
||||
The following procedure applies if you build your Kafka Connect container image from a Dockerfile.
|
||||
If you use {StreamsName} to create the Kafka Connect image, follow the instructions in the deployment topic for your connector.
|
||||
|
||||
.Procedure
|
||||
. From a browser, open the link:{DebeziumDownload}, and download the {prodname} scripting SMT archive (`debezium-scripting-{debezium-version}.tar.gz`).
|
||||
. Extract the contents of the archive into the {prodname} plug-in directories of your Kafka Connect environment.
|
||||
|
||||
@ -2,7 +2,10 @@ In the example that follows, the custom resource is configured to download the f
|
||||
|
||||
* The {prodname} {connector-name} connector archive.
|
||||
* The {registry-name-full} archive. The {registry} is an optional component.
|
||||
* The {prodname} scripting SMT archive. The SMT archive is an optional component.
|
||||
Add the {registry} component only if you intend to use Avro serialization with the connector.
|
||||
* The {prodname} scripting SMT archive and the associated language dependencies that you want to use with the Debezium connector.
|
||||
The SMT archive and language dependencies are optional components.
|
||||
Add these components only if you intend to use the {prodname} {link-prefix}:{link-content-based-routing}#content-based-routing[content-based routing SMT] or {link-prefix}:{link-filtering}#message-filtering[filter SMT].
|
||||
* The {connector-name} JDBC driver, which is required to connect to {connector-name} databases, but is not included in the connector archive.
|
||||
|
||||
[source%nowrap,yaml,subs="+attributes,+quotes"]
|
||||
@ -25,10 +28,16 @@ spec:
|
||||
- type: zip // <6>
|
||||
url: {red-hat-maven-repository}debezium/debezium-connector-{connector-file}/{debezium-version}-redhat-__<build_number>__/debezium-connector-{connector-file}-{debezium-version}-redhat-__<build_number>__-plugin.zip // <7>
|
||||
- type: zip
|
||||
url: {red-hat-maven-repository}apicurio/apicurio-registry-distro-connect-converter/{registry-version}-redhat-_<build-number>_/apicurio-registry-distro-connect-converter-{registry-version}-redhat-_<build-number>_.zip
|
||||
url: {red-hat-maven-repository}apicurio/apicurio-registry-distro-connect-converter/{registry-version}-redhat-_<build-number>_/apicurio-registry-distro-connect-converter-{registry-version}-redhat-_<build-number>_.zip // <8>
|
||||
- type: zip
|
||||
url: {red-hat-maven-repository}debezium/debezium-scripting/{debezium-version}/debezium-scripting-{debezium-version}.zip
|
||||
- type: jar // <8>
|
||||
url: {red-hat-maven-repository}debezium/debezium-scripting/{debezium-version}/debezium-scripting-{debezium-version}.zip // <9>
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy/{groovy-version}/groovy-{groovy-version}.jar // <10>
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-jsr223/{groovy-version}/groovy-jsr223-{groovy-version}.jar
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-json{groovy-version}/groovy-json-{groovy-version}.jar
|
||||
- type: jar // <11>
|
||||
url: https://repo1.maven.org/maven2/com/ibm/db2/jcc/{db2-version}/jcc-{db2-version}.jar
|
||||
|
||||
bootstrapServers: debezium-kafka-cluster-kafka-bootstrap:9093
|
||||
@ -70,8 +79,34 @@ The `type` value must match the type of the file that is referenced in the `url`
|
||||
|The value of `artifacts.url` specifies the address of an HTTP server, such as a Maven repository, that stores the file for the connector artifact.
|
||||
The OpenShift cluster must have access to the specified server.
|
||||
|
||||
|
||||
|8
|
||||
|(Optional) Specifies the artifact `type` and `url` for downloading the {registry} component.
|
||||
Include the {registry} artifact, only if you want the connector to use Apache Avro to serialize event keys and values with the {registry-name-full}, instead of using the default JSON converter.
|
||||
|
||||
|9
|
||||
|(Optional) Specifies the artifact `type` and `url` for the {prodname} scripting SMT archive to use with the Debezium connector.
|
||||
Include the scripting SMT only if you intend to use the {prodname} {link-prefix}:{link-content-based-routing}#content-based-routing[content-based routing SMT] or {link-prefix}:{link-filtering}#message-filtering[filter SMT]
|
||||
To use the scripting SMT, you must also deploy a JSR 223-compliant scripting implementation, such as groovy.
|
||||
|
||||
|10
|
||||
a|(Optional) Specifies the artifact `type` and `url` for the JAR files of a JSR 223-compliant scripting implementation, which is required by the {prodname} scripting SMT.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
If you use {StreamsName} to incorporate the connector plug-in into your Kafka Connect image, for each of the required scripting language components, `artifacts.url` must specify the location of a JAR file,
|
||||
and the value of `artifacts.type` must also be set to `jar`.
|
||||
Invalid values cause the connector fails at runtime.
|
||||
====
|
||||
|
||||
To enable use of the Apache Groovy language with the scripting SMT, the custom resource in the example retrieves JAR files for the following libraries:
|
||||
|
||||
- `groovy`
|
||||
- `groovy-jsr223` (scripting agent)
|
||||
- `groovy-json` (module for parsing JSON strings)
|
||||
|
||||
The {prodname} scripting SMT also supports the use of the JSR 223 implementation of GraalVM JavaScript.
|
||||
|
||||
|11
|
||||
|Specifies the location of the {connector-name} JDBC driver in Maven Central.
|
||||
The required driver is not included in the {prodname} {connector-name} connector archive.
|
||||
|
||||
|
||||
@ -2,7 +2,10 @@ In the example that follows, the custom resource is configured to download the f
|
||||
|
||||
* The {prodname} {connector-name} connector archive.
|
||||
* The {registry-name-full} archive. The {registry} is an optional component.
|
||||
* The {prodname} scripting SMT archive. The SMT archive is an optional component.
|
||||
Add the {registry} component only if you intend to use Avro serialization with the connector.
|
||||
* The {prodname} scripting SMT archive and the associated language dependencies that you want to use with the Debezium connector.
|
||||
The SMT archive and language dependencies are optional components.
|
||||
Add these components only if you intend to use the {prodname} {link-prefix}:{link-content-based-routing}#content-based-routing[content-based routing SMT] or {link-prefix}:{link-filtering}#message-filtering[filter SMT].
|
||||
* The {connector-name} JDBC driver, which is required to connect to {connector-name} databases, but is not included in the connector archive.
|
||||
|
||||
[source%nowrap,yaml,subs="+attributes,+quotes"]
|
||||
@ -25,10 +28,16 @@ spec:
|
||||
- type: zip // <6>
|
||||
url: {red-hat-maven-repository}debezium/debezium-connector-{connector-file}/{debezium-version}-redhat-__<build_number>__/debezium-connector-{connector-file}-{debezium-version}-redhat-__<build_number>__-plugin.zip // <7>
|
||||
- type: zip
|
||||
url: {red-hat-maven-repository}apicurio/apicurio-registry-distro-connect-converter/{registry-version}-redhat-_<build-number>_/apicurio-registry-distro-connect-converter-{registry-version}-redhat-_<build-number>_.zip
|
||||
url: {red-hat-maven-repository}apicurio/apicurio-registry-distro-connect-converter/{registry-version}-redhat-_<build-number>_/apicurio-registry-distro-connect-converter-{registry-version}-redhat-_<build-number>_.zip // <8>
|
||||
- type: zip
|
||||
url: {red-hat-maven-repository}debezium/debezium-scripting/{debezium-version}/debezium-scripting-{debezium-version}.zip
|
||||
- type: jar // <8>
|
||||
url: {red-hat-maven-repository}debezium/debezium-scripting/{debezium-version}/debezium-scripting-{debezium-version}.zip // <9>
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy/{groovy-version}/groovy-{groovy-version}.jar // <10>
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-jsr223/{groovy-version}/groovy-jsr223-{groovy-version}.jar
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-json{groovy-version}/groovy-json-{groovy-version}.jar
|
||||
- type: jar // <11>
|
||||
url: https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/{ojdbc8-version}/ojdbc8-{ojdbc8-version}.jar
|
||||
|
||||
bootstrapServers: debezium-kafka-cluster-kafka-bootstrap:9093
|
||||
@ -72,6 +81,33 @@ The `type` value must match the type of the file that is referenced in the `url`
|
||||
The OpenShift cluster must have access to the specified server.
|
||||
|
||||
|8
|
||||
|(Optional) Specifies the artifact `type` and `url` for downloading the {registry} component.
|
||||
Include the {registry} artifact, only if you want the connector to use Apache Avro to serialize event keys and values with the {registry-name-full}, instead of using the default JSON converter.
|
||||
|
||||
|9
|
||||
|(Optional) Specifies the artifact `type` and `url` for the {prodname} scripting SMT archive to use with the Debezium connector.
|
||||
Include the scripting SMT only if you intend to use the {prodname} {link-prefix}:{link-content-based-routing}#content-based-routing[content-based routing SMT] or {link-prefix}:{link-filtering}#message-filtering[filter SMT]
|
||||
To use the scripting SMT, you must also deploy a JSR 223-compliant scripting implementation, such as groovy.
|
||||
|
||||
|10
|
||||
a|(Optional) Specifies the artifact `type` and `url` for the JAR files of a JSR 223-compliant scripting implementation, which is required by the {prodname} scripting SMT.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
If you use {StreamsName} to incorporate the connector plug-in into your Kafka Connect image, for each of the required scripting language components `artifacts.url` must specify the location of a JAR file,
|
||||
and the value of `artifacts.type` must also be set to `jar`.
|
||||
Invalid values cause the connector fails at runtime.
|
||||
====
|
||||
|
||||
To enable use of the Apache Groovy language with the scripting SMT, the custom resource in the example retrieves JAR files for the following libraries:
|
||||
|
||||
- `groovy`
|
||||
- `groovy-jsr223` (scripting agent)
|
||||
- `groovy-json` (module for parsing JSON strings)
|
||||
|
||||
The {prodname} scripting SMT also supports the use of the JSR 223 implementation of GraalVM JavaScript.
|
||||
|
||||
|11
|
||||
|Specifies the location of the {connector-name} JDBC driver in Maven Central.
|
||||
The required driver is not included in the {prodname} {connector-name} connector archive.
|
||||
|
||||
|
||||
@ -2,7 +2,10 @@ In the example that follows, the custom resource is configured to download the f
|
||||
|
||||
* The {prodname} {connector-name} connector archive.
|
||||
* The {registry-name-full} archive. The {registry} is an optional component.
|
||||
* The {prodname} scripting SMT archive. The SMT archive is an optional component.
|
||||
Add the {registry} component only if you intend to use Avro serialization with the connector.
|
||||
* The {prodname} scripting SMT archive and the associated scripting engine that you want to use with the Debezium connector.
|
||||
The SMT archive and scripting language dependencies are optional components.
|
||||
Add these components only if you intend to use the {prodname} {link-prefix}:{link-content-based-routing}#content-based-routing[content-based routing SMT] or {link-prefix}:{link-filtering}#message-filtering[filter SMT].
|
||||
|
||||
[source%nowrap,yaml,subs="+attributes,+quotes"]
|
||||
----
|
||||
@ -24,9 +27,15 @@ spec:
|
||||
- type: zip // <6>
|
||||
url: {red-hat-maven-repository}debezium/debezium-connector-{connector-file}/{debezium-version}-redhat-__<build_number>__/debezium-connector-{connector-file}-{debezium-version}-redhat-__<build_number>__-plugin.zip // <7>
|
||||
- type: zip
|
||||
url: {red-hat-maven-repository}apicurio/apicurio-registry-distro-connect-converter/{registry-version}-redhat-_<build-number>_/apicurio-registry-distro-connect-converter-{registry-version}-redhat-_<build-number>_.zip
|
||||
url: {red-hat-maven-repository}apicurio/apicurio-registry-distro-connect-converter/{registry-version}-redhat-_<build-number>_/apicurio-registry-distro-connect-converter-{registry-version}-redhat-_<build-number>_.zip // <8>
|
||||
- type: zip
|
||||
url: {red-hat-maven-repository}debezium/debezium-scripting/{debezium-version}/debezium-scripting-{debezium-version}.zip
|
||||
url: {red-hat-maven-repository}debezium/debezium-scripting/{debezium-version}/debezium-scripting-{debezium-version}.zip // <9>
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy/{groovy-version}/groovy-{groovy-version}.jar // <10>
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-jsr223/{groovy-version}/groovy-jsr223-{groovy-version}.jar
|
||||
- type: jar
|
||||
url: https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-json{groovy-version}/groovy-json-{groovy-version}.jar
|
||||
|
||||
bootstrapServers: debezium-kafka-cluster-kafka-bootstrap:9093
|
||||
----
|
||||
@ -46,7 +55,7 @@ spec:
|
||||
|
||||
|4
|
||||
|Specifies the name and image name for the image output.
|
||||
Valid values for `output.type` are `docker` to push into a container registry like Docker Hub or Quay, or `imagestream` to push the image to an internal OpenShift ImageStream.
|
||||
Valid values for `output.type` are `docker` to push into a container registry such as Docker Hub or Quay, or `imagestream` to push the image to an internal OpenShift ImageStream.
|
||||
To use an ImageStream, an ImageStream resource must be deployed to the cluster.
|
||||
For more information about specifying the `build.output` in the KafkaConnect configuration, see the link:{LinkStreamsOpenShift}#type-Build-reference[{StreamsName} Build schema reference documentation].
|
||||
|
||||
@ -67,5 +76,32 @@ The `type` value must match the type of the file that is referenced in the `url`
|
||||
{prodname} connector artifacts are available in the Red Hat Maven repository.
|
||||
The OpenShift cluster must have access to the specified server.
|
||||
|
||||
|8
|
||||
|(Optional) Specifies the artifact `type` and `url` for downloading the {registry} component.
|
||||
Include the {registry} artifact, only if you want the connector to use Apache Avro to serialize event keys and values with the {registry-name-full}, instead of using the default JSON converter.
|
||||
|
||||
|9
|
||||
|(Optional) Specifies the artifact `type` and `url` for the {prodname} scripting SMT archive to use with the Debezium connector.
|
||||
Include the scripting SMT only if you intend to use the {prodname} {link-prefix}:{link-content-based-routing}#content-based-routing[content-based routing SMT] or {link-prefix}:{link-filtering}#message-filtering[filter SMT]
|
||||
To use the scripting SMT, you must also deploy a JSR 223-compliant scripting implementation, such as groovy.
|
||||
|
||||
|10
|
||||
a|(Optional) Specifies the artifact `type` and `url` for the JAR files of a JSR 223-compliant scripting implementation, which is required by the {prodname} scripting SMT.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
If you use {StreamsName} to incorporate the connector plug-in into your Kafka Connect image, for each of the required scripting language components `artifacts.url` must specify the location of a JAR file,
|
||||
and the value of `artifacts.type` must also be set to `jar`.
|
||||
Invalid values cause the connector fails at runtime.
|
||||
====
|
||||
|
||||
To enable use of the Apache Groovy language with the scripting SMT, the custom resource in the example retrieves JAR files for the following libraries:
|
||||
|
||||
- `groovy`
|
||||
- `groovy-jsr223` (scripting agent)
|
||||
- `groovy-json` (module for parsing JSON strings)
|
||||
|
||||
As an alternative, the {prodname} scripting SMT also supports the use of the JSR 223 implementation of GraalVM JavaScript.
|
||||
|
||||
|===
|
||||
=====================================================================
|
||||
|
||||
Loading…
Reference in New Issue
Block a user