= Deploying the Debezium 1.0 Technology Preview with AMQ Streams include::../_attributes.adoc[] :linkattrs: :icons: font :toc: :toc-placement: macro toc::[] The following describes how to set up the https://developers.redhat.com/products/amq/download/[Debezium 1.0.0 Technology Preview connectors] for change data capture on Red Hat's https://www.openshift.com/[OpenShift] container platform. [NOTE] ==== This and other documentation pages mention ```Debezium 1.0.0 Technology Preview```, please note that this version corresponds to version ```1.0.0.Beta2``` of upstream https://github.com/debezium/debezium/tree/v1.0.0.Beta2[Debezium project]. ==== == Apache Kafka Deployment For setting up Apache Kafka and Apache Kafka Connect on OpenShift, https://access.redhat.com/products/red-hat-amq#streams[Red Hat AMQ Streams] can be used, which offers "Kafka as a Service". It consists of an enterprise grade operator and images that bring Kafka to OpenShift. Follow these steps to prepare your Kafka cluster: * Install the AMQ Streams operator by following steps in https://access.redhat.com/documentation/en-us/red_hat_amq/7.5/html/using_amq_streams_on_openshift/getting-started-str#downloads-str[AMQ docs]. * Select the desired configuration and https://access.redhat.com/documentation/en-us/red_hat_amq/7.5/html/using_amq_streams_on_openshift/getting-started-str#downloads-str[deploy your Kafka Cluster]. * Deploy https://access.redhat.com/documentation/en-us/red_hat_amq/7.5/html/using_amq_streams_on_openshift/getting-started-str#using-kafka-connect-with-plug-ins-str[Kafka Connect s2i]. We now have a working ```Kafka cluster``` running within ```OpenShift``` with ```Kafka Connect s2i```. After a while and depending on your Kafka configuration, these pods should be running (assuming you used "debezium-kafka-cluster" as the AMQ Streams Kafka cluster name): [source%nowrap,bash] ---- $ oc get pods NAME READY STATUS RESTARTS AGE debezium-kafka-cluster-entity-operator-7b6b9d4c5f-k7b92 3/3 Running 0 5m debezium-kafka-cluster-kafka-0 2/2 Running 0 4m debezium-kafka-cluster-zookeeper-0 2/2 Running 0 4m strimzi-cluster-operator-97cd5cf7b-l58bq 1/1 Running 0 6m ---- In addition to running pods you should have a ```DeploymentConfig``` associated with your Connect ```s2i```. [NOTE] ==== Instead of utilising the ```KafkaConnectS2I``` you can create a new ```Dockerfile``` based on AMQ's Kafka image, including the required Debezium connectors. The precise instruction can also be found in https://access.redhat.com/documentation/en-us/red_hat_amq/7.3/html/using_amq_streams_on_openshift_container_platform/getting-started-str#using-kafka-connect-with-plug-ins-str[AMQ Streams documentation]. In this case the last item in the list above can be omitted. ==== == Debezium Deployment To deploy a Kafka Cluster with the Debezium Technology Preview connectors we need to * Download the connector archives * Prepare the plugin directory structure * Start the OpenShift s2i build to deploy the final Kafka Connect cluster === Preparing Directory Structure Download the connector archives from https://developers.redhat.com/products/amq/download[Red Hat Developer Portal], extract the archives, and download the required driver dependencies. Once complete you should end with the following directory structure: [source%nowrap,bash] ---- $ tree ./plugins plugins ├── debezium-connector-mongodb │   ├── CHANGELOG.md │   ├── CONTRIBUTE.md │   ├── COPYRIGHT.txt │   ├── LICENSE-3rd-PARTIES.txt │   ├── LICENSE.txt │   ├── README.md │   ├── bson-3.10.1.redhat-00001.jar │   ├── debezium-connector-mongodb-1.0.0.Beta2-redhat-00001.jar │   ├── debezium-core-1.0.0.Beta2-redhat-00001.jar │   ├── mongodb-driver-3.10.1.redhat-00001.jar │   ├── mongodb-driver-core-3.10.1.redhat-00001.jar │   └── util-3.10.1.redhat-00001.jar ├── debezium-connector-mysql │   ├── CHANGELOG.md │   ├── CONTRIBUTE.md │   ├── COPYRIGHT.txt │   ├── LICENSE-3rd-PARTIES.txt │   ├── LICENSE.txt │   ├── README.md │   ├── antlr4-runtime-4.7.0.redhat-00013.jar │   ├── debezium-connector-mysql-1.0.0.Beta2-redhat-00001.jar │   ├── debezium-core-1.0.0.Beta2-redhat-00001.jar │   ├── debezium-ddl-parser-1.0.0.Beta2-redhat-00001.jar │   ├── mysql-binlog-connector-java-0.19.1.redhat-00002.jar │   └── mysql-connector-java-8.0.16.redhat-00001.jar ├── debezium-connector-postgres │   ├── CHANGELOG.md │   ├── CONTRIBUTE.md │   ├── COPYRIGHT.txt │   ├── LICENSE-3rd-PARTIES.txt │   ├── LICENSE.txt │   ├── README.md │   ├── debezium-connector-postgres-1.0.0.Beta2-redhat-00001.jar │   ├── debezium-core-1.0.0.Beta2-redhat-00001.jar │   ├── postgresql-42.2.8.redhat-00001.jar │   └── protobuf-java-3.8.0.redhat-00001.jar └── debezium-connector-sqlserver ├── CHANGELOG.md ├── CONTRIBUTE.md ├── COPYRIGHT.txt ├── LICENSE-3rd-PARTIES.txt ├── LICENSE.txt ├── README.md ├── debezium-connector-sqlserver-1.0.0.Beta2-redhat-00001.jar ├── debezium-core-1.0.0.Beta2-redhat-00001.jar └── mssql-jdbc-7.2.2.jre8-redhat-00001.jar ---- === Deploying Kafka Connect with the Debezium Technology Preview With plugin directory structure ready, all we have to do now in order to deploy our Kafka Connect cluster is triggering the ```s2i``` build. [source%nowrap,bash] ---- # Check the name of our build config $ oc get buildconfigs NAME TYPE FROM LATEST debezium-kafka-connect-cluster-connect Source Binary 2 # Build and deploy Kafka Connect cluster with Debezium $ oc start-build debezium-kafka-connect-cluster-connect --from-dir=./plugins ---- After a while all parts should be up and running: [source%nowrap,bash] ---- oc get pods NAME READY STATUS RESTARTS AGE debezium-kafka-cluster-entity-operator-7b6b9d4c5f-k7b92 3/3 Running 0 10m debezium-kafka-cluster-kafka-0 2/2 Running 0 9m debezium-kafka-cluster-zookeeper-0 2/2 Running 0 9m debezium-kafka-connect-cluster-connect-2-jw695 1/1 Running 0 1m debezium-kafka-connect-cluster-connect-2-deploy 0/1 Completed 0 3m strimzi-cluster-operator-97cd5cf7b-l58bq 1/1 Running 0 11m ---- Alternatively, you can go to the "Pods" view of your OpenShift Web Console to confirm all pods are up and running: image::/images/openshift_amq_pods.png[width=771,align="center"] == Verifying the Deployment Next we are going to verify whether the deployment is correct by emulating the xref:tutorial.adoc[Debezium Tutorial] and following the steps in xref:operations/openshift.adoc#verifying_the_deployment[OpenShift Installation] [NOTE] ==== You will want to change the pod names to correspond with your AMQ Streams deployment. ==== == Questions and Resources In case of any requests or questions related to running the Debezium Technology Preview with AMQ Streams on OpenShift, please let us know by sending an e-mail to mailto:debezium-cdc-preview@redhat.com[debezium-cdc-preview] mailing list.