Change data capture with Debezium: A simple how-to

Click for: original source

Eric Deandrea wrote this piece about one question that always comes up as organizations moving towards being cloud-native, twelve-factor, and stateless: How do you get an organization’s data to these new applications?

There are many different patterns out there, but one pattern we will look in this artticle is change data capture. This post is a simple how-to on how to build out a change data capture solution using Debezium within an OpenShift environment. Future posts will also add to this and add additional capabilities.

Change data capture (CDC) is a pattern that enables database changes to be monitored and propagated to downstream systems. It is an effective way of enabling reliable microservices integration and solving typical challenges, such as gradually extracting microservices from existing monoliths.

Sadhana Nandakumar

The article main chapters:

  • What is change data capture?
  • What is Debezium?
  • Application overview
  • Setting up the database
  • Create an OpenShift project
  • Create the MySQL configuration
  • Starting the application
  • Deploy AMQ Streams
  • Deploy Kafka Connect

… and more. This step by step detailed tutorial and if you want to automate the provisioning of everything, feel free to take a look at an Ansible Playbook that is capable of doing this. Excellent read!

[Read More]

Tags software-architecture streaming apache data-science queues