haocheng
Senior Engineer at LY corp, interested in Java, JVM, Continuous Integration and Test-Driven Development, also organizer of JCConf and TWJUG
Session
I would like to share how we have utilized Kafka Connect to complete DB migration with zero downtime. Kafka Connect is a tool for building scalable and reliable data pipelines between Apache Kafka and other data systems. Kafka Connect allows us to ingest entire DB into Kafka topics so that export connector can migrate the data from Kafka topics to another DB. Kafka Connect also supports rollback to previous DB in case any issue happens after the migration is done which is usually very hard for built-in DB migration tools. Moreover, Kafka Connect not only supports DB to DB or table to table migrations, but also supports custom write strategy which could let us migrate only part of the data we need.