2024-10-26 –, CLASS #2 - 4B
Language: English
Kafka is a widely used open-source event streaming platform that aggregates data events and works as a pub-sub. To make sure each application can only have access to messages they are allowed to consume and produce Kafka Admins can implement Kafka ACL rules to determine which principals are allowed to produce and consume messages to certain topics. Unfortunately reading and changing ACLs through Kafka CLI can be unintuitive and prone to mistakes for users who are not familiar with how Kafka ACLs work. A simple error in changing Kafka ACLs can cause system failures on multiple services that rely on Kafka, especially any changes that include wildcard. To prevent this, a quick Python script leveraging the confluent_kafka library can safeguard these changes by reviewing their changes before applying them, preventing catastrophic incidents from happening.
In this talk I would like to present a quick Python script I created to create and remove Kafka ACLs, this script can be created quickly and allows us to review our changes before any change on Kafka clusters. This would show how Python can be easily leveraged to make simple scripts that can automate and simplify changes for sysadmins, it can also safeguard critical changes.
Theodore Gautama Chandra or Theo works as a system engineer on Goto Financial, more specifically on the Stateful Components Team that manages Stateful Components such as Postgres, Redis and Kafka. Theo specifically works on Kafka and currently setting up a new Kafka environment for the whole GoTo Financial engineering ecosystem. Previously Theo worked as a devops engineer in the midtrans team where he handles the whole midtrans systems infrastructure. Theo has a degree in engineering physics and has been using Python for data analysis and machine learning for his university projects and currently using Python to automate sysadmins tasks in his current work.