Airflow the perfect match in our Analytics Pipeline
07-15, 23:00–23:45 (US/Pacific), Melbourne Meetup, [Sessions start: Thursday 16.07 2pm (Wednesday 15.07, 9 pm PDT)]

Working with Airflow is no breeze. For three years we at LOVOO, a market-leading dating app, have been using the Google Cloud managed version of Airflow, a product we’ve been familiar with since its Alpha release. We took a calculated risk and integrated the Alpha into our product, and, luckily, it was a match. Since then, we have been leveraging this software to build out not only our data pipeline, but also boost the way we do analytics and BI.


I’ll begin the talk by giving a brief outline of Google Composer. Following this, I want to highlight the various use cases we have here at LOVOO; how we as a business take advantage of Airflow; as well as how we deploy version-controlled DAGs.

To give more insight into our processes, I will present an overview of the software’s usability for Pipeline Error Alerting through SlackOperators. I will touch upon how we built our Analytics Pipeline (deployment and growth) and currently batch big amounts of data from different sources effectively using Airflow. More detailed examples will be featured here, focusing on Python,- and Kubernetes Operators and their interaction with our self-written data importers.
As part of the hands-on, I will showcase our PythonOperators-driven RedShift to BigQuery data migration process, as well as offer a guide for creating fully dynamic tasks inside DAG.

Born in Bogotá Colombia in 1990, studied Systems Engineering and then migrated to Germany in 2012. Worked as a Software Engineer and always loved to work with data. I did my Master of Science in Business Intelligence & Analytics and still continue taking online courses to learn more about Data Science. Currently working as Business Intelligence Architect for Lovoo (a The Meet Group Dating App)