2023-10-15 –, Tutorial Track C
As data-driven organizations grow, life gets... complicated
- data pipelines requirements grow
- more data sources and sinks are added to the mix
- tasks become dependent on each other in complicated ways
- service agreements need to be adhered to and monitored
- schedules need to be maintained...
Apache Airflow, initially developed and open-sourced by those nice folks at Airbnb, solves these problems and more.
In this tutorial session, we'll be starting off by getting to grips with Airflow as a stand-alone tool, and then we'll see how we can get it to play nice with Django.
It would be useful if attendees were familiar with Django coming into this session. No prior experience with Airflow is needed.
What we'll cover:
- What Airflow is all about and what kinds of problems you could use it to solve
- Airflow development setup
- Getting to grips with Dags and operators
- Integration with Django
I'm currently the CTO of Umuzi, our mission is to reduce social inequality in Africa through digital education. My role involves: figuring out what to teach and how to teach it, writing software (we use Django a LOT), building management systems, creating new products, and mentorship.
Outside of work I like to juggle as many hobbies as I can - BJJ, climbing, hiking and music are high on my list.