Ed Crewe
Cloud developer at EDB, the Postgres company.
Bristol, United Kingdom
I have been a cloud engineer for the last 9 years. Working in Golang for the last 6, and Python for the last 20 years. Before that I was a web developer. I have spoken at Kubecon, Djangocon.Eu and EuroPython as well as many times at local techie groups, including regularly at my local Golang meetup.
Currently I spend my time developing microservices for EDB's cross cloud Postgres AI product, built on our CNCF kubernetes postgres operator cloudnative-pg.
Plus the occasional Python automation. I am interested in improving DevX for the full lifecycle of Cloud development, and the cost and sustainability of cloud use.
https://github.com/edcrewe/django-csvimport
https://github.com/cloudnative-pg/cloudnative-pg
https://github.com/postgres/postgres (My company EDB is a core contributor)
Session
A case study of rewriting a simple data pipeline involving Python, a pinch of Go, Git workflows, Airflow, Postgres and Cloud. Investigating some common assumptions and principles of designing data pipelines.
The benefits and issues with the tools and how these may be handled.
I hope this case study of a pipeline rewrite will give you insights that are applicable to Python use for your own data pipelines, and into cloud pricing.