2025-09-19 –, Space 2
A client was happy with a prototype I delivered, except for one detail: they needed my code to be about 100 times faster. I got there eventually, but through much confusion and mistakes, which I will share with you, so you don't have to go through them yourself.
A few years ago, I took on a project to rewrite an old C project in Python. Once I delivered the prototype, the client was happy, except for one little detail: they ran some tasks against my code, which we had never discussed in detail, that used to take 10 minutes, and now took over 10 hours. So, I needed to make it about 100 times faster.
I did think early on about the most likely bottlenecks, and anticipated what might need improvement later. I considered the size of the data I had to work with, and how it would be queried. It didn't seem like a hard problem, the dataset wasn't that large. But as it turned out, all my assumptions and plans were wrong.
Being a generalist, I had never done Python performance optimisation at this level - most things I built were fast enough with minor effort. So I tried to collect more measurements, in ways I did not understand, drew the wrong conclusions and optimised the wrong components. And it didn't help that my project is based on obscure protocols from the late 90s with weird legacy compatibility requirements.
I found my way around poor assumptions ("surely PostgreSQL will be the main bottleneck"), unknown expectations ("what do you mean, you want to run 1 million queries?"), and misunderstood measurements ("I can't believe SQLAlchemy is so slow - or is it?"). But after numerous red herrings, aha-moments, and questioning my career choices, I got the project to where it needed to be.
This talk will take you through the path I took, the lessons I learned from it, and things I would do different today. You'll learn something from it if you have or ever might write code where performance might matter.
Intermediate
Sasha is an independent developer and community organiser with over a decade of Python experience. A former Django core developer and co-organiser of several Django conferences, they now work mostly on open source software related to internet infrastructure, network engineering, and internet standards. They've also been part of the Write the Docs conference team since 2018 and enjoy side projects like the Less Obvious Conference Checklist and Happiness Packets, which focus on improving the tech community in small but meaningful ways.