PyCon UK 2025

Observing production systems with OpenTelemetry
2025-09-21 , Space 4

Understanding how your production systems are working is vitally important, but hard to do.

In this workshop, we will hands-on learn how to use OpenTelemetry to instrument a Django application, and how to use it to diagnose problems in production.


When running your code in production, it is essential be able to observe what the code is doing, and how long it is taking. This allows you to diagnose problems, improve performance, and plan for growth. This property is known as Observability - the ability to determine the internal state of the system from observing its outputs.

But production systems are often large and complex, which makes this difficult, and developers do not always have unrestricted access to inspect the internals of the system, for valid reasons. There are many tools and methods that provide aggregated logging, metrics and tracing, all in order to try find ways to answer these questions.

Over the last few years, OpenTelemetry has emerged as the leading open standard and tool-set to instrument code to provide deep observability. It provides a standard way to augment your application code to emit telemetry that can be use to answer complex questions about your system. It has seen rapid adoption across different language ecosystems and observabilty vendors. However, it introduces many new concepts, and its cross-platform and design-by-committee nature can make it a bit of steep learning curve.

This workshop will introduce OpenTelemetry concepts, and provide a series of hands on interactive exercises to work through, based on a provided Django application, which will include:
- adding basic auto-instrumentation
- emitting it to Honeycomb to explore
- diagnosing a slow request
- adding custom application specific telemetry
- linking telemetry across services

This is intended to be an intermediate workshop - you will need basic familiarity with python and python packages, and Django and its ecosystem, to get the most of out of it. To do the exercises, you will need a computing device with either uv installed, or a GitHub account to use Codespaces.


What level of experience do you expect from your audience for this session?:

Intermediate

I am a infrastructure focussed developer from Leeds, UK, and a PyCon UK regular, having been coming since 2008,

I been using python since around 2003, initially in an academic context, for my PhD in AI Scheduling, and then commercially for various companies. I worked for Canonical on Ubuntu for 9 years, and currently work at the Bennett Institute for Applied Data Science at the University of Oxford, on the OpenSAFELY project.

I am particularly interested in all the fun and hard things about running complex production software - security, scalability, testing, user experience, developer experience, and observability.