PyConDE & PyData Berlin 2024

Advanced Observability with OpenTelemetry and Python
2024-04-23 , B05-B06

As Python expands into serverless and cloud environments, popularizing distributed microservice architectures, we often face observability challenges that impact efficiency and complicate error tracing. This presentation introduces OpenTelemetry, an emerging industry standard that provides a framework for tracking the performance of not just our Python code, but also other system components like databases and message queues. Its API and SDK integrate seamlessly with Python, enabling a unified approach to gather, process, and export telemetry data from various sources within a distributed system.

We will explore the setup and usage of OpenTelemetry's Python SDK through a practical scenario. The session will demonstrate how to convert an existing Flask microservice to use OpenTelemetry, using both automatic and manual instrumentation. Finally, we will examine how to utilize the exported data for enhanced system monitoring.


With the rise of serverless architectures and cloud technologies, Python has become increasingly popular for building microservices. Yet, as these systems expand, they face observability challenges leading to reduced efficiency and complexities in error tracing.

To address these challenges, this presentation introduces OpenTelemetry, an emerging industry standard providing a framework for tracking the performance of not only our Python code but also other system components such as databases or message queues. It integrates seamlessly into Python environments, offering a common way to gather, process, and export telemetry data from various sources of a distributed system.

The session will begin by revisiting the concept of observability and its critical importance in distributed systems. We will then introduce OpenTelemetry, and check the fundamentals of its' Python SDK. A practical use case will be presented, demonstrating the integration of OpenTelemetry into an existing Python microservice, using both automatic instrumentation mode and manual traces. Finally, we will discuss how to utilize the data collected by OpenTelemetry for system monitoring.


Expected audience expertise: Domain:

Intermediate

Expected audience expertise: Python:

Novice

Abstract as a tweet (X) or toot (Mastodon):

Facing observability challenges in Python microservices? 🐍 ☁️ Learn how #OpenTelemetry can streamline your system monitoring in cloud environments. We're diving into its Python SDK integration, showcasing real-world implementation for enhanced efficiency.

I manage a Python agency with a strong focus on community engagement, evidenced by my involvement in organizing PyCons and leading PyMunich meetups. Being a Python Software Foundation fellow, I am dedicated to a journey of constant learning and sharing.