Large Language Models (LLMs) are becoming core components of modern digital products. However, their non-deterministic nature means that their behaviour cannot be fully predicted or tested before deployment. This makes observability an essential practice for building and maintaining applications with generative AI features.

This session focuses on observability in LLM-based systems.

We start by motivating why monitoring and understanding your application is key to ensuring quality, reliability, and scalability. We’ll analyze three leading tools for observability in this domain: Langfuse, OpenLIT, and Phoenix. Each has unique strengths and challenges that make them suitable for different use cases.

Through examples and real-world scenarios, we’ll explore:

• How Langfuse provides detailed tracing and quality monitoring through developer-friendly APIs. While it supports multi-step workflows effectively, it lacks support for the OpenTelemetry protocol and can be difficult to customize for non-standard use cases.
• Why OpenLIT, built on OpenTelemetry, offers strong observability for distributed systems. Although it is the least mature of the three tools, it integrates well with established observability stacks and has promising potential for future growth.
• Where Phoenix fits into the process by combining experimentation and debugging capabilities with evaluation pipelines. Its strength lies in development-focused observability, but it has limitations in handling real-time tracing once systems are in production.

This talk will provide a clear, straightforward comparison of these tools, helping you understand which option best fits your LLM applications.

You’ll leave with practical insights into how observability can enhance the reliability and performance of your generative AI systems.