Frank Rust
Frank is deeply passionate about technological advancements and a co-founder of neunzehn innovations, a company specializing in AI solutions. His professional background combines entrepreneurial experience—having established an innovation and strategy consultancy focused on strategy and deep tech—with several years at a major software corporation. Throughout his tenure in the software industry, he contributed to multiple product and service launches, working across various teams to bring new offerings to market. Outside the office, he enjoys discovering new horizons in the camper van.
Session
LLM applications frequently pass tests but fail users in production. This talk examines the gap between evaluation metrics and user experience through three lenses: Expectations (what "working" means to users), Functional (system-level vs. component-level success), and Operational (real-world reliability).
Drawing from production experience, we'll share scenarios of expectation mismatches, silent failures, and undetected drift—plus practical strategies for bridging the gap. The core message: evaluation should answer whether your system serves users, not whether it passes tests.