Hitendri Bomble
Hitendri Bomble is a Data Scientist at Red Hat, where she builds Generative AI solutions to solve complex business problems. She specializes in working with Large Language Models (LLMs) to create tools that make everyday work more efficient. Deeply rooted in the open-source community, Hitendri focuses on using the latest AI innovations to automate tasks and bring fresh ideas to her team.
Session
We rely on dashboards to tell us if our RAG system is working. But most standard metrics, Cosine Similarity, BLEU, and even BERTScore, are fundamentally broken for measuring factual correctness. They measure text overlap or semantic drift, not truth.
This means you can have a "90% Accurate" system on paper that hallucinates dangerous misinformation in production. This talk dismantles the current state of RAG evaluation. We will look at why "Golden Datasets" are often contaminated, why "LLM-as-a-Judge" is biased towards its own output, and how to build a robust, adversarial evaluation pipeline that actually catches failures before your users do.