2026-11-06 –, Workshops room 03 Language: English
Large Language Models are confidently wrong more often than most users realize, and the problem is worse than previously understood. In this workshop, we build a hallucination detection system in Python that combines four uncertainty signals — self-consistency, semantic entropy, logit calibration, and NLI-based contradiction checking — and introduces "uncertainty fingerprinting," a novel method that classifies hallucinations into distinct types based on their signal pattern.
We go beyond single-model, single-dataset evaluation. Testing across three models (Mistral-7B, Qwen2-7B, Qwen2-1.5B) reveals that smaller models produce 6x more "confident fabrications" — the most dangerous hallucination type where every detector says the answer looks fine but it is wrong. Testing across four datasets (TriviaQA, NaturalQuestions, HotpotQA, MedQA) reveals that on medical questions, the model is wrong 91.5% of the time yet never admits uncertainty — producing zero knowledge gaps and 47.5% confident fabrications.
Our fingerprint-aware adaptive policy improves accuracy from 76.5% to 93.9% on TriviaQA while maintaining 81.5% coverage. A cascading uncertainty approach reduces latency from 63 seconds to 19 seconds per question, making real-time deployment feasible. Attendees will leave with working Python code using PyTorch, HuggingFace Transformers, and Gradio, plus a live demo they can try at huggingface.co/spaces/Shafiya1234/metacognitive-uncertainty-layer.
No prior experience with hallucination detection is required. Familiarity with Python and basic ML concepts is sufficient.
I am an Associate Professor in AI and Robotics at Aivancity based in Paris, France. I got my PhD from the University of Galway in Ireland in Electrical and Electronic Engineering. I then worked at ENS Lyon in collaboration with Inria and Inrae on deep learning for 3D biological image analysis, then joined the Paris Brain Institute with Inria on deep learning for data analysis of Alzheimer's patients, and the Pasteur Institute in Paris on applications of deep learning in the field of drug discovery. My research and teaching interests focus on applications of deep learning in computer vision, computational biology and health, as well as human-machine interactions and intelligent systems.
I am an AI Engineer at INSEAD who enjoys turning messy, real-world data problems into intelligent, working systems. My work blends agentic AI, retrieval-augmented generation, and multimodal machine learning to build practical tools, from enterprise knowledge assistants to predictive and automation pipelines. With a foundation in scientific Python and deep learning, i focus on reproducible, scalable solutions that move from prototype to production. Coming from a physics–math background and now specializing in AI & Data Science, she brings both analytical rigor and creative experimentation to my projects. I am especially interested in human-centered, Responsible AI, scientific computing, and building AI systems that are not just smart, but genuinely useful.