BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//pretalx.com//pycones-2026//talk//7UBKUN
BEGIN:VTIMEZONE
TZID:Europe/Madrid
BEGIN:STANDARD
DTSTART:20251106T000000
TZNAME:CET
TZOFFSETFROM:+0100
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20260329T030000
RDATE:20270328T030000
TZNAME:CEST
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:20261025T030000
RDATE:20271031T030000
TZNAME:CET
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
SUMMARY:Detecting LLM Hallucinations with Uncertainty Fingerprinting: A Py
 thon Implementation - Anuradha KAR\, Shafiya Kausar
DTSTART;TZID=Europe/Madrid:20261106T160000
DTEND;TZID=Europe/Madrid:20261106T173000
DTSTAMP:20260726T231311Z
UID:pretalx-pycones-2026-7UBKUN@pretalx.com
DESCRIPTION:Large Language Models are confidently wrong more often than mo
 st users realize\, and the problem is worse than previously understood. In
  this workshop\, we build a hallucination detection system in Python that 
 combines four uncertainty signals — self-consistency\, semantic entropy\
 , logit calibration\, and NLI-based contradiction checking — and introdu
 ces "uncertainty fingerprinting\," a novel method that classifies hallucin
 ations into distinct types based on their signal pattern.\n\nWe go beyond 
 single-model\, single-dataset evaluation. Testing across three models (Mis
 tral-7B\, Qwen2-7B\, Qwen2-1.5B) reveals that smaller models produce 6x mo
 re "confident fabrications" — the most dangerous hallucination type wher
 e every detector says the answer looks fine but it is wrong. Testing acros
 s four datasets (TriviaQA\, NaturalQuestions\, HotpotQA\, MedQA) reveals t
 hat on medical questions\, the model is wrong 91.5% of the time yet never 
 admits uncertainty — producing zero knowledge gaps and 47.5% confident f
 abrications.\n\nOur fingerprint-aware adaptive policy improves accuracy fr
 om 76.5% to 93.9% on TriviaQA while maintaining 81.5% coverage. A cascadin
 g uncertainty approach reduces latency from 63 seconds to 19 seconds per q
 uestion\, making real-time deployment feasible. Attendees will leave with 
 working Python code using PyTorch\, HuggingFace Transformers\, and Gradio\
 , plus a live demo they can try at huggingface.co/spaces/Shafiya1234/metac
 ognitive-uncertainty-layer.\n\nNo prior experience with hallucination dete
 ction is required. Familiarity with Python and basic ML concepts is suffic
 ient.
LOCATION:Talleres aula 03
URL:https://pretalx.com/pycones-2026/talk/7UBKUN/
END:VEVENT
END:VCALENDAR