Hongsup Shin
Hongsup Shin is a Senior AI & LLM Engineer at NVIDIA's Silicon Co-design Group, building agentic systems and automation for silicon engineering workflows. His work spans production RAG infrastructure, multi-agent systems, learning-to-rank for hardware verification, and human-in-the-loop system design. He has published at IEEE SOCC (2022, 2024) and previously presented at SciPy 2019 on ML applications for hardware failure detection. He currently serves as SciPy Conference Proceedings Chair, founded the Austin ML Journal Club, and has volunteered with the Texas Justice Initiative since 2019, where he authored TJI's first data analytics report on officer-involved shootings in Texas. He holds a Ph.D. in Neuroscience from Baylor College of Medicine.
Session
Automated data enrichment, filling missing fields in structured records from unstructured sources, is the canonical case for pointing an autonomous agent at a database and letting it fill every blank. In high-stakes data that instinct is dangerous. A confidently wrong value is worse than a blank, and retrieval-grounded extraction reduces but does not remove the tendency to assert what the source never stated. An LLM can extract these fields; this paper asks where agentic judgment earns its place and where it becomes a liability.
We study this on the Texas Justice Initiative's police shooting databases, where nearly two thousand records are missing the weapon, the subject's race, or the outcome, whose fields volunteers typically recover by hand, fifteen to thirty minutes each. Our LangGraph pipeline, deterministic in its control flow, searches, validates, extracts, and escalates hard cases to a human. It completes 92% of officer and 70% of civilian records and invents zero facts across twenty fabricated incidents. The recovery itself came from a deterministic prompt fix without any agent. An autonomous agent pointed at the same fabricated incidents, with more freedom, commits a wrong-article fabrication the pipeline escalates.
If the deterministic core does the recovering, the agentic layer earns its place by making those recovered values trustworthy. Agency lives only in this thin judgment layer above extraction, and its components act in one of two ways. One acts on the pipeline's control flow: a relevance judge reads the retrieved articles and, when none actually report this incident, routes the record to a human instead of completing it. The other two pass judgment on what extraction produced: one deletes a value the source never states, and the other explains to the reviewer why the sources disagree on a value. Extraction calls an LLM too, but because it only proposes values for these judges to rule on, we do not count it as agentic. Every judge had to clear a reward-hacking-resistant evaluation gate before it shipped. The main contribution of this paper is a discipline, an "earn-it" protocol, for drawing the line between what a high-stakes pipeline should settle deterministically and where it is worth granting agentic judgment.