2025-04-25 –, Europium2
This talk explores the use of logits - the raw confidence scores that language models generate before selecting each token. Working directly with logits enables finer control over model behavior.
The session covers practical techniques for accessing and utilizing these scores through local models. Topics include detecting model uncertainty, implementing custom stopping conditions, and steering generation without prompt modifications.
You will learn how to analyze model confidence patterns and apply this knowledge to real-life use cases.
Logits are the raw numerical scores that language models compute for each token in their vocabulary before making a selection. These scores are typically converted to probabilities and used internally for token selection. Accessing and analyzing them directly opens up possibilities for controlling and understanding model behavior.
Logits provide insights into model uncertainty, help detect potential hallucinations, and enable fine-grained control over generation without modifying prompts.
In this session, attendees will learn how to access logits through local models, visualize confidence patterns, and implement practical techniques like uncertainty detection and generation steering. You can get practical insight which you can apply to your own projects.
Intermediate
Expected audience expertise: Python:Novice
atypical techie. software engineer & researcher building ai/ml tools with keen interest in edtech. co-founder and builder of Quipu.
also working as a part-time engineer at MICE Portal, where he supports transformation of the company processes with agentic ai-backed approaches.