2025-04-23 –, Europium2
This talk explores logits - the raw confidence scores that language models generate before selecting each token. Understanding and manipulating these scores gives you practical control over how models generate text.
In this introductory session, we'll explore the token-by-token generation process, examining how tokenizers work and why vocabulary matters. You'll learn about the relationship between logits, probabilities, and tokens. Then we will cover constrained decoding approaches and talk about structured generation.
Logits are the raw numerical scores that language models compute for each token in their vocabulary before making a selection. These scores are converted to probabilities and used internally for token selection. Accessing and analyzing them directly opens up possibilities for controlling and understanding model behavior.
We'll cover common sampling techniques like temperature adjustment, top-k, and top-p filtering, and beam search.
Then we will see how logits can be used to evaluate model uncertainty, causing hallucinations.
And we will talk about structured generation to use language models in deterministic projects. We will see how the logit values can be used to guide the generation process. Lastly we will explore the libraries like outlines and guidance by showcasing some example snippets about how to use them.
If "token by token" is your only answer when someone asks how LLMs generate text, come join us and let's dig deeper together!
Intermediate
Expected audience expertise: Python:Novice
Public link to supporting material, e.g. videos, Github, etc.:
techie, software engineer & researcher building ai/ml tools with keen interest in edtech. co-founder and builder of Quipu.
also working as a part-time engineer at MICE Portal, where he supports transformation of the company processes with agentic ai-backed approaches.