SciPy 2026

LLMs are powerful, but their knowledge is frozen — they can't access your private documents or recent information. When asked about topics outside their training data, they either refuse to answer or hallucinate confident-sounding responses. Retrieval-Augmented Generation (RAG) solves this by searching relevant documents and including them in the prompt, grounding responses in real information. While long context windows (100K+ tokens) might seem to make retrieval unnecessary, research on "lost in the middle" effects shows that LLMs lose track of information buried in long prompts. RAG provides precision: the model sees a handful of relevant paragraphs instead of hundreds of irrelevant pages. But building a good retrieval system involves many steps: reading diverse file formats, chunking text at sensible boundaries, computing embeddings, and combining search strategies. Each step has pitfalls — HTML-to-text conversion is messy, naive fixed-size chunking splits code blocks and paragraphs in half, and pure vector search misses exact keyword matches. This talk introduces raghilda, an open-source Python framework that handles the full retrieval pipeline with sensible defaults while keeping every step exposed and replaceable. We'll build a RAG system from scratch, walking through each stage of the pipeline:

Ingestion: turning raw documents into a searchable store. We'll cover how to read diverse sources (URLs, PDFs, DOCX files) and convert them to a common format, how to crawl websites to discover pages automatically, how to chunk text at semantic boundaries (headings, paragraphs, sentences) rather than at arbitrary character offsets, why preserving heading hierarchy as context metadata matters for retrieval quality, and how embeddings are computed and stored alongside the text.

Retrieval: finding the right chunks given a query. We'll explore why pure vector similarity search isn't enough, how BM25 keyword matching complements semantic search, how attribute filters let you scope queries by metadata (source URL, document type, custom fields), and how these strategies combine into hybrid retrieval.

Integration: connecting retrieval to an LLM and measuring how well it works. We'll show how to register a search function as a tool that the LLM calls when it needs information, and demonstrate the difference in answer quality between an augmented and unaugmented model on domain-specific questions. We'll also discuss how to evaluate a RAG system: both the retrieval component and the end-to-end generation, and how tuning chunking parameters, search strategies, and reranking affects downstream answer quality.

Throughout, we'll use raghilda to implement each step, showing both the high-level one-liner workflow and the lower-level components so attendees understand what's happening at each stage and how to customize it for their own use cases.

• Source code: https://github.com/posit-dev/raghilda
• Documentation: https://posit-dev.github.io/raghilda/