SciPy 2026

Retrieval is the foundation of modern LLM applications in science, engineering, and industry. However, most RAG implementations rely on naive chunking and basic vector similarity search, leading to brittle systems, hallucinations, and poor performance on structured and multimodal data.

This tutorial provides a structured, engineering-focused approach to optimizing retrieval pipelines using a practical framework, a modular Python toolkit for experimentation, benchmarking, and evaluation.

Participants will iteratively build a RAG pipeline and improve it across three stages:

1. Pre-Retrieval Optimization - Preparing data and queries correctly
2. Mid-Retrieval Optimization - Improving search quality and diversity
3. Post-Retrieval Optimization - Filtering, refining, compressing, and assembling context before generation

We will also cover structured and multimodal parsing for RAG with Docling, including:

• Typed text, table, and image chunks from PDFs
• Hybrid Docling chunking alongside baseline, recursive, parent-child, and contextual strategies
• Multimodal-aware metadata for richer retrieval (without separate SQL or ad-hoc query pipelines)

The tutorial is designed for active coding, experimentation, and measurable benchmarking. More than 70% of the session is hands-on coding in Jupyter notebooks. Attendees will implement techniques step-by-step and evaluate performance improvements live.

Detailed Outline (4 Hours Total)

Part 1: Foundations - Lecture + Guided Setup (40 minutes)

• Introduction to Retrieval in RAG Systems
• Why retrieval fails in real-world systems
• The three-stage optimization framework
• Overview of the Retrieval Playground toolkit and notebook flow (1A → 5)
• Evaluation overview: retrieval, generation, and tool/agent metrics (RAGAS + custom)
• Dataset introduction

Part 2: Pre-Retrieval Optimization - Hands-On Notebook (50 minutes)

1. Document Chunking
   - Recursive chunking
   - Contextual
   - Parent-child
   - Docling-based structured and multimodal chunking (text, tables, images)

2. Query Enhancement

• Query expansion
• Multi-query / RAG Fusion
• Query decomposition
• Query rewriting
• Step-back prompting
• Complexity classification and auto-orchestration

1. Semantic routing

Part 3: Mid-Retrieval Optimization - Hands-On Notebook (60 minutes)

• Dense Search
• Hybrid Search
• Reranking
• Parent-Child Retrieval
• Multi-Query Hybrid
• Route-Driven Retrieval
• Adaptive Retrieval

Interval: 15 minutes

Part 4: Post-Retrieval, Evaluation & Agentic RAG - Hands-On Notebook (60 minutes)

1. Post-Retrieval Context Preparation
   - Retrieval grading (relevant / irrelevant / ambiguous)
   - Knowledge refinement (sentence- or passage-level tightening)
   - Context compression (extractive and abstractive)
   - Document assembly (stuff chain for final generation)

2. Systematic Evaluation

• Classical retrieval checks (hit rate@k, MRR, keyword overlap)
• RAGAS context precision/recall, faithfulness and answer accuracy
• Tool-selection metrics from routing and agent traces
• Baseline vs post-retrieval A/B comparison and pipeline scorecard

1. Agentic RAG Capstone (intro + demo)

• LangGraph ReAct agent with a retrieval tool backed by the workshop RAG stack
• Prompt-based routing (direct answers vs retrieval)
• Lightweight tool-selection evaluation

Final 15 Minutes: Wrap-Up, Future Directions and Q&A

• Best practices and limitations
• Production considerations and scaling strategies
• Open discussion and troubleshooting

Expected Level: Beginner to intermediate.

Target Audience: ML engineers, data scientists, developers working with LLMs in production, and anyone looking to learn how to build robust AI workflows using open source tools.