PyData Amsterdam 2026

Topic and Relevance:
While the industry relies on RAG for targeted retrieval, RAG falls apart on tasks requiring dense, cross-document reasoning (e.g., repository-wide codebase migrations, comprehensive financial audits). Furthermore, while 1M-token windows exist, a 10M-token workload is fundamentally out of reach for a single inference call. This talk addresses a critical engineering bottleneck: how do we build agents capable of acting on 10M+ tokens of necessary context using today’s constrained models?

Target Audience & Required Background:

• Roles: AI Engineers, System Architects, Backend Developers.
• Experience Level: Intermediate to Advanced.
• Required Background: Familiarity with LLM limitations, standard RAG pipelines, and general system design.

Takeaways:

• Concrete strategies for handling 10M+ tokens, including hierarchical summarization, MapReduce-style LLM workflows, and multi-agent routing.
• A framework for deciding when to use standard RAG, when to max out a long-context window, and when to deploy complex chunking and aggregation strategies.

Structure:

• Introduction (5 min): The 10M token wall. Why standard RAG fails at holistic reasoning, and why we can't just rely on larger context windows.
• Evaluating the Limits (10 min): Benchmarking degradation. Analyzing recall drop-off, "lost in the middle" phenomena, and latency spikes as context scales up to 1M+ tokens.
• Strategies for 10M+ Tokens (10 min): Architectural tactics to process impossible volumes. Covering hierarchical processing, iterative graph-based retrieval, and agentic data synthesis.
• Lessons Learned and Q&A (5 min): Best practices, cost considerations, and audience questions.