2025-10-31 –, Tutorial Room
Most AI agent tutorials end with a working demo, but production systems require fundamentally different design patterns. This hands-on workshop teaches the 4Ds framework for building AI agents that actually scale: Distributed, mostly Deterministic, Durable, and Debuggable.
You'll start by running a typical "demo-ware" document classifier that breaks under real-world conditions—rate limits, network failures, memory constraints, and concurrent requests. Then we'll systematically rebuild it using production patterns.
Distributed: Decompose AI workflows into parallelizable units using Hatchet's task orchestration. Transform sequential document processing into concurrent operations that scale horizontally.
mostly Deterministic: Constrain LLM outputs using Pydantic schemas and structured prompts. Build predictable fallback mechanisms for low-confidence results, turning non-deterministic AI into reliable system components.
Durable: Implement automatic retry logic and stateful checkpoints that survive failures without losing progress. Your workflows recover gracefully from API timeouts, service restarts, and infrastructure issues.
Debuggable: Add comprehensive observability through structured logging, distributed tracing, and real-time metrics. Watch your AI agents execute with full visibility into performance bottlenecks and failure patterns.
Each principle builds on practical Python code you'll run locally against cloud infrastructure. By the workshop's end, you'll have a complete production-ready document classification system processing hundreds of documents in parallel, with automatic recovery and full observability.
This is an infrastructure-focused training for Python developers who need to deploy AI agents that actually work in production. You'll leave with patterns immediately applicable to context/RAG systems, document processing pipelines, and multi-step AI workflows.
Prerequisites: Python 3.11+, basic asyncio knowledge, laptop with internet access.
Setup (10 min): Install Hatchet SDK, configure local environment against hosted infrastructure, verify connectivity.
The Problem (15 min): Run a "typical tutorial" document classifier that works until it doesn't. Experience rate limiting, network failures, and resource contention. Discuss why demo-ware fails: no parallelization, no error recovery, no observability.
Distributed Processing (15 min): Transform sequential processing into parallel task execution. Write Hatchet workflows that fan-out document classification across workers, then fan-in results. Process 100 documents in parallel, observing throughput improvements.
Durable Execution (15 min): Add automatic retries, timeouts, and stateful checkpoints. Simulate infrastructure failures and watch workflows resume from the last successful step without data loss. Experience how durability prevents expensive recomputation.
Deterministic Behavior (15 min): Replace unpredictable LLM outputs with structured schemas using Pydantic validation. Implement confidence thresholds and automatic fallbacks. See how constraints reduce debugging time and improve reliability.
Debuggable Systems (15 min): Instrument workflows with structured logging, metrics, and distributed tracing. Use Hatchet's dashboard to identify bottlenecks, track success rates, and diagnose failures across execution graphs.
Production Integration (5 min): Review complete example integrating all patterns. Discuss deployment strategies and monitoring best practices.
Target Audience: Python developers building AI applications who need reliable, scalable infrastructure patterns. Especially valuable for teams transitioning from prototypes to production.
Interactive Elements: Live coding with dashboard feedback, simulated failure scenarios, real-time parallel execution metrics, complete working system to take home.
Takeaways: Production-ready code patterns, infrastructure best practices, and immediately applicable solutions for AI workflow scaling challenges every Python developer eventually faces.
