BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//pretalx.com//pycones-2026//talk//BKUJLJ
BEGIN:VTIMEZONE
TZID:Europe/Madrid
BEGIN:STANDARD
DTSTART:20251108T000000
TZNAME:CET
TZOFFSETFROM:+0100
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20260329T030000
RDATE:20270328T030000
TZNAME:CEST
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:20261025T030000
RDATE:20271031T030000
TZNAME:CET
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
SUMMARY:Surviving LLM Traffic Spikes: Routing\, Rate Limits\, and Failover
  in Python - Sergi Porta
DTSTART;TZID=Europe/Madrid:20261108T121000
DTEND;TZID=Europe/Madrid:20261108T125000
DTSTAMP:20260920T160837Z
UID:pretalx-pycones-2026-BKUJLJ@pretalx.com
DESCRIPTION:Your team ships an AI feature and users love it. Then one vira
 l post turns "normal load" into hundreds of LLM requests per second.\n\nLL
 M calls don't behave like traditional API requests. They're slow (sometime
 s seconds)\, expensive\, rate-limited by providers\, and a single provider
  outage can take your entire feature down. You can't just "add more server
 s." You need a routing layer that knows where to send traffic\, when to ba
 ck off\, and how to fail without taking everything with it.\n\nIn this tal
 k\, we'll walk through the LLM traffic routing architecture we built in Py
 thon at Manychat\, where we serve AI-powered automation to thousands of In
 stagram and messaging accounts. Everything we'll show is running in produc
 tion.\n\nWe'll cover the core gateway patterns for multi-provider LLM traf
 fic\, implemented using LiteLLM Router as a reference design.\n\nBy the en
 d\, you'll walk away with:\n\n- A weighted routing blueprint you can adapt
  to your own provider mix\n- Fallback and cooldown rules designed to survi
 ve real outages\n- Practical rate limiting (requests and tokens) with retr
 y backoff\n- The monitoring baseline (latency\, tokens\, errors by provide
 r\, weight drift) to catch issues before they cascade\n- A checklist for r
 olling this out safely\, incrementally
LOCATION:Track 03 - Manychat
URL:https://pretalx.com/pycones-2026/talk/BKUJLJ/
END:VEVENT
END:VCALENDAR