2025-10-30 –, Auditorium
Most Python teams already run PostgreSQL—but when it comes to background jobs, they often add Redis, RabbitMQ, or Celery on top. pgqueuer takes a different approach: it turns Postgres itself into a complete task queue.
At its core, pgqueuer uses simple tables to capture jobs and their metadata. Workers safely lease jobs with FOR UPDATE SKIP LOCKED, a Postgres feature that enables concurrent consumers without collisions. Idle workers wake instantly on new jobs via LISTEN/NOTIFY, while a fallback polling loop ensures progress even if notifications are lost.
On top of this foundation, pgqueuer adds the pieces needed in production: exactly-once dequeueing, heartbeats and cancellation for long-running tasks, and retry semantics for failure recovery. A built-in CLI dashboard provides live insight into throughput, latency, and error counts—no extra services required.
This talk walks through pgqueuer’s design and shows how PostgreSQL features can form a reliable, observable job system. Attendees will leave with a understanding of how to run production-ready background jobs directly on the database they already trust.
1 · Queue tables and job flow
We start by outlining the core tables: how jobs are stored, tracked through states, and enriched with metadata like priority and timestamps. This sets the stage for understanding how pgqueuer manages work safely inside Postgres.
2 · Concurrency and coordination
Next, we’ll dive into the mechanics: workers leasing jobs with FOR UPDATE SKIP LOCKED, and why this pattern allows safe parallel processing without global locks. We’ll then look at LISTEN/NOTIFY for instant wake-ups, and why a lightweight polling fallback is still necessary when notifications fail.
3 · Reliability and observability
We’ll explore how retries, heartbeats, and cancellation keep the system robust under load and failures. Finally, we’ll step into the CLI dashboard to see live throughput and error stats—giving operators immediate visibility without extra tooling.
Jan Bjørge (JeyBee) is a Norwegian software engineer with a master’s degree in Cybernetics (University of Stavanger, 2015) and a decade of Python experience. He began as a field telecoms engineer before moving into software, eventually becoming Tech Lead at conversational-AI firm boost.ai.
After a short period as Principal Analyst at Equinor, delivering data-intensive tooling for subsurface modelling, Jan joined Pio in 2024 as Senior Software Engineer. At Pio he works on the backend that powers the company’s automated storage-and-retrieval warehouse platform, exposing APIs that connect e-commerce stores, shipping providers and ERP systems to AutoStore robotics.
Outside work he maintains pgqueuer—a PostgreSQL-native task queue and scheduler created on his own time.