2026-09-11 –, Room 1 (170)
- Why free-threaded Python matters for native extensions (4 min)
- What Python 3.14 changes
- Why existing C/C++ extensions do not benefit automatically
- Where our embedded engine still hit the GIL (5 min)
- Python UDF calls
- String-heavy columns
- DataFrame integration
- Three changes that unlocked parallelism (12 min)
- Running Python UDFs in-process
- Declaring the module GIL-independent and fixing shared state
- Reworking string hot paths
- What broke and what surprised us (4 min)
- Hidden GIL assumptions
- Stable ABI trade-offs
- Places where no-GIL did not help
- Benchmarks and a migration checklist (5 min)
- Real workloads on 16 cores
- A practical checklist for extension authors
Python 3.14’s free-threaded build makes it possible for threads to run Python code in parallel — but C and C++ extension modules do not benefit automatically. To see real speedups, extension authors need to opt in explicitly, remove hidden GIL assumptions, and audit shared state carefully.
In this talk, I’ll walk through a real migration of a Python extension that embeds a multi-threaded C++ analytical engine. The engine already had parallel execution internally, but several interactions with Python still serialized execution: calling Python UDFs, accessing DataFrame-backed columns, and handling large numbers of strings.
I’ll show three concrete changes that made free-threading pay off in practice: moving Python UDF execution in-process, declaring the extension module GIL-independent and making its global state thread-safe, and selectively replacing high-level Unicode APIs with lower-level CPython string access on hot paths. Along the way, I’ll discuss what broke, what did not scale as expected, and where the stable ABI stopped being compatible with our performance goals.
You’ll leave with a practical checklist for porting extension modules to free-threaded Python 3.14, a clearer picture of when no-GIL actually helps, and benchmark results from real analytical workloads on a 16-core machine. No ClickHouse knowledge is required — the talk is aimed at Python developers who maintain native extensions, build data tooling, or care about making Python scale across cores.
Auxten
- 👨🏻💻 Experience in RecSys, Database
- Technical Director of ClickHouse core team
Principal Engineer in Shopee (ML Platform)
❤️ Love Open Source!
- Contributed to ClickHouse, Jemalloc, K8s, Memcached, CockroachDB, Superset
- Creator of chDB(Acquired), CovenantSQL