2024-11-14 –, Auditorium
Step into the future of Python with Parallel Python: Embracing the Future with Sub-Interpreters and Free Threading. This talk dives into the groundbreaking per-interpreter GIL in Python 3.12 and the enhancements in 3.13, which make parallel execution more accessible. We’ll explore the NoGIL compile option for true parallelism, compare traditional multiprocessing with sub-interpreters and free-threading, and tackle thread safety challenges. The session wraps up with a real-world ASGI web server example, showcasing the power of these new features in action.
We will start by examining the new per-interpreter GIL introduced in Python 3.12, which allows sub-interpreters to run Python code concurrently. We'll explore the enhancements in Python 3.13, which include support for sub-interpreters in the standard library, making parallel execution more accessible. Additionally, we will discuss the revolutionary NoGIL compile option, paving the way for Python threads to run truly in parallel, potentially becoming the default in the future.
The talk will cover the traditional multiprocessing approach, comparing it with the newer methods of sub-interpreters and free-threading. We will assess the pros and cons of each approach, considering their complexities and suitability for various types of problems, whether CPU/IO bound.
Thread safety remains a crucial aspect of parallel programming. We will address common pitfalls such as race conditions and provide practical solutions using synchronization primitives to ensure thread-safe code.
Lastly, we will explore a real-world application combining sub-interpreters and free-threading in a multi-worker, multi-thread ASGI web server, benchmarking its performance against traditional web server architectures.
Machine learning Engineer at Identv.
Model Inference Architecture Design, Large scale Recommendation system.