2023-10-27 –, track 2
Being a slow interpreter, Python may drive a system to deliver utmost speed if some guidelines are followed. The key is to treat programming languages as syntactic sugar to the machine code. It expedites the workflow of timing, iterative design, automatic testing, optimization, and realize an HPC system balancing the time to market and quality of code. I will share how to make it happen.
Speed is the king. 10x productive developers change business. So does 10x faster code. Python is 100x slower than C++ but it only matters when you really use Python to implement number-crunching algorithms. We should not do that, and instead go directly with C++ for speed. It calls for strict disciplines of software engineering and code quality, but it should be noted that here the quality is defined by the runtime and the time to market.
The talk will focus on the Python side of the development workflow. It is made possible by confining C++ in architecture defined by the Python code, which realizes most of the software engineering. The room for writing fast C++ code is provided by pybind11 and careful design of typed data objects. The data objects hold memory buffers exposed to Python as numpy ndarrays for direct access for speed.
In a nutshell, the workflow is:
- Understand your problem.
- Prototype with quick and dirty Python code.
- Define data objects. The data objects will be the API between high-level Python and the low-level C++ implementation.
- Complete unit testing as much as possible.
- Design data structures. Reorganize or rewrite the Python code in C++.
For each new request the workflow repeats itself.
In addition to the workflow, the talk will also include the way to use pybind11 and ndarray interface. C++ will be mentioned but outside the scope of discussions.
The objective of the talk is to show how to get started with writing fast code in Python and keep improving it to become a product. Computational scientists and engineers are particularly suitable to the subject. The talk will clearly describe a course to develop the practical coding skills. The speaker has practiced the workflow for academic research, commercial software development, and graduate-level teaching and trained more than 100 students.
Yung-Yu is a numerical analyst and HPC software architect. He has experience in different application domains, including computational fluid dynamics (CFD) and optical proximity correction (OPC). Yung-Yu is teaching software development for engineering at NYCU and an R&D engineer at Synopsys. He has contributed to the Python community for many years and is a PSF fellow.