2023-08-17 –, Aula
Python versioning is a critical aspect of maintaining a consistent ecosystem of packages, yet it can be challenging to get right. In this talk, we will explore the difficulties of Python versioning, including the need for upper bounds, and discuss mitigation strategies such as lockfiles in the Python packaging ecosystem (pip, poetry, and conda / mamba). We will also highlight a new community effort to analyze Python libraries dynamically and statically to detect the symbols (or libraries) they are using. By analyzing symbol usage, we can predict when package combinations will start breaking with each other, achieving a high rate of correct predictions. Our goal is to gather more community inputs to create a robust compatibility matrix. Additionally, we are doing similar work in C/C++ using libabigail to address ABI problems.
Python versioning is crucial for ensuring compatibility between different packages, but it can be challenging to get right. In this talk, we will discuss the challenges of Python versioning and present mitigation strategies in different Python packaging systems (lockfiles in poetry and conda-lock, repodata patching in the conda-forge ecosystem). We then introduce a new community effort to analyze Python libraries dynamically and statically to detect the symbols (or libraries) they are using.
By analyzing symbol usage, we can predict when package combinations will start breaking with each other, achieving a high rate of correct predictions. For this we are using the caliper tooling.
Our approach relies on building a compatibility matrix that takes into account the usage of symbols in Python libraries. We will discuss how this matrix can be used to help developers make informed decisions about which package combinations to use to avoid compatibility issues. We will also present some preliminary results showing the effectiveness of our approach.
In addition to Python, we are also working on similar efforts for C/C++ using libabigail to address ABI problems. We believe that this approach will be useful for developers across a wide range of industries and projects, and we are eager to gather more community inputs to further improve the accuracy of our compatibility matrix.
Overall, this talk will be of interest to anyone working with Python or C/C++ who wants to ensure compatibility between different packages and maintain a consistent ecosystem of tools and libraries. We look forward to sharing our work with the SciPy community and hearing your feedback and suggestions.
Predicting Python package compatibility with static analysis - first results!
Category [Scientific Applications]:Other
Expected audience expertise: Domain:some
Expected audience expertise: Python:some
Project Homepage / Git:
Wolf is the CEO of prefix.dev, a company that specializes in cross-platform package management with the open source mamba package manager and more.
He is a core member of the conda-forge project, the RoboStack project and main author of the mamba package manager.