2021-11-09 –, Plenaries
To a newbie in the packaging world, writing recipes could seem quite intimidating. Even people who are not so new would agree that writing package recipes is tiresome, not to say highly errorprone. Example recipes and templates help, but one would rather their package recipe was generated automatically and was perfectlyconcise.
Of course, Anaconda provides Conda Skeleton. Although Conda Skeleton is a helpful tool, it falls short of being the perfect recipe generator for several reasons: it's slow in generating recipes, cannot be deployed on systems without conda, andhas a huge number of dependencies. The recipes itgenerates are also not always concise.
Grayskull solves all these problems.
Grayskull is an automatic conda recipe generator. It generates concise conda recipes for Python packages available on PyPI specially customized for (but not limited to) the conda-forge ecosystem.
Grayskull significantly improves upon existing recipe generators in terms of speed, conciseness of the recipes, packaging environment specificity, and memory usage.
Grayskull has proved to be an extremely useful tool for the packaging ecosystem by generating accurate recipes quickly.
Grayskull, by making it possible to generate conda recipes for PyPI packages, brings PyPI closer to the Conda and reduces fragmentation inthe packaging ecosystem.
After introducing Grayskull, I will talk in detail about some of its inner workings that make it faster and more efficient than existing recipe generators:
-
Grayskull generates recipes taking in consideration the platform, Python version available, selectors, compilers (Fortran, C and C++), package constraints, license type etc.
-
It uses metadata available from multiple sources to create the best recipe possible.
-
In the case of noarch: python, Grayskull is smart enough to detect when the recipe supports it, which is not done by Skeleton. It is important to highlight that Skeleton does not detect compilers either. Whereas Grayskull always tries to detect them.
-
The dependencies of Grayskull are quite reduced when compared to Conda Skeleton. Conda Skeleton relies on conda which is a huge project, therefore Conda Skeleton has a lot of dependencies.
Grayskull is a standalone application which does not rely on conda. It can be easily deployed on systems without conda. It is pip installable. -
Conda Skeleton creates a separate conda environment when it tries to generate the recipe and it takes a lot of time because it also runs the solver.
Grayskull, on the other hand, creates a small and temporary virtual environment to simulate the installation of the package using the source tarball for Python projects. -
Conda Skeleton sometimes mixes some dependencies and generates a quite "fat" recipe. Grayskull does not.
I will demonstrate live how easy it is to generate recipes with Grayskull (really, it’s just a single command: grayskull pypi <package-name> ).
I will also demonstrate how to use the online version of Grayskull.
After the live demonstration, I will talk about the features that Grayskull is presently missing; the addition of which will make Grayskull an even more versatile tool for recipe generation. I will also discuss the work I have been doing on Grayskull via my internship at Quansight Labs under the mentorship of Jaime Rodríguez-Guerra and Vinicius D. Cerutti:
- Presently Grayskull generates recipes for Python packages available on PyPI. I am working on adding more package origins to Grayskull; ability to generate recipes for packages available only as Github, Gitlab repositories, ability to generate recipes for PyProject packages.
-
The next version of Grayskull could have the ability to generate recipes for R packages available on CRAN.
-
And the next could have the ability to generate recipes for C++ packages.
It also has an online version: https://www.marcelotrevisani.com/grayskull
Mahe is a Computer Engineering undergrad student from India. She codes in Python and C++ and is a Packaging fan.
She is an intern at conda-forge, helping writing the conda-forge documentation. She is also an intern at Quansight Labs where she is adding new cool features to Grayskull.
Her hobbies are reading novels and eating tasty food.