2025-09-30 –, Gaston Berger
In this talk we focus on installing software (stacks) beyond just the Python ecosystem. In the first part of the talk we give an introduction to using the package manager Spack (https://spack.readthedocs.io). In the second part we explain how we use Spack at our institute to manage the software stack on the local HPC.
All research and data science software relies on existing libraries for various functionalities such as low-level math operations, FFTs, IO, or other domain-specific operations. Installing these dependencies, potentially based on different compilers or in multiple versions, with all inter-dependencies fulfilled is notoriously difficult.
In the first part of this talk we introduce the open-source package manager Spack (https://spack.readthedocs.io), which has a strong focus on HPC and research software. Spack can install software in multiple versions and variants, and supports optimised compilation for the underlying hardware, including compiling on exotic hardware. It comes with a large, community-provided collection of commonly used packages. Spack's Python-based packaging files make it easy to specify required dependencies, provide optional features of a software, and ensure compatibility with other libraries.
In the second part we present the concrete setup at our institute. We use Spack to provide the software stack on the local HPC, including pre-compiled packages and toolchains (sets of compilers and libraries) for users to compile their own software. We report on requirements and challenges, and how we address these with Spack. We also touch on scripting the Spack-based installation process including the option to recreate the HPC software environment on a scientist's laptop, for which we wrote a thin Python wrapper around Spack tailored to our purposes (https://gitlab.gwdg.de/mpsd-cs/mpsd-software-manager-2).
Some basic knowledge of installing software and package managers is helpful but not required. We expect no prior experience with Spack.
Martin Lang is a computational scientist working at the Max Planck Institute for the Structure and Dynamics of Matter, Hamburg, Germany. He has a PhD in physics from the University of Southampton, UK.
Hans Fangohr is a computational scientist and open source advocate. He is heading the scientific support unit Computational Science at the Max Planck Institute for Structure and Dynamics of Matter in Hamburg, Germany, and is Professor of Computational Modelling at the University of Southampton in the United Kingdom. He is working on research software engineering, including high performance computing, data analysis and appropriate software engineering methods in computational science. He has contributed to open source software through tools such as Nmag, Ubermag, Postopus and NBVAL.