Almost every Python data science workflow depends on PyPI, but few people see the maintenance work behind it. PyPI hosts over 780,000 packages, making it the largest package repository to gather the necessary toolkit to develop everything that happens in the Data Science stack: from data engineering and machine learning to visualization and AI tooling.

Despite its importance for the Python ecosystem, PyPI shares a problem common to many OSS projects. Although it forms part of our critical digital infrastructure, its maintenance depends heavily on the often unseen labour of volunteer communities. The “care” work in OSS maintenance is often overlooked and undervalued: these maintainers do not receive reciprocal contributions for their work, making entire ecosystems vulnerable to collapse if communities can no longer sustain their efforts.

This talk examines what happens when public institutions invest in that infrastructure, and how we might evaluate whether such investments actually help. Between 2023 and 2024, the Sovereign Tech Agency invested over €1 million in PyPI and related parts of the Python cryptographic ecosystem Work was done across the cryptographic ecosystem (PyCA Cryptography, pyOpenSSL, M2Crypto, BoringSSL, the ssl module, and PyPI/Warehouse), covering security improvements, API modernization, legacy deprecation, and alignment with sigstore for supply chain verification.

Two years later, the difficult question is impact: how can we measure the effects of public investment in open source without reducing maintenance to simplistic metrics? This question comes with a lot of caveats: we need to find comparable software projects to find our counterfactual, gather data, control for confounders without flattening the socio-technical realities of maintenance. In this talk, we open up these challenges by introducing an impact evaluation framework for public investment in OSS projects. We construct a goals-to-metrics dictionary to evaluate change over time, and apply a causal inference framework combining Propensity Score Matching with Generalized Synthetic Control Methods to estimate the effects of the funding by comparing treated repositories with a donor pool.

Attendees will gain a practical understanding of how to evaluate the impact of open source funding: how to translate funding goals into measurable indicators, gather relevant OSS data, and use causal methods responsibly while preserving the socio-technical context of maintenance work. We also encourage feedback from maintainers, researchers, and funders on what responsible, community-sensitive evaluation should look like.