Simon Brugman
Simon has previously presented at SciPy 2022 (popmon) and EuroSciPy 2024 (pycodehash). This year, we'll have a meta talk on "Pure Functions + Separate I/O: Functional Python Pipelines for Reproducible Experiments". Rather than going into the details of the framework itself, which we did at PyData Amsterdam 2025 (ordeq), we will provide our brutally honest learnings from applying this design pattern to actual research and development.
Simon has actively developed various open-source projects, such as pandas-profiling, and been an outside contributor to ruff and uv.
- popmon: https://proceedings.scipy.org/articles/majora-212e5952-01d
- pycodehash: https://pycodehash.github.io/pycodehash/
- Ordeq at PyData Amsterdam: https://cfp.pydata.org/pydata-amsterdam-2025/talk/9WEFB3/
- Ordeq code: https://github.com/ing-bank/ordeq
- pandas-profiling: https://github.com/Data-Centric-AI-Community/fg-data-profiling
Session
Scaling data science pipelines in research and industry poses well-known maintainability challenges (big ball of mud). Research codebases must support rapid iteration as insights evolve, while industry systems must scale amid changing business needs and organizational complexity. Effective projects should remain maintainable without overhauling the entire code base for each change. Ideally, evolving from a notebook experiment to a production-grade application should feel natural, with minimal overhead.
In this talk, we show how data science projects benefit from established software engineering principles, particularly those inspired by functional programming, in Python. The first part of the talk outlines the design principles. The second part, will go into our (brutally honest) insights from applying these in various research projects, spanning from master student experiments to the applications in our R&D teams.