2.0 -//Pentabarf//Schedule//EN PUBLISH Q3FERF@@pretalx.com -Q3FERF Guardians of Science: A Python Tutorial on a RAG-Powered Compliance Plug-In and Ethical AI tools en en 20250818T083000 20250818T100000 013000 Guardians of Science: A Python Tutorial on a RAG-Powered Compliance Plug-In and Ethical AI tools ## Background The growing integration of AI in research and everyday applications brings significant challenges related to ethics, compliance, and reproducibility, which are key pillars for maintaining our trust in AI's capabilities. Compliance challenges in AI applications arise from the need to navigate evolving regulatory standards, ensure fairness and transparency, and maintain data privacy and accountability across diverse and complex systems. Bias related issues , for example bias across demographic groups pose serious risks, while reproducibility issues, such as incomplete experiment documentation or unstable software environments, further erode confidence in AI-driven applications. To tackle these problems, we present a Retrieval-Augmented Generation (RAG) framework that combines vector-based retrieval methods (e.g., FAISS for fast data access) with transformer-based language models (e.g., Mistral) to deliver transparent, standards-compliant recommendations. We also familiarize participants with complementary tools like IBM’s AI Fairness 360. that support bias detection and mitigation through metrics such as disparate impact and reweighing techniques. ## Significance As AI becomes central to research across disciplines, maintaining ethical standards and reproducibility is essential. Compliance with frameworks like the EU Artificial Intelligence Act and FAIR principles helps prevent biased outcomes and irreproducible results that can damage scientific credibility. In fields such as biomedicine, social sciences, and environmental science, biased or unreliable models can have serious consequences. To address these challenges, this tutorial offers hands-on exercises with a Retrieval-Augmented Generation (RAG) compliance plugin and the IBM AI Fairness 360 toolkit. These tools help researchers detect bias, improve transparency, and ensure reproducibility, supporting the creation of trustworthy, accountable AI systems across diverse domains. ## Objectives The tutorial aims to teach participants about the concepts like ethical compliance, RAG capabilities and tools for estimating and mitigating bias in AI workflows. We will have focused demos using the python RAG plug-in and Fairness 360 that will show how these tools solve compliance issues, using a healthcare case study. Participants will gain skills in ethical AI checks and reproducible methods, preparing them to meet global standards in their research. ## Tutorial Breakdown * Introduction (5 min): Describe AI research challenges, like bias and reproducibility, introduce RAG, Fairness360, noting links to standards like the European Union Artificial Intelligence Act and FAIR principles. * Setup and Data Preparation (15 min): Look at setting up a Jupyter environment (Python 3.8+, FAISS, Transformers, Fairness360, MLflow, Docker, Plotly, Gradio) and review a synthetic healthcare dataset (10,000+ records, World Health Organization-aligned, with age, gender, ethnicity, and ICD-10 codes) and a pre-built compliance knowledge base (EU AI Act, FAIR, GDPR). * Demonstration 1: Retrieval-Augmented Generation (35 min): Explore RAG’s system, showing vector-based retrieval (e.g., FAISS searching EU AI Act rules on data use) and LLM generation (e.g., Mistral creating compliance tips). Demonstrate the plug-in’s process for finding standards and making checks in the healthcare case study, like ensuring GDPR-compliant data use for diagnostic models, with MLflow tracking results. * Demonstration 2: Fairness360 (35 min): Study Fairness360’s bias-checking tools, calculating measures like disparate impact and statistical parity for the healthcare diagnostic model (e.g., comparing accuracy for gender and ethnicity groups). Show how to fix bias (e.g., reweighing to balance groups), using Plotly to display bias measures (e.g., bar charts of disparity) and Gradio to interact with audit reports, focusing on fair results. * Q&A and Wrap-Up (5 min): Talk about uses in biomedicine (e.g., fair diagnostics), social sciences (e.g., unbiased data analysis), and environmental science (e.g., repeatable climate models), and answer questions to clarify concepts. ## Outcomes Participants will deeply understand Retrieval-Augmented Generation and Fairness360, and see how they work in a compliance plug-in. They will build skills in ethical AI checks and reproducible methods, ready to use in biomedicine, social sciences, and environmental science, meeting standards like the European Union Artificial Intelligence Act and FAIR principles. PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/Q3FERF/ Room 1.38 (Ground Floor) Anuradha Kar, PhD Anuradha KAR Likhita Yerra PUBLISH 8KK8UC@@pretalx.com -8KK8UC Skrub: machine learning for dataframes en en 20250818T103000 20250818T120000 013000 Skrub: machine learning for dataframes In the tutorial we will teach how to use skrub to easily tackle datasets that would be challenging to analyze using only scikit-learn. In this regards, we show how skrub can be combined with scikit-learn to address some time series forecasting use case. First, we give a short introduction regarding the scope of the skrub library. We show that some tedious tasks around machine learning can be reduce with a couple of out of the shelve functionalities. Then, we focus on the skrub DataOps that allows to combine data wrangling operations, using commonly tools such as pandas or polars, with common machine learning pipeline, using scikit-learn. We focus on time series forecasting. First, we show how to build some common time series preprocessing using polars. Then, we show that we can record the set of such transformation into a graph allowing us to replay the same transformation in the future on a new set of data. Then, we combine this preprocessing stage together with a classic scikit-learn regressor to predict the desired target. Finally, we show how to evaluate this pipeline with cross-validation as well as how to perform hyperparameter search. We conclude with a brief summary of ongoing development and future enhancements to skrub. The material and instructions for the tutorial will be available here: - [static website](http://skrub-data.org/EuroSciPy2025/) - [git repository](https://github.com/skrub-data/EuroSciPy2025) You can find an extended version [here](https://github.com/probabl-ai/forecasting) PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/8KK8UC/ Room 1.38 (Ground Floor) Guillaume Lemaitre Jérôme Dockès Riccardo Cappuzzo PUBLISH PTLGMW@@pretalx.com -PTLGMW Use napari for easier interactive extraction of knowledge from images and other spatial data en en 20250818T133000 20250818T150000 013000 Use napari for easier interactive extraction of knowledge from images and other spatial data As we collect more and more images and other spatial information, we need to improve tooling to be able to process all the data that we gather from microscopes, telescopes, satellites, MRI machines and a myriad of other sensors. For the phase of data exploration, we need a tool that allows smooth comparison of results from various knowledge extraction methods, in order to choose the best method of processing our data. This tutorial is aimed at people who have some experience in scientific computing with Python. To get the most out of it, you should be familiar with NumPy arrays, Jupyter notebooks, and Python scripts. Ideally, you should have some idea of how images can be represented as arrays of numbers, and the types of analyses that might be performed on these arrays, e.g. filtering and segmentation. You don’t necessarily need to be familiar with how these tools and methods work - it’s enough to know that they are out there! This tutorial will be split into three parts: Part 1: Introduction: using napari, exploring the interface, using sample plugins to perform basic data manipulation In this section, we will introduce participants to the napari interface and show how to perform basic operations. It will also ensure that participants have properly created the virtual environment. Part 2: Extending napari with custom functionalities: custom widgets and mouse callbacks, from a Python script or Jupyter notebook. In this section, we will teach: How to allow trigger custom functions from napari GUI. How to create a simple widget from a function and how to prepare a more complex widget to fit better into workflow. How to integrate the napari viewer with an interactive session in jupyter. Part 3: How to convert your custom extension of napari into a plugin, Python package, that can be easily shared with other people. In this section, we will show how to pack custom widgets into plugins and share them as Python packages. ## Installation instruction In this tutorial we assume usage of [`uv`](https://docs.astral.sh/uv/) for environment management. If you prefer to use `conda` or `pip`, then you may use our [installation instruction](https://napari.org/stable/tutorials/fundamentals/installation.html#napari-installation) To setup your workspace: 1. Install `uv` (here instruction https://docs.astral.sh/uv/#installation) 2. Clone (`git clone https://github.com/brisvag/napari-tutorial-euroscipy2025.git`) or [download](https://github.com/brisvag/napari-tutorial-euroscipy2025/archive/refs/heads/main.zip) the repository 3. Go into the project directory 4. Execute `uv sync` 5. Check if everything works by executing `uv run napari`. If the application starts your environment is ready! You may activate the environment by typing * [Linux/MacOs] `source .venv/bin/activate` * [Windows] `venv\Scripts\activate.bat` (or `venv\Scripts\Activate.ps1` if using PowerShell) PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/PTLGMW/ Room 1.38 (Ground Floor) Grzegorz Bokota Lorenzo PUBLISH FPYRSB@@pretalx.com -FPYRSB Compress, Compute, and Conquer: Python-Blosc2 for Efficient Data Analysis en en 20250818T153000 20250818T170000 013000 Compress, Compute, and Conquer: Python-Blosc2 for Efficient Data Analysis Blosc and Blosc2 are well-known and widely used libraries for high-performance data compression. They are particularly effective for compressing large datasets, such as those encountered in data science and high-performance computing. The Blosc library has been around for more than a decade, and its design has always prioritized speed, with a focus on achieving compression and decompression speeds that are close to or even exceed memory bandwidth limits. With the introduction of a new compute engine in Python-Blosc2 3.0, the guiding principle has evolved to "Compress Better, Compute Bigger." This enhancement enables computations on datasets that are over 100 times larger than the available RAM, all while maintaining high performance. In this hands-on tutorial, participants will learn how to effectively use Python-Blosc2 through practical exercises divided into three sections: **Section 1: Getting Started with Python-Blosc2 (20 minutes)** - Introduction to compression concepts and Blosc2 architecture - Setting up your environment and installing Python-Blosc2 - Basic compression/decompression operations with various codecs - Hands-on: Creating your first compressed arrays **Section 2: The Compute Engine (30 minutes)** - Understanding the Blosc2 compute engine architecture - Processing data larger than available RAM - Hands-on: Implementing calculations on out-of-memory datasets **Section 3: Advanced Usage and Remote Access (20 minutes)** - Caterva2: serving Blosc2 files remotely - Hands-on: Solving a complex data analysis challenge Throughout the tutorial, we'll work with practical examples demonstrating how to analyze datasets that exceed available RAM without specialized hardware. By the end, participants will have hands-on experience implementing Python-Blosc2 in data workflows and will understand how to compress data while maintaining computational efficiency. This tutorial will help you expand your capabilities for scientific computing and data analysis while reducing memory footprint and improving processing speed. Attendees should bring laptops with Python installed. Tutorial setup instructions are accessible at: https://github.com/Blosc/EuroSciPy2025-CCC-Tutorial PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/FPYRSB/ Room 1.38 (Ground Floor) Francesc Alted Luke Shaw PUBLISH URGTCT@@pretalx.com -URGTCT Introduction to Python and JupyterLab en en 20250818T083000 20250818T100000 013000 Introduction to Python and JupyterLab Please install JupyterLab **before** the tutorial. Depending on the way you install Python packages you chose on of these options: pip install jupyterlab or conda install jupyterlab or mamba install jupyterlab or pixi add jupyterlab or uv add jupyterlab or Your own way. I use [Pixi](https://pixi.sh/latest/). If you like to try it, please [install](https://pixi.sh/latest/#installation) and create an environment: pixi init jupyterlab_tutorial cd jupyterlab_tutorial pixi add jupyterlab python=3.13 pixi shell No matter what installation method you use, you can start JupyterLab with: jupyter-lab or python -m jupyter-lab PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/URGTCT/ Room 1.20 (Ground Floor, Shannon) Mike Müller PUBLISH UWMH7B@@pretalx.com -UWMH7B Introduction to NumPy and DataFrames en en 20250818T103000 20250818T120000 013000 Introduction to NumPy and DataFrames ## **Title: Introduction to NumPy and DataFrames (Pandas & Polars)** This tutorial is targeted for beginners with basic Python knowledge and will give an understand of the basics of NumPy arrays and DataFrames, as well as perform simple data analysis tasks. ### Welcome and Setup (~ 10 min) * Quick introduction to the topic and objectives * Ensure environments are setup * Overview of what NumPy and DataFrames are used for ### Introduction to NumPy (~25 min) * What is NumPy and why use it? * Creating arrays: * `np.array`, `np.zeros`, `np.ones`, `np.arange`, `np.linspace` * Array shapes and reshaping: `.shape`, `.reshape()` * Indexing and slicing * Vectorized operations vs Python loops (brief performance motivation) * Basic operations: * Arithmetic, broadcasting, `.mean()`, `.sum()`, `.axis` * Hands-on exercises: * Create a 2D array and compute row-wise and column-wise means * Element-wise multiplication of arrays ### Introduction to Pandas DataFrames (~25 min) * What is a DataFrame? * Creating a DataFrame (from dicts, CSV, etc.) * Exploring data: * `.head()`, `.info()`, `.describe()` * Accessing columns and rows: `df['col']`, `.loc`, `.iloc` * Filtering and boolean indexing * Common operations: * Sorting (`.sort_values()`), grouping (`.groupby()`), aggregation * Handling missing values: `.isna()`, `.fillna()`, `.dropna()` * Simple data visualization with `.plot()` (optional if time) * Hands-on exercises: * Load a small CSV * Filter rows by condition * Group by a column and compute summary stats ### Polars (~25 min) * Why Polars? Performance and parallelism * Quick comparison with Pandas (syntax similarities/differences) * Lazy vs eager evaluation * Basic usage: * `pl.read_csv`, `df.select`, `df.filter`, `df.groupby` * Hands-on mini demo (load and filter data) ### Recap, Tips & Q\&A (~ 5 min) * Summary of key concepts * When to use what (NumPy vs Pandas vs Polars) * Tips for continued learning * Q\&A PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/UWMH7B/ Room 1.20 (Ground Floor, Shannon) Umut Nefta Kanilmaz PUBLISH LMDYUQ@@pretalx.com -LMDYUQ Introduction to Scikit-learn en en 20250818T133000 20250818T150000 013000 Introduction to Scikit-learn This beginner tutorial will include: 1. Brief introduction to machine learning 2. Why Python? 3. Where to get the data? 4. Introduction to scikit-learn - basic machine learning algorithms 5. Hands-on tutorial on how to create your own machine learning model 6. Instructions on how and why to validate your machine learning model 7. Where, when, and why use machine learning? All the code will be provided to you in a GitHub repository. There will also be time prepared for additional questions regarding the topic. PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/LMDYUQ/ Room 1.20 (Ground Floor, Shannon) Justyna Szydłowska-Samsel PUBLISH GYBLDL@@pretalx.com -GYBLDL Beyond the Basics: Data Visualization in Python en en 20250818T153000 20250818T170000 013000 Beyond the Basics: Data Visualization in Python #### Section 1: Getting Started With Matplotlib While there are many plotting libraries to choose from, the prolific Matplotlib library is always a great place to start. Since various Python data science libraries utilize Matplotlib under the hood, familiarity with Matplotlib itself gives you the flexibility to fine tune the resulting visualizations (e.g., add annotations, animate, etc.). Moving beyond the default options, we will explore how to customize various aspects of our visualizations. Afterward, you will be able to generate plots using the Matplotlib API directly, as well as customize the plots that other libraries create for you. #### Section 2: Moving Beyond Static Visualizations While static visualizations are limited in how much information they can show, animations make it possible for our visualizations to tell a story through movement of the plot components (e.g., bars, points, lines), which can encode another dimension of the data. In this section, we will focus on creating animated visualizations before moving on to create interactive visualizations in the next section. #### Section 3: Building Interactive Visualizations for Data Exploration When exploring our data, interactive visualizations can provide the most value. Without having to create multiple iterations of the same plot, we can use mouse actions (e.g., click, hover, zoom, etc.) to explore different aspects and subsets of the data. In this section, we will learn how to use HoloViz to create interactive visualizations for exploring our data utilizing the Bokeh backend. ## Setup instructions Please set up your laptop beforehand by following these [setup instructions](https://github.com/stefmolin/python-data-viz-workshop?tab=readme-ov-file#setup-instructions). PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/GYBLDL/ Room 1.20 (Ground Floor, Shannon) Stefanie Molin PUBLISH ASR3XL@@pretalx.com -ASR3XL Predictive modeling for imbalanced classification using scikit-learn en en 20250819T083000 20250819T100000 013000 Predictive modeling for imbalanced classification using scikit-learn Real-world applications utilize machine learning (or artificial intelligence) to assist in decision-making or planning. In this context, probabilistic models represent the standard approach for data scientists to link input data with probabilistic predictions for possible outcomes, which serve as the foundation for subsequent operational decisions or actions. An additional challenge encountered in real-world applications is that the outcome of interest to be predicted often occurs rarely compared to the total number of observations. This scenario is commonly referred to as an "imbalanced" dataset. Several examples illustrate such applications: (i) medical screening where detecting a specific disease represents a rare outcome compared to the general population, (ii) fraud detection where fraudulent events constitute a small fraction of total transactions, and (iii) detection of physical phenomena such as the Higgs boson where observations confirming its existence represent rare events compared to all observations. This tutorial addresses this particular problem and examines specific methodological considerations when approaching imbalanced datasets. One important consideration relates to resampling and its various effects: with numerous negative outcomes compared to positive outcomes, existing literature supports reducing the ratio between these event types. With these techniques in mind, we examine and study the following aspects: (i) the impact of imbalanced datasets on probability outcomes and classifier calibration; (ii) the effect of resampling on model overfitting and underfitting and its relationship to model regularization; and (iii) the impact of resampling on computational performance versus statistical performance. This tutorial provides a hands-on approach through programmatic notebooks to offer insights into each of these concepts. The material and instructions to follow the tutorial is available here: [https://github.com/probabl-ai/calibration-cost-sensitive-learning](https://github.com/probabl-ai/calibration-cost-sensitive-learning) PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/ASR3XL/ Room 1.38 (Ground Floor) Guillaume Lemaitre Olivier Grisel PUBLISH D78ZJP@@pretalx.com -D78ZJP Using Cython and C++ kernels to speed up Python libraries en en 20250819T103000 20250819T120000 013000 Using Cython and C++ kernels to speed up Python libraries The main goal of this tutorial is to give a broader audience an overview of how high-performance libraries in Python are developed. Attendees will work with ready-made code templates and fill in missing parts to learn by doing. ### **Session Outline** - **(5 min) Motivation:** Problems with performance in Python and what to do with them - **(10 min) Understanding Linear Regression & Normal Equations:** A brief introduction to the mathematical background, deriving the formulas that would be used for training - **(10 min) Implementing a Python Baseline:** Writing a reference implementation on Python - **(20 min) Optimizing with Cython:** Implementing a fast Cython-based kernel - **(20 min) Example of pybind11 use:** Example of pybind11 usage for integrating kernels implemented in C++ - **(15 min) Performance Comparison:** Benchmarking Python vs. Cython vs. C++ & pybind11 implementations - **(10 min) Q&A and Discussion** By the end of this session, attendees will: - Understand how Python libraries achieve high performance using Cython and C++ - Learn how to write and integrate custom Cython or C++ kernels - Know when to optimize Python code and when to rely on existing libraries We also hope that this tutorial will inspire more people to contribute to open-source high-performance libraries such as scikit-learn-intelex https://github.com/uxlfoundation/scikit-learn-intelex PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/D78ZJP/ Room 1.38 (Ground Floor) Anatoly Volkov David Cortes PUBLISH CBPMEK@@pretalx.com -CBPMEK GPU Python for the Real World: Practical GPU-Accelerated Python with RAPIDS en en 20250819T133000 20250819T150000 013000 GPU Python for the Real World: Practical GPU-Accelerated Python with RAPIDS In this tutorial we will cover: - Introduction to cuDF, cuML and more that showcases a simple example of data processing and model training on GPUs. - Answers to questions like: “Where do I get a GPU?”, “How do I run a container on a VM with a GPU?”, “How do I install GPU packages into an existing environment?”, as well as follow along examples to get a GPU up and running. - Troubleshooting and monitoring: Examples of performance analysis, diagnostics, and debugging. This is a hands-on tutorial, with multiple examples to get familiarized with the RAPIDS ecosystem. Participants should ideally have some experience using Python, pandas and sci-kit learn. We'll use cloud-based VMs, so familiarity with the cloud and resource creation is helpful but not required. No prior GPU knowledge is needed. PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/CBPMEK/ Room 1.38 (Ground Floor) Jacob Tomlinson PUBLISH MU9HAJ@@pretalx.com -MU9HAJ Beyond Likelihoods: Bayesian Parameter Inference for Black-Box Simulators with sbi en en 20250819T153000 20250819T170000 013000 Beyond Likelihoods: Bayesian Parameter Inference for Black-Box Simulators with sbi Material for preparation is now available at: [https://github.com/janfb/euroscipy-2025-sbi-tutorial](https://github.com/janfb/euroscipy-2025-sbi-tutorial) Many scientific and engineering fields rely on complex computer simulations – for example, in particle physics, epidemiology, or computational neuroscience – to understand complex phenomena. A common challenge is finding the right input parameter settings for these simulators so that their output matches real-world observations. Determining these parameters accurately can be difficult, especially when the simulator is intricate or stochastic. Traditional methods like grid search or numerical optimization algorithms can find a single 'best' set of parameters. However, they often struggle when there are many parameters, and more importantly, they usually don't quantify the uncertainty associated with the result. Is this the only good parameter set? How much could the parameters change and still produce similar results? Answering these questions is vital for robust scientific understanding. Simulation-Based Inference (SBI) is a modern approach, drawing on machine learning, designed specifically for this problem. SBI methods learn a statistical relationship directly from running your simulator multiple times with different inputs. Their key advantage is the ability to estimate the range of parameter values (and their probabilities) that are consistent with your observed data, providing a measure of uncertainty. This works even for complex 'black-box' simulators where the internal equations might be unknown or intractable. For instance, when modeling disease spread like COVID-19, knowing the uncertainty around estimated infection rates is crucial for making informed decisions – SBI provides exactly this, but for any kind of simulator as long as we can simulate enough data. This tutorial provides a comprehensive, practical introduction to SBI using the `sbi` [Python package](https://sbi.readthedocs.io/en/latest/). `sbi` implements state-of-the-art SBI algorithms, often using neural networks, and is actively developed by a large community (it's a NumFOCUS affiliated project with over 70 contributors and yearly collaborative hackathons). We will guide you through the entire practical workflow: - A-prior checks: Defining parameter ranges of interest and running initial simulations to check if the model can plausibly generate data similar to observations. - Choosing and applying SBI methods: Learning about different SBI strategies (like those that directly estimate parameter probabilities, approximate likelihoods, or work with probability ratios) and selecting one suitable for your specific problem. - A-posteriori checks: Evaluating how accurately the underlying machine learning models have learned to infer parameters. - Interpretation: understanding and drawing scientific conclusions from SBI results. The tutorial combines accessible explanations of the concepts with hands-on coding exercises using `sbi`, enabling you to apply these techniques to your own research problems. **Target Audience:** This tutorial is aimed at individuals comfortable with Python programming who work with computational simulation models in science or engineering and need to estimate parameters from data. Researchers and practitioners looking for practical methods to quantify parameter uncertainty in complex systems will find this useful. **Prerequisites:** - Proficiency in Python programming. - No prior knowledge of Bayesian statistics is required. **Learning Objectives:** Upon completion, participants will be able to: - Understand the concept of Simulation-Based Inference (SBI) and its advantages for parameter estimation in complex simulators. - Apply the `sbi` Python package to estimate parameter ranges and uncertainty from simulation outputs. - Become familiar with common neural network-based SBI techniques and their use cases. - Evaluate the results of an SBI analysis to ensure reliability. - Perform the complete SBI workflow using the `sbi` package on their own problems. This tutorial will equip participants with the practical skills to effectively use the sbi package for more reliable parameter estimation and uncertainty quantification in their simulation-based models. PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/MU9HAJ/ Room 1.38 (Ground Floor) Jan Boelts (Teusen) Maternus Herold PUBLISH DASHT7@@pretalx.com -DASHT7 Annotating the dynamic: Type Annotation for DataFrames en en 20250819T083000 20250819T100000 013000 Annotating the dynamic: Type Annotation for DataFrames The introduction of type annotation in Python has sparked debates, but it is now widely accepted as a best practice in modern development. In Python, type annotation is used primarily during development and is typically ignored during runtime. Development tools use type annotation to validate code. Type annotation offers numerous benefits, such as enhancing suggestions provided by Integrated Development Environments (IDEs), improving the maintainability of existing code bases, and enabling features like dependency injection. While type annotation has significantly improved the readability and structure of general application code, its applicability to DataFrames—a fundamental component in data science—has yet to be fully realized. The dynamic and runtime-defined nature of DataFrames contrasts the development-time nature of type annotation. As a DataFrame schema is often only known at runtime, e.g., after reading an input file, utilizing type annotations to enhance schema validation and code readability presents a challenge. The tutorial is intended for hands-on Python enthusiasts who work with DataFrames. The tutorial introduces type annotation and presents its advantages regarding readability, maintainability, tooling, and static code analysis. The tutorial explores libraries and tools to leverage the advantages of type annotation at development-time and libraries to enforce runtime validation. The tutorial dives into the benefits of type annotations of DataFrames and highlights the limitations of type annotations specific to the dynamic nature of DataFrames. Therefore, the tutorial will present best practices for leveraging type annotations with DataFrames. PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/DASHT7/ Room 1.20 (Ground Floor, Shannon) Frank Sauerburger PUBLISH Z83QKH@@pretalx.com -Z83QKH Deploy your Machine Learning model with Fast API en en 20250819T103000 20250819T120000 013000 Deploy your Machine Learning model with Fast API In this workshop, we will go deeper into how to prototype a machine-learning project with Fast API. Fast API allows the creation API server with very little effort, it is easy to deploy a pre-trained model, but for models that require re-training, the challenge of when and how to retrain a model and update for a service in use becomes complicated. We will cover the aspect of delivering a pre-trained model and the design of re-training the model. This workshop will also provide suggestions for deploying the machine learning project so it can migrate from a prototype to a functional service in production. ## Goal The workshop aims to equip a data science team capability to convert their machine learning project into a prototype service using Fast API, at the end of the workshop, they will not just be able to deliver API calls to a pre-trained model, but they will also be able to design when to re-train and update the model and be ready to migrate the prototype into production. ## Target audience Data scientists who have little or no experience using Fast API or putting a machine learning model into production. This workshop will assume the audience already knows how to build and train a basic machine learning model (e.g. using Sci-kit learn). ## Outline Part 1 - Introduction to Fast APi and prediction on demand - Understand the basics of Fast API - Using a pre-trained model for prediction with API calls - Validating the query parameters Part 2 - Re-train and update models - Problem with updating model: Race conditions - Scheduled re-training - Re-training on demand with Fast API Part 3 - Machine learning model in production - Fast API in docker containers - Fast API on the cloud PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/Z83QKH/ Room 1.20 (Ground Floor, Shannon) Cheuk Ting Ho PUBLISH CHDNML@@pretalx.com -CHDNML How to become a software detective and perform security research en en 20250819T133000 20250819T150000 013000 How to become a software detective and perform security research Participants of this tutorial will gain a solid foundation in software analysis, with a strong emphasis on security. We will explore the significance of security research in software development and consider various resources and tools to discover vulnerabilities. To illustrate these concepts, we’ll perform static analysis with Bandit on a vulnerable Python library as a case study. Additionally, we’ll understand different approaches and techniques to security-oriented analysis. Participants will gain essential knowledge to identify vulnerabilities, find potential targets for analysis, and apply research methodology. This tutorial will cover * Introduction to security research * Automated software analysis - SAST vs DAST * Research methodologies and resources * Basics of static code analysis * Practical examples using vulnerable software to test acquired skills Key takeaways * Basic concepts related to vulnerability research * Software analysis fundamentals * Security analysis tools PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/CHDNML/ Room 1.20 (Ground Floor, Shannon) Przemek PUBLISH HUTEDK@@pretalx.com -HUTEDK Managing Scientific Data and Workflows with DataLad en en 20250819T153000 20250819T170000 013000 Managing Scientific Data and Workflows with DataLad ## Installation To set up your environment, follow the description on the [tutorial website](https://olebialas.github.io/DataLad-EuroScipy25/) ## Description The tutorial will begin with a short introduction to DataLad, describe typical use-cases and explain how DataLad uses the version control systems git and git-annex to manage, track and transport data. After this short introduction, we will start with the first hands-on block where participants will explore the core concepts of DataLad that enable reproducible research, such as version control and digital provenance. We will learn how to create and configure a DataLad data set (datalad create), how to add and modify data (datalad [status, save, unlock]) and how to see changes in the data set's history (git log). We will see how DataLad can be used to run a Python script while keeping track of the input and output of that script (datalad run). The record of running the script that DataLad produced can then be used to conveniently re-run parts of the analysis pipeline after making changes to the script (datalad rerun). After this hand-on session, participants will understand the basic DataLad functionalities required to manage their data analysis project on their local machines. Next, we will present how DataLad can be used to install existing datasets and collaborate with others, how it supports the FAIR (Findable, Accessible, Interoperable, and Reusable) principles for data sharing and discover and how it integrates with open-science platforms like the Open Science Framework (OSF). This is followed by another hand-on session where participants will learn how to install existing online datasets (datalad [clone, get]) and check the identity and availability of files (git-annex [info, whereis]). We will also explore how DataLad can create and manage siblings of a data set, allowing the user to back up and share their data (datalad [create-sibling-*, push, update]). After this session, participants will have the tools for working with existing data sets provided by open-science platforms and for creating collaborative workflows. Participants are expected to bring their own computer. Before the tutorial, we will provide installation instructions and be available for potential troubleshooting to ensure that every participant is able to follow the exercises. Prior knowledge of Git or Git-annex is not required, but participants who are familiar with these tools may get a deeper understanding of the inner workings of DataLad. Familiarity with a Unix-like terminal is also an advantage. PUBLIC CONFIRMED Tutorial https://pretalx.com/euroscipy-2025/talk/HUTEDK/ Room 1.20 (Ground Floor, Shannon) Ole Bialas Michał Szczepanik PUBLISH BTXXNU@@pretalx.com -BTXXNU Women in HPC – Breaking Barriers and Shaping the Future en en 20250820T090000 20250820T100000 010000 Women in HPC – Breaking Barriers and Shaping the Future Why aren’t there more women in the High-Performance Computing (HPC) community? This simple question led to the creation of the international organisation Women in High Performance Computing (WHPC). The members of this network are committed to greater equality, diversity and integration in the HPC community. The initiative is active at major HPC conferences, offers workshops and mentoring programmes, and aims to raise awareness in the HPC community with the slogan “Diversity creates a stronger community”. Three years ago a group at Jülich Computing Centre decided that it is time to establish a local group of WHPC – Jülich Women in HPC (JuWinHPC) – to strengthen the community of women in HPC at Forschungszentrum Jülich and to promote diversity. This talk presents the activities of JuWinHPC, from casual lunch meetings to the organisation of conference sessions, and summarises experiences gained and lessons learned striving to establish a local network of women in HPC and to increase diversity, inclusion and female visibility within the community. PUBLIC CONFIRMED Keynote https://pretalx.com/euroscipy-2025/talk/BTXXNU/ Room 1.38 (Ground Floor) Anna Lührs PUBLISH X9TUZ8@@pretalx.com -X9TUZ8 A Hitchhiker's Guide to the Array API Standard Ecosystem en en 20250820T103000 20250820T110000 003000 A Hitchhiker's Guide to the Array API Standard Ecosystem Support for alternative array libraries is one of the most common requests put to libraries written on top of NumPy. Users are often interested in boosting their performance by using libraries which can make the most of the hardware available to them, as well as techniques like Just-In-Time compilation. Unfortunately, supporting multiple array libraries is not simple due to API differences, so such support has historically meant heavy duplication of work, with a whole new implementation for each alternative array library. This duplication of work is at best inefficient, and at worst infeasible for maintainers of many libraries. The Python array API standard aims to standardise functionality which exists in most array libraries. It specifies an API which 'array-consumer' libraries can use to write 'array-agnostic' code, where the same codebase can support many array libraries at once. In this talk, I give a brief introduction to the standard, before taking you on a tour of the standard's ecosystem, explaining how each library built around the standard fits into the bigger picture. I'll explain what the standard enables for SciPy and scikit-learn, and what else may be needed in the future. After this talk, you'll be well-acquainted with the following libraries from the data-apis org: - array-api - array-api-tests - array-api-compat - array-api-strict - array-api-extra I'll also be mentioning some exciting new tools like MArray. Rough talk outline: - 5 mins — what is the array API standard, and what problem is it addressing? - 10 mins — a tour of the ecosystem: what do all of these libraries do, and how can you use them? - 10 mins — updates from the past year, status of where we are now, what is coming in the future? - Any spare time — try some live coding! Slides: https://lucascolley.github.io/talks/euroscipy-25-array-api/ PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/X9TUZ8/ Room 1.38 (Ground Floor) Lucas Colley PUBLISH L9JPZE@@pretalx.com -L9JPZE Accelerate your scientific Python code with Rust en en 20250820T110500 20250820T113500 003000 Accelerate your scientific Python code with Rust The outline will be roughly as follows: 1. Python is slow, you say? (3 min) 2. Python + Rust = 🤜🤛 (5 min) 3. Gentle introduction to PyO3 (5 min) 4. Passing NumPy arrays in and out with ndarray (5 min) 5. Your first Rust extension (live demo) (5 min) 6. Conclusions (2 min) The demo will showcase some simple numerical algorithms that benefit from having a Rust implementation. Light performance benchmarks will be shown. Familiarity with Python is required. No prior exposure to Rust is needed. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/L9JPZE/ Room 1.38 (Ground Floor) Juan Luis Cano Rodríguez PUBLISH KCYYTF@@pretalx.com -KCYYTF Pyro Meets SBI: Unlocking Hierarchical Bayesian Inference for Complex Simulators en en 20250820T114000 20250820T120000 002000 Pyro Meets SBI: Unlocking Hierarchical Bayesian Inference for Complex Simulators Hierarchical Bayesian inference is a powerful framework for analyzing structured data common in complex experimental settings, like multi-subject decision-making research. Probabilistic programming languages (PPLs) such as [Pyro](https://pyro.ai/) provide excellent tools for defining and inferring these hierarchical models, leveraging features like plate notation for concisely representing repeated structures and managing dependencies. However, a significant challenge arises when the underlying scientific model is a complex simulator with an intractable likelihood function, rendering standard PPL-based inference inapplicable. While Simulation-Based Inference (SBI) techniques can handle such simulators by learning likelihood (or posterior) approximations from simulations, they often lack native support for easily specifying and inferring complex hierarchical dependencies. This talk introduces a novel approach that bridges the SBI package [sbi](https://sbi.readthedocs.io/en/latest/) and `pyro`, enabling effective simulation-based hierarchical Bayesian inference. We demonstrate how likelihood approximations learned via `sbi` can be seamlessly integrated as custom components within `pyro` models. This synergistic approach combines the strengths of both methodologies: SBI's ability to perform inference on intractable simulators and Pyro's expressive power and efficiency in handling complex hierarchical structures. We will illustrate the potential and practicality of this integrated methodology using a key example from cognitive science: fitting a hierarchical drift-diffusion model (DDM) to choice data. The focus will be on how this combined "Pyro meets SBI" approach successfully allows for Bayesian inference over the parameters of the hierarchical model, effectively combining the information across multiple simulated subjects while handling the intractable DDM likelihood. This integration significantly expands the scope of rigorous Bayesian inference, opening new possibilities for analyzing complex, simulation-based models across various scientific disciplines. We will also briefly highlight how recent developments in the `sbi` package facilitate this powerful workflow, making advanced hierarchical modeling accessible for simulator-based research. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/KCYYTF/ Room 1.38 (Ground Floor) Jan Boelts (Teusen) PUBLISH LR7S8P@@pretalx.com -LR7S8P Recent Developments in Pytensor, the Successor Package to Theano en en 20250820T133000 20250820T140000 003000 Recent Developments in Pytensor, the Successor Package to Theano After MILA offically stopped development of Theano in 2018, the PyMC project forked the project and continued developing the package under a series of names: Theano-PyMC, Aesara, and now Pytensor. This choice was motivated by Theano's decision to put static computational graphs directly in front of the user. This choice turned out to be ideal for Bayesian inference, because it allows for powerful graph-to-graph transformations. For example, a graph that defines a data generating process via draws from random variables can be automatically translated into a backwards process describing the log probability of observed data, conditioned on draws from prior distributions. This is precisely the same logic that underpins reverse-mode automatic differentiation. It turns out this idea of graph transformation is extremely powerful, and development of Pytensor has focused on fully leveraging this power. Recently, new features have been added to Pytensor, making it a powerful tool for workflows in statistics, machine learning, and Bayesian modeling. One of the most powerful features of Theano was it's use of graph rewrites to optimize computation beyond what a compiler is willing or able to do. A canonical example is to rewrite the express `log(1 + x)` to `log1p(x)`, a version of the computation that remains numerically stable for small values of `x`. Pytensor has fully embraced the system of graph rewrites, and has expanded it to include many new cases, including: - Op fusion: finding sub-graphs that are re-used multiple times on different inputs. These are fused into a single composite Op, compiled once, and re-used. - Aggressive inplacing: identifying all cases where intermediate computations can performed destructively "in-place", alleviated the need for expensive memory allocations - Pre-gradient stabilization: maintain a library of mathematically "non-destructive" rewrites that simplify arbitrary user expressions into simpler forms, leading to cleaner gradient graphs after applying autodiff - Linear algebra optimization: reason about the types of matrices in a graph, and automatically apply specializations to expensive operations. For example, `det(diag(x))` is rewritten to `prod(diag(x))`, and `inv(kron(a, b))` to `kron(inv(a), inv(b))`. These rewrites are possible because as a user writes Pytensor code, a static graph is constructed. At any time, the user can inspect the graph, or directly intervene on it. These interventions include extracting sub-graphs, replacing or removing inputs or functions, or applying entire graph-to-graph transformations. The majority of our talk will focus on this last operation, which represents one of the most unique and powerful features of Pytensor. This capability was first used in Theano to perform automatic differentiation. Given a forward graph of a scalar loss function, a gradient graph can be created by walking backwards from the loss in topological order, applying the chain rule. Graph-to-graph transformations can go far beyond this application. We present the following examples: - *Vectorization*. Pytensor can replace an n-dimensional input with an (n+k) dimensional input, then propagate the consequences through the graph. For example, `dot(matrix, vector)` can be replaced with `dot(tensor3, vector)`, and Pytensor will automatically apply a batched dot product across the left-most dimension of the `tensor3` input. The new output shape will also be propagated to any down-stream computations. - *Automatic probability derivation*. The PyMC package uses to Pytensor to transform a user-defined generative graph, which produces samples of data from priors and inputs, to a graph that gives the probability of observing an output sample, given parameters. In this way, a single model declaration is reused for both inference via Bayes' Law, and for pre- and post-estimation tasks that require forward sampling - *Automatic marginalization*. Similarly, having access to the full graph lets Pytensor reason about non-trivial graph replacements, such as automatic marginalization of random variables. This can remove nuisance parameters from a model and significantly speed up inference algorithms. In addition, because pytensor can graphically verify that batch dimensions remain independent, the algorithm is bounded by the size of the domain of the random variables, rather than by their actual input size. When marginalizing a hidden markov chain, for example, this means we only have to do `states * steps` computations, instead of `states ^ steps` (this is the celebrated "forward algorithm"). We also give several examples where Pytensor offers advantages in deep learning contexts, including: - *Training to prediction transformation*: A common source of errors when working with deep learning models is forgetting to manually remove training-only Ops like dropout and batch norm. Pytensor can automate these tasks using rewrites, which can be automatically applied by a deep learning API built on top of pytensor -*Op Specialization*: Certain extremely expensive layers used in deep learning, including convolutions and transformers, have specialized forms that can be used when the right conditions are met. Examples include FFT-convolution and FlashAttention. Pytensor can recognize Op sequences of the form `matmul-mask-softmax-dropout-matmul` and replace them with a single `FlashAttention` Op. Similarly, convolution Ops can be replaced with FFT convolutions when the kernels are sufficiently large. Both of these optimizations can be done automatically, without users necessarily having to know the intricacies of when and how these specialized versions should be used. We conclude with a short description of planned features for our upcoming Pytensor 3.0 release, and an invitation for interested audience members to try the package and submit issues/PRs. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/LR7S8P/ Room 1.38 (Ground Floor) Jesse Grabowski Ricardo Vieira PUBLISH NKLD7R@@pretalx.com -NKLD7R Let's rewrite optimagic from scratch in half an hour and see what we can learn en en 20250820T140500 20250820T143500 003000 Let's rewrite optimagic from scratch in half an hour and see what we can learn Optimagic provides a unified interface to optimizers from SciPy, NlOpt, Pygmo and many other packages. In a relatively small code base, we add many convenience features to the algorithms we wrap. Collecting and plotting optimizer histories, error handling, and flexible parameter formats are just a few examples. All of this is done without accessing or modifying the source code of the original optimizers, which makes it very simple to add more optimizers to optimagic. This is made possible by a few simple architectural ideas that could also be applied in other scientific packages or research code. In this talk we create a super-simplified educational rewrite of optimagic that illustrates the core ideas and explains how we use them to implement powerful features. We finish by collecting a few examples where the same ideas could be applied to create simple, robust and user-friendly code in other open-source packages or research projects. The talk is meant for users and maintainers of scientific packages alike. No previous experience or interest in numerical optimization is required. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/NKLD7R/ Room 1.38 (Ground Floor) Janos Gabler PUBLISH EAAE9B@@pretalx.com -EAAE9B Enhancing SymPy Algorithms with MatchPy's Efficient Pattern Matching en en 20250820T144000 20250820T150000 002000 Enhancing SymPy Algorithms with MatchPy's Efficient Pattern Matching SymPy and MatchPy are powerful, open-source Python libraries for symbolic mathematics and pattern matching, respectively. While SymPy excels in symbolic manipulation, its built-in pattern matching can be limiting for complex tasks. MatchPy, designed for associative-commutative (AC) pattern matching, offers a unique advantage: its data structures enable the simultaneous matching of multiple patterns against a symbolic expression with only a single tree traversal, significantly boosting efficiency. This talk presents an experimental module that integrates these libraries, demonstrating how MatchPy's capabilities can enhance SymPy's algorithms. We will explore practical applications, including improved equation solving, derivative and integral computations, and differential equation solutions. Notably, we will showcase an experimental implementation of the RUBI integration algorithm, illustrating how MatchPy simplifies the application of intricate integration rules. This work highlights the benefits of combining these tools, fostering the development of more robust and efficient symbolic computation workflows. SymPy website: https://sympy.org/ MatchPy documentation: https://matchpy.readthedocs.io/en/latest SymPy paper: https://peerj.com/articles/cs-103/ MatchPy paper: https://arxiv.org/abs/1710.06915 PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/EAAE9B/ Room 1.38 (Ground Floor) Francesco Bonazzi PUBLISH T88GQE@@pretalx.com -T88GQE Machine learning for ecotoxicology and bee pesticide toxicity prediction en en 20250820T103000 20250820T110000 003000 Machine learning for ecotoxicology and bee pesticide toxicity prediction Agrochemistry, in contrast to medicinal chemistry, is a relatively unexplored area in terms of rational drug design and molecular ML. Data science techniques and predictive ML models, exemplified by ADMET QSAR models, have long been used in pharmaceutical industry. Pesticides are the largest, and most economically important, group of agrochemicals. They need to pass multiple regulatory requirements in order to be used, showing safety not only to humans (toxicology), but also to a variety of wildlife organisms, such as honey bees, earthworms, birds, and fish (ecotoxicology). This is in many ways much more challenging, due to a wide variety of properties that need to be analyzed and predicted. At the same time, we actually require strong toxicity from pesticides, but highly selective, killing preferably only target organisms, e.g. weeds in case of herbicides. Recently created ApisTox (https://doi.org/10.1038/s41597-024-04232-w) is the largest dataset in the literature concerning toxicity of pesticides to honey bees (Apis mellifera). It allows broad analyses of agrochemicals and building ML models for predicting toxicity of pesticides to honey bees. This required creating a complex data processing workflow, which utilized freely available data sources, like ECOTOX database. In this talk, we will go over tools and techniques used, so that attendees will understand challenges related to such tasks, and how to create other similar datasets for practical usage. ApisTox paper was followed up by additional molecular datasets' analyzes and building ML models (currently under review). In this talk, we will also explore initial results of pesticide toxicity classification and how we can approach building molecular ML models for agrochemistry, e.g. molecular fingerprints, graph kernels, and graph neural networks. Results are highly distinct from those on molecular chemistry datasets, indicating a lot of unexplored potential. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/T88GQE/ Room 1.20 (Ground Floor, Shannon) Jakub Adamczyk PUBLISH 9F88K8@@pretalx.com -9F88K8 Python for subsea engineering: A case study on seabed object detection using AI/ML en en 20250820T110500 20250820T113500 003000 Python for subsea engineering: A case study on seabed object detection using AI/ML Recent advancements in artificial intelligence are transforming seabed imaging and geospatial data analysis, enabling more efficient and accurate engineering hazard assessments. Traditionally, seabed object detection relied on manual interpretation of sonar and bathymetric datasets, a time-intensive and subjective process. By leveraging Convolutional Neural Networks (CNNs), we can now automate feature extraction, reducing processing time while improving consistency and scalability in offshore wind farm installations, oil and gas site assessments, and carbon sequestration projects. Objectives: This talk will outline our experience in developing an AI-assisted seabed object detection workflow, addressing the unique challenges of underwater imagery. Unlike terrestrial datasets, subsea images suffer from noise, inconsistent lighting, and varying sensor resolutions, making deep learning adaptation non-trivial. I will share insights on choosing an appropriate deep learning framework, based on usability, flexibility, and performance. Pre-processing underwater imagery is a critical step, requiring techniques to mitigate noise, distortion, and lighting variations in seabed survey data. Talk Structure: 1. Introduction: Importance of seabed object detection and AI's role in offshore wind, marine, and geospatial analysis (3 Minutes) 2. Data Acquisition & Preprocessing: Handling high-resolution seabed survey data and potential applications across other remote sensing using Python libraries (NumPy, Rasterio, GDAL, OpenCV, Scikit-Image) (5 Minutes) 3. Deep Learning Approach: Model selection, training process, and feature extraction using TensorFlow, PyTorch, Scikit-learn, and XGBoost (6 Minutes) 4. Validation & Accuracy Assessment: Comparing AI predictions with expert-labelled datasets (4 Minutes) 5. Results & Visualization: Mapping classified seabed features with Geopandas and Matplotlib (5 Minutes) 6. Applications & Future Scope: Expanding AI use in geospatial workflows, including aerial imaging, LiDAR, sonar analysis, and real-time object detection in underwater robotics (2 Minutes) PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/9F88K8/ Room 1.20 (Ground Floor, Shannon) Samarth Bachkheti PUBLISH ELXEN7@@pretalx.com -ELXEN7 Python Framework for Large-Scale Radar Data Generation and Visualization en en 20250820T114000 20250820T120000 002000 Python Framework for Large-Scale Radar Data Generation and Visualization The advancement of machine learning in automotive radar systems is significantly impeded by the limited availability of annotated raw radar data, specifically tailored to distinct radar sensor configurations. To overcome these limitations, we present a robust Python-based computational framework specifically designed to facilitate efficient large-scale radar data generation and comprehensive visualization. Our framework leverages radar point cloud data, which is stored in the HDF5 format and collected directly from automotive radar sensors deployed in real-world environments. The point cloud data undergoes sophisticated preprocessing where radar detections are accumulated over multiple measurement cycles. This accumulation process is carefully corrected for both ego-vehicle velocity and the velocities of detected objects, ensuring spatial and temporal consistency. Through this preprocessing, our method reliably identifies dominant scatter centers within the radar's operational environment, creating high-quality datasets, termed "pseudo scatter points," that effectively represent persistent environmental features. These pseudo scatter points serve as a foundation for generating synthetic raw radar data suitable for virtual radar systems with arbitrary antenna arrangements and configurations within the widely used 77 GHz automotive radar band. The framework implements advanced radar signal processing algorithms including multi-dimensional Fast Fourier Transforms (FFTs), digital beamforming, Constant False Alarm Rate (CFAR) detection algorithms in range-Doppler domains, specialized peak-finding algorithms in azimuth and elevation dimensions, and interpolation methods to enhance angular resolution. Additionally, synthetic clutter generation methods are integrated to further improve the realism of the generated radar datasets. A key strength of our framework is its adaptability and flexibility, achieved through the utilization of a specialized and extensible Python data class structure. This structure simplifies the step-by-step management of radar data processing pipelines, facilitating customization, scalability, and straightforward integration of additional processing steps or alternative radar parameter configurations. To address the computational demands of generating radar data at scale, the framework employs GPU acceleration via CuPy, a CUDA-enabled implementation of NumPy, enabling significant performance improvements. Currently, it achieves processing rates of approximately 10,000 radar frames per day, depending on radar resolution, thus offering practical scalability for both research and commercial development scenarios. Visualization capabilities are seamlessly integrated, leveraging PyQt5 combined with OpenGL to deliver a powerful inbuilt visualization tool. This visualizer provides detailed, interactive representations of radar cubes in real time, significantly aiding debugging, data exploration, and deeper insight into radar signal characteristics and potential anomalies. Practical applications of our framework include the generation of extensive radar datasets to support the training and validation of advanced deep neural networks for automotive radar perception tasks, as well as early-stage validation and benchmarking against reference radar sensor systems. Validation experiments, conducted through direct comparison with evaluation radar sensor data collected under identical conditions, have demonstrated the framework's capability to reliably reproduce realistic radar spectra, highlighting its value as a cost-effective alternative or complementary solution to expensive field-testing campaigns or high-fidelity ray tracing methods. From a scientific research perspective, our framework directly addresses a critical bottleneck in deep learning-based radar applications—the scalability and availability of data needed to train sophisticated machine learning models. Its flexibility, ease of use, and high computational efficiency make it an attractive tool for researchers and developers aiming to push the boundaries of radar-based perception in automotive safety systems and advanced driver assistance systems (ADAS). We welcome feedback and collaboration from the scientific community at EuroSciPy 2025 to further refine and extend the capabilities of our framework, exploring its broader applicability in scientific computing and automotive research domains. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/ELXEN7/ Room 1.20 (Ground Floor, Shannon) Manuel Jürgensen PUBLISH TAXVPC@@pretalx.com -TAXVPC Sensor data processing on microcontrollers with MicroPython en en 20250820T133000 20250820T140000 003000 Sensor data processing on microcontrollers with MicroPython Here is an overview of the topics you can learn about in this presentation. #### Sensor nodes and Wireless Sensor Networks A sensor node is a combined hardware + software system that can sense things in the physical world. It uses sensor elements such as camera, microphone, accelerometer, radar, temperature et.c. A node has a microcontroller that does data aquition and processing, and also some way of storing data, or transmitting it to another system for further processing and storage. The typical functional blocks of firmware for a sensor node are: - Data readout. Fetching data from each of the attached sensors. - Processing. Extracting useful information from the data. - Data storage. Storing either for long term, or as a transmission buffer. - Data transmission. Sending the extracted information to external systems. - Power management. Transitioning between sleep and aware as needed. For low cost installation and operation, many sensor nodes are battery-powered and use wireless connectivity. And they are often deployed together as part of larger Wireless Sensor Networks. ### About MicroPython MicroPython is an implementation of Python that runs on practically all microcontrollers with 16kB+ RAM. It provides access to the microcontroller hardware, functions for interacting with sensors and external pheripherals, as well as connectivity options such as WiFi, Ethernet, Bluetooth Low Energy, etc. While MicroPython (and the emlearn library) can target a very wide range of hardware, we will focus on the Espressif ESP32 family of devices. These are very powerful and affordable, with good WiFi+BLE connectivity support, gpod open-source toolchains, are very popular both among hobbyist and companies, and have many good ready-to-use hardware development kits. #### Challenges and constraints of microcontroller-based sensor nodes While microcontrollers are getting more powerful year by year, it is still important to fit within the limited RAM, program size and CPU time available. For sensors with low datarates (like accelerometers) this is rather doable, but for higher datarates such as audio or images good practices can be critical. Furthermore we may wish to operate on low-power with long battery life. In that case it is critical to maximize sleeping, which means to reduce device wakeups, and to quickly return back to sleep. Ensuring that we stay within the resource budgets requires some care (in any programming language), and a high-level language like Python poses some particular challenges. #### Tools and practices for efficient sensor data processing in MicroPython We will go through the tools which MicroPython provides for efficient sensor data processing. This includes: - Ways of writing (Micro)Python code that are faster. For example reducing allocations - Optimizing subsets of Python using the @native and @viper code emitters - The built-in Python-based assembler for ARM Cortex M chip - Dynamic native C modules. Can be installed at runtime - User C modules. Can be baked into a custom MicroPython image We will compare these approaches on a few algorithms that are often used of typical sensor data processing. This includes algorithms from the world of Digital Signal Processing as well as Machine Learning. Candidates include Fast Fourier Transform (FFT), Root Mean Square (RMS), Convolutional Neural Network (CNN), and Random Forest (RF). The emlearn-micropython packages provided a set of MicroPython modules that can be installed onto a device, without having to recompile any C code. This preserves the ease-of-use that Python developers are used to on a desktop system. Compared to pure-Python approaches, the emlearn-micropython modules are typically 10-100x faster. #### Intended audience and expected background Any developer or data scientist curious about sensor data processing, IoT, and how Python scales down to the smallest of devices. The audience is expected to have a basic literacy in Python, proficiency in programming, and know the basics of data processing. Some familiarity in time-series processing, Digital Signal Processing or Machine Learning, will make the talk much more relevant to you. Familiarity with microcontrollers and embedded systems is of course an advantage - but the talk should be approachable to those who are new to this area. #### Focus and scope of the talk The main part of the talk will be how to built sensor-nodes for scientific applications with MicroPython, The general introduction to MicroPython will be kept rather brief, as there are many resources for this available already. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/TAXVPC/ Room 1.20 (Ground Floor, Shannon) Jon Nordby PUBLISH LNU8UV@@pretalx.com -LNU8UV PyPI in the face: running jokes that PyPI download stats can play on you en en 20250820T140500 20250820T143500 003000 PyPI in the face: running jokes that PyPI download stats can play on you We all love to tell stories with data and we all love to listen to them. Wouldn't it be great if we could also draw actionable insights from these nice stories? As scikit-learn maintainers, we would love to use PyPI download stats and other proxy metrics (website analytics, github repository statistics, etc ...) to help inform some of our decisions like: - how do we increase user awareness of best practices (please use Pipeline and cross-validation)? - how do we advertise our recent improvements (use HistGradientBoosting rather than GradientBoosting, TunedThresholdClassifier, PCA and a few other models can run on GPU) ? - do users care more about new features from recent releases or consolidation of what already exists? - how long should we support older versions of Python, numpy or scipy ? In the context of scikit-learn, we will present the kind of surprises and caveats we discovered when trying to make sense of the PyPI download stats. Highlights include: - the most downloaded scikit-learn release is from 5 years ago, maybe people actually don't care about our latest developments? - how on earth can a package that errors on install be downloaded 50k a day? - is there any hope to differentiate "real users" vs "automation users" (e.g. Continuous Integration)? We will then zoom out a bit and talk about other metrics we looked at, for example scikit-learn.org website analytics, GitHub stars and "Used by" stats. After presenting all the inherent biases of these data, we will see present the kind of insights we gained by combining them. During the presentation, we will also highlight a few tools and websites we used along the journey to make it easier to look at PyPI download stats numbers in more details. We will conclude with some thoughts about how to use this kind of metrics to inform some of our decisions, while at the same time not falling in love too much with the stories we tell with them. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/LNU8UV/ Room 1.20 (Ground Floor, Shannon) Loïc Estève PUBLISH SNPKGF@@pretalx.com -SNPKGF Industrial-Level Documentation for Scientific Projects en en 20250820T144000 20250820T150000 002000 Industrial-Level Documentation for Scientific Projects This talk focuses on making documentation effortless by centralizing key updates and methodologies, ensuring consistency across multiple libraries. By automating documentation workflows, organizations can reduce manual effort while maintaining high standards of clarity and usability. Key topics covered include: - **Automated API Documentation with AutoAPI:** Using Sphinx AutoAPI, Jinja, and Sphinx Design to create dynamic, auto-generated documentation that communicates all methods, classes, attributes, and imports depending on the module. Proper structuring and a solid table of contents ensure clarity and navigability. - **Changelog Generation with Towncrier:** Implementing automatic changelog generation that creates a dynamic "What's New" section, clearly displaying issues and links in tabs for concise, understandable release notes. - **Real-Time search with Fuse.js:** The importance of real-time search results as users type. We use Fuse.js to integrate live search into the documentation with minimal configuration. The search is browser-based, ensuring it works efficiently, even when servers or third-party libraries are not available. - **SEO for Better Discoverability:** Understanding the key steps to improve SEO for documentation, ensuring that the content is easily discoverable on Google. We will discuss techniques to enhance the visibility of your docs and increase search clicks using Google Search Console (GSC). - **Accessibility and UX Enhancements:** The importance of ensuring documentation is accessible to everyone, regardless of ability. We will explore how to implement dark/light themes, improve visual elements like icons, fonts, and snippets, and ensure the overall UI/UX design promotes inclusivity. These approaches are applicable to organizations of all sizes, offering scalable solutions that not only streamline documentation processes but also enhance the overall user experience. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/SNPKGF/ Room 1.20 (Ground Floor, Shannon) Revathy Venugopal PUBLISH DN7SMP@@pretalx.com -DN7SMP Array API and library dispatching en en 20250820T110500 20250820T115000 004500 Array API and library dispatching This session will be split in two parts. First we wish discuss Array API adoption into libraries, how to continue this work and what needs to be done. There has been a lot of progress in adoption across several libraries and interesting efforts like https://github.com/mdhaber/marray. What are patterns that work well or need improvement in the future? In the second part, we want to discuss adoption of dispatching and backend selection across libraries. NetworkX has this for a long time and scikit-image has experimented with it in the past year. We plan on showing the spatch design, but wish to focus discussion on what the missing pieces are for libraries to adopt spatch or similar patterns. PUBLIC CONFIRMED Maintainer track https://pretalx.com/euroscipy-2025/talk/DN7SMP/ Room 2.41 (First Floor, Turing) Tim Head Sebastian Berg PUBLISH QPF9N7@@pretalx.com -QPF9N7 Maintaining People, Not Just Projects: Attracting and Retaining Talent in FOSS en en 20250820T133000 20250820T150000 013000 Maintaining People, Not Just Projects: Attracting and Retaining Talent in FOSS The scientific Python ecosystem powers research, education, and innovation across disciplines from physics and biology to finance and AI. But its long-term sustainability doesn’t just depend on code, it depends on people! As maintainers, we often focus on the technical infrastructure of our projects: CI pipelines, packaging, release cycles. Yet the more enduring challenge is sustaining the human infrastructure that makes open source possible in the first place. While new contributors continue to find their way into the ecosystem, many don’t stay. Unclear career pathways, emotional labor, burnout, limited funding, and governance challenges all play a role in attrition. This round table is an opportunity for maintainers to step back from triage and talk candidly with peers about the human side of open source: mentorship, collaboration, recognition, and belonging. We’ll explore the shared challenges we face in attracting and retaining contributors and the practical strategies that have helped us build healthier, more resilient communities. This session is designed as a participatory, peer-to-peer discussion. However some topics that we could explore are: * Why contributors join—and why they leave * Mentorship models that actually scale * Recognition and credit (in academia and beyond) * Balancing paid and volunteer contributions * Avoiding burnout (yours and others’) PUBLIC CONFIRMED Maintainer track long https://pretalx.com/euroscipy-2025/talk/QPF9N7/ Room 2.41 (First Floor, Turing) Kai Striega PUBLISH VCLRCU@@pretalx.com -VCLRCU The Fellowship of the Stack: Scientific Discovery in Python en en 20250821T090000 20250821T100000 010000 The Fellowship of the Stack: Scientific Discovery in Python Transformed from humble beginnings into a catalyst for breakthrough science—decoding cosmic mysteries, healing human diseases, predicting planetary futures—our hero joins millions of computational explorers pushing the boundaries of human understanding. Every scientific revolution starts with a single spark of code. What epic will you write? In this keynote we'll take a narrative approach to the evolution of scientific experimentation, the daunting challenges that await and how we as the protagonists can find refuge with each other in the Python ecosystem. PUBLIC CONFIRMED Keynote https://pretalx.com/euroscipy-2025/talk/VCLRCU/ Room 1.38 (Ground Floor) Dawn Wages PUBLISH K9L3VH@@pretalx.com -K9L3VH Units next to your Data: Arrays with Scipp en en 20250821T103000 20250821T110000 003000 Units next to your Data: Arrays with Scipp This talk will introduce the Scipp library, originally developed for neutron science experiments, and how it can be useful for record-based "tabular"/"event" data in general. One of Scipp's key features is the possibility of binning to sort record-based data into arrays of bins. This provides fast and flexible binning, rebinning, and filtering operations, all while preserving the original individual records. If your use case requires one or several of the items on the following list, using Scipp may be worth considering: - Physical units are stored with each data or coord array and are handled in arithmetic operations. - Histograms, i.e., bin-edge axes, which are by 1 longer than the data extent. - Support for non-regular or scattered data and non-destructive binning. - Support for masks stored with data. - Propagation of uncertainties. - Internals written in C++ for better performance (for certain applications), in combination with Python bindings. In the talk we will cover: - Why units are important? What's the current landscape? (5 mins) - Labeled dimensions, Units (in Scipp) and data structures in scipp (5 mins) - Bins, Histograms and Uncertainties in Scipp (10 mins) - Tips and tricks of multi dimensional data handling (5 mins) - Buffer and Q/A (5 mins) PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/K9L3VH/ Room 1.38 (Ground Floor) Mridul Seth PUBLISH G3WXJM@@pretalx.com -G3WXJM Routing Strategies for Heterogeneous GenAI Systems: Lessons from Real-World Practice en en 20250821T110500 20250821T113500 003000 Routing Strategies for Heterogeneous GenAI Systems: Lessons from Real-World Practice Modern Generative AI (GenAI) systems combine prompts, language models, inference servers, and specialized hardware into sophisticated stacks. As no single large GenAI system excels at all tasks, we at Techniker Krankenkasse are increasingly adopting a multi-system approach, employing different models tailored to specific tasks, domains, cost, or latency requirements. While this approach enhances robustness and efficiency, it introduces a critical operational challenge: effectively routing each incoming query to the most suitable GenAI system. In this talk, we present our real-world experiences developing dynamic routing pipelines for selecting the optimal GenAI system based on input content and task specificity. We detail the evolution and refinement of our routing strategies, including: - Regular-expression filters to quickly capture clear-cut topics and enforce guardrails; - Off-the-shelf Named Entity Recognition (NER) modules to integrate domain-specific contextual signals; - Few-shot fine-tuning intent classifiers capable of generalizing beyond simple keyword matching - Lightweight generative LLMs that enable cost-effective, context-aware decision-making; - Selective escalation strategies employing state-of-the-art LLMs exclusively when more economical routes provide insufficient confidence. We share insights and best practices from our real-world implementation experience. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/G3WXJM/ Room 1.38 (Ground Floor) Oliver Zeigemann PUBLISH 7BXY7G@@pretalx.com -7BXY7G Breaking the Constraints of Linear Notebook Environments en en 20250821T114000 20250821T120000 002000 Breaking the Constraints of Linear Notebook Environments The talk will explore the limitations of current interactive notebook paradigms and introduce Twinsong, an experimental alternative to Jupyter that reimagines interactive programming for scientific computing. The talk will explore the design philosophy, technical implementation, and potential impact on scientific computing workflows. Unlike linear notebooks, TwinSong decouples code execution from outputs. This approach preserves execution history, allows multiple kernel instances, and provides a cleaner, more manageable computational environment. Users can also organize code hierarchically, enabling: batch subtree evaluation, hierarchically scoped memory spaces that prevent variable leakage, and more intuitive code structuring for complex scientific workflows. TwinSong offers visibility into computational states, including detailed variable inspection and chronological output tracking. TwinSong also provides Git-friendly notebook formats that minimize collaboration friction. Rust-based backend ensures performance and reliability and clean Python kernel implementation without loading Python modules and starting new Python threads. TwinSong is an open source and available at: github.com/spirali/twinsong. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/7BXY7G/ Room 1.38 (Ground Floor) Ada Böhm PUBLISH 9LAVJW@@pretalx.com -9LAVJW Understanding Dispatching Approaches in the Scientific Python Ecosystem en en 20250821T133000 20250821T140000 003000 Understanding Dispatching Approaches in the Scientific Python Ecosystem # Description In the first few minutes we will go over what dispatching is, why it's needed, what kind of projects may benefit from it, and the kinds of dispatching i.e the user decides which backend to use explicitly or the library implicitly dispatch to the "right" implementation for the user or implicitly dispatch based on the input type(s). ## Array API standard and NumPy predecessors (10-12 mins) In this section we will start with NumPy's old dispatching, briefly explaining how it works, and also showing what it can do, such as allowing to use a NumPy function with cupy. We will then continue with the Array API which has momentum as it is being used by libraries such as SciPy and sklearn and with existing support for numpy, pytorch, JAX, CuPy, Xarray, etc. We will highlight it's use by briefly show-casing the speed-up when a user switches e.g. from NumPy to pytorch. These examples will help us expose key difference in type dispatching approaches both for it's users and for the implementation. A main difference being that the Array API is library orientated while the NumPy dispatching was user-orientated. ## Backend-library based dispatching (10-12 mins) We will then continue with the more general backend-library based dispatching approach that is implemented in libraries like NetworkX and scikit-image. This approach is more general and opens up additional possibilities as it is based on Python [`entry_point`](https://packaging.python.org/en/latest/specifications/entry-points), which are generally used to extend the functionality of a package. We will understand this `entry-point` based dispatching through a quick demo. Then we will see a demo of how NetworkX speeds-up with [nx-parallel](https://github.com/networkx/nx-parallel) and [nx-cugraph](https://github.com/rapidsai/nx-cugraph) backends, and the different user APIs in NetworkX to dispatch a call: based on backend-specific graph's type, `backend=` kwarg, environment variables, global configurations, context manager. Then we will go over some of the learnings gathered while integrating this mechanism in [scikit-image](https://github.com/scikit-image/scikit-image) and challenges faced with dispatching arrays inputs. ## Summary and Comparison We will end with a summary of how dispatching approaches differ and when to use which ones, both as a developer and a user of a project. In this summary we wish to expand beyond the previous examples and also mention for example [DataFrame API standards](https://data-apis.org/dataframe-api/draft/) and [narwhals](https://github.com/narwhals-dev/narwhals). ## Intended audience: - Contributors/Maintainers of Python libraries who are interested in providing faster algorithms without changing much of their codebase and user-API - People who work with large graph, image and array datasets - Anyone interested in API dispatching or any of the above stuff :) Some basic knowledge of Python is expected; should know what objects and classes are. Thank you :) PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/9LAVJW/ Room 1.38 (Ground Floor) Sebastian Berg Aditi Juneja PUBLISH ZXQJVQ@@pretalx.com -ZXQJVQ Numba-MPI & PyMPDATA: JIT-compilation and multi-threading for high performance computing in Python en en 20250821T140500 20250821T143500 003000 Numba-MPI & PyMPDATA: JIT-compilation and multi-threading for high performance computing in Python Message Parsing Interface (MPI) is a communication standard allowing for universal, portable high performance computing in distributed systems. Written for C, C++ and Fortran users it has since been incorporated into many pararell programming libraries, including implementations in Python. Numba-MPI is a pure Python package that allows to bring low-level high performance computing on processor clusters to high-level Python simulations by the combined use of Numba JIT compilation and multi-threading. PyMPDATA-MPI is an extension to PyMPDATA, that allows for high performance computation using the Multidimensional Positive Defined Advection Transport Algorithm (MPDATA) for solving advection-diffusion PDEs. Thanks to being based on Numba-MPI, this package allows the user to precisely control domain decomposition in calculations and gain the advantage of hybrid shared- and distributed-memory parallelism using multi-threading and MPI. In this talk I will explain the basic concepts behind JIT compilation and distributed-memory calculations. Then I will outline the features and architecture of aforementioned packages, followed by usage examples of Numba-MPI coupled with PyMPDATA. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/ZXQJVQ/ Room 1.38 (Ground Floor) Michał Wroński PUBLISH TP8ZB7@@pretalx.com -TP8ZB7 Python-Blosc2: Compress Better, Compute Bigger! en en 20250821T144000 20250821T150000 002000 Python-Blosc2: Compress Better, Compute Bigger! Blosc and Blosc2 are well-known and widely used libraries for high-performance data compression. They are particularly effective for compressing large datasets, such as those encountered in data science and high-performance computing. The Blosc library has been around for over a decade, and its design has always prioritized speed, with a focus on achieving compression and decompression speeds that are close to or even exceed memory bandwidth limits. With the introduction of a new compute engine in Python-Blosc2 3.0, the guiding principle has evolved to "Compress Better, Compute Bigger." This enhancement enables computations on datasets that are over 100 times larger than the available RAM, all while maintaining high performance. During our talk, we will delve into the latest features of Python-Blosc2, including: * Seamless integration with NumPy and the Python Data ecosystem * High-performance compression and decompression * The new compute engine and its capabilities * A JIT (Just-In-Time) compiler for Python functions including almost all NumPy functions * The ability to perform computations on datasets that exceed available RAM To illustrate this, we will present an example of using Python-Blosc2 to analyze a dataset that largely exceeds the capacity of the available RAM. We will demonstrate how to leverage the new compute engine to perform computations efficiently, without the need for specialized hardware or infrastructure. By the end of this talk, attendees will understand how Python-Blosc2 can help overcome memory constraints in their data workflows. Whether you're working with medium-sized datasets on modest hardware or large datasets on high-performance systems, you'll learn practical techniques to compress data while maintaining computational efficiency. Join us to explore how this powerful library can expand your capabilities for scientific computing and data analysis while reducing memory footprint and improving processing speed. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/TP8ZB7/ Room 1.38 (Ground Floor) Francesc Alted Luke Shaw PUBLISH 7XUYKH@@pretalx.com -7XUYKH Python Profiling and Optimisation—A Training Course for Researchers en en 20250821T153000 20250821T155000 002000 Python Profiling and Optimisation—A Training Course for Researchers Most researchers writing software are not classically trained programmers. Instead, they learn Python organically, often developing unpythonic habits that negatively impact their software‘s performance. As research software engineers working with researchers from different areas, we decided to develop training on Python profiling and optimisation to share best practices. In the first part of that course, we introduce learners to different profiling approaches such as function-level profiling (using `cProfile` and `snakeviz`) or line-level profiling (using `line_profiler`) to identify which parts of a program run slowest. In the second part, we introduce a wide range of optimisations to speed up those sections of the program. Optimisations described in the course include effective usage of language features and the standard library, choosing suitable data structures, using NumPy and other scientific Python packages, and more. Using these optimisations in real-life workloads, we have observed significant performance improvements, sometimes by more than an order of magnitude. We ran early versions of the course at multiple universities for audiences consisting of students, researchers and research software engineers, using feedback we have received from learners to guide further development of the course. All course materials are available online via the [Carpentries Incubator](https://carpentries-incubator.org). We welcome contributions to the course materials and encourage attendees to run the course at their own institutions. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/7XUYKH/ Room 1.38 (Ground Floor) Jost Migenda PUBLISH ZRA3GV@@pretalx.com -ZRA3GV Solving Hard Optimization Problems with Pyomo and HiGHS: A Practical Introduction en en 20250821T160000 20250821T162000 002000 Solving Hard Optimization Problems with Pyomo and HiGHS: A Practical Introduction ### **1. Introduction (3 min) – What Kind of Problems Can MIP Solve?** - MIP is widely used in scheduling, logistics, and operations research. - Examples of real-world problems: workforce scheduling, vehicle routing, and conference scheduling. - Why are these problems hard? The explosion of possible solutions in combinatorial optimization. ### **2. Why Are These Problems Difficult? (2 min) – The Challenge of Combinatorial Optimization** - Theoretical complexity: Why brute-force search is infeasible. - How solvers like **HiGHS** approach the problem efficiently: - **Branch-and-Bound**: How it systematically narrows down the search space. - **Cutting Planes & Presolve Techniques**: Reducing problem size before solving. - **Heuristics vs. Exact Solutions**: Trade-offs in computation time. ### **3. Introduction to Pyomo & HiGHS (4 min) – The Optimization Toolkit** - **Pyomo**: A structured way to define optimization models in Python. - **HiGHS**: A high-performance solver for linear and mixed-integer problems. - How they work together: Model in Pyomo → Solve with HiGHS → Interpret results. ### **4. Conference Scheduling (5 min) – A Practical Example** - Formulating the problem as a **MIP model**: - Decision variables: Assigning talks to rooms and time slots. - Constraints: Speaker availability, room capacities, topic grouping. - Objective function: Maximizing session coherence and fairness. - Broad overview of the **Python implementation**: - Writing the model in Pyomo. - Solving it using HiGHS. - Evaluating the solution. ### **5. Conclusion & Wrap-up (1 min) – Key Takeaways** - Why Pyomo and HiGHS? Flexibility, efficiency, and scalability. - Where to go next: Learning resources and real-world applications beyond scheduling. **Slides**: [https://florianwilhelm.info/documents/euro_scipy_talk_2025.pdf](https://florianwilhelm.info/documents/euro_scipy_talk_2025.pdf) PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/ZRA3GV/ Room 1.38 (Ground Floor) Florian Wilhelm PUBLISH MLCQQL@@pretalx.com -MLCQQL Automated Chess Analysis: Real-Time Move Detection and Game Narration Using Computer Vision and Large Language Models en en 20250821T163000 20250821T165000 002000 Automated Chess Analysis: Real-Time Move Detection and Game Narration Using Computer Vision and Large Language Models This talk presents the development of a python application for the detection and interpretation of chess moves from video footage, blending deep learning based computer vision, motion tracking, and LLM based sequence analysis. The system is designed to identify all 12 chess piece types—pawn, rook, knight, bishop, queen, and king in both black and white—on an 8×8 board, track their movements across frames. It then converts these actions into standard algebraic notation (e.g., "e4", "Nf3", "Qxd5"). A key feature of this application is the ability to distinguish between valid moves and incidental adjustments, like nudging a piece. In addition, based on the chess moves an LLM is used to generate an educational commentary on the game which adds an engaging narrative dimension for users, making it suitable for learners and casual viewers alike. The application workflow begins with object detection using a YOLOv8 model trained on a labeled chess dataset, which outputs bounding boxes and class probabilities for each chess piece. The centroids of these detected bounding boxes are then mapped to corresponding chessboard squares (e.g., "a1" to "h8"). By comparing piece positions across consecutive video frames, the system infers potential moves, which are subsequently validated using the python-chess library to ensure legality—such as preventing illegal pawn movements. Once a move is confirmed, it is passed to OpenAI’s GPT-4, which generates educational and context-aware commentary. This commentary is then converted to audio using Google Text-to-Speech (gTTS), creating an engaging and informative user experience. Finally this application is packaged within a Streamlit app that provides an interactive platform, allowing users to upload videos, view annotated outputs, and download commentary audio. This pipeline combines YOLOv8’s speed, chess-specific logic, and AI-driven narration into a cohesive system. The computer vision and LLM based workflow successfully automates move detection in chess games by leveraging a YOLOv8s model to process user-submitted videos, accurately generating legal move sequences and producing annotated output videos. Building on this, the Streamlit application seamlessly combines visual move annotations, structured move lists, and GPT-4-generated audio commentary, delivering a rich and interactive user experience. This integrated pipeline highlights the powerful synergy between computer vision and large language models, demonstrating a practical, real-world application where automated visual recognition and natural language generation come together to create dynamic and educational chess commentary. Furthermore, integrating large language models for commentary generation opens new possibilities for smart chess boards, coaching applications, live-streamed matches with narration, and automated game archiving for tournaments and classrooms. Overall, this fusion of computer vision and natural language generation bridges the gap between physical and digital chess, fostering greater inclusivity, deeper engagement, and accelerated learning across the chess community. Advancements in this work may include expanding to multi-player support by incorporating hand tracking or multi-camera systems to accurately detect player turns and interactions, as well as enhancing analysis capabilities to assess move quality, providing detailed feedback on mistakes and exceptional plays. This is an open source project and the GitHub repository details and steps for installation/running this application will be shared during the talk. Github: https://github.com/LikhitaYerra/Chess-Vision-Narrator Presentation: https://docs.google.com/presentation/d/1ZkafIz_0lOLsXR2Ct11pox1E1HZLYi_NkxoVaz5UF_I/edit?usp=sharing Magazine:https://drive.google.com/file/d/1bZJJ5V2K_a6TVgSa2sDLqR-9hy1T7Be1/view?usp=sharing PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/MLCQQL/ Room 1.38 (Ground Floor) Anuradha Kar, PhD Anuradha KAR Likhita Yerra PUBLISH CTCYRB@@pretalx.com -CTCYRB Efficient and accurate models for peptide function prediction en en 20250821T103000 20250821T110000 003000 Efficient and accurate models for peptide function prediction Peptides, as small proteins, play crucial roles in biological processes and offer immense therapeutic potential in areas such as antimicrobial resistance, cancer treatment, and antiviral therapies. While deep learning methods like graph neural networks (GNNs) and protein language models (PLMs) have been widely explored for peptide function prediction, they often face scalability challenges and require significant computational resources. We present methods and results from our paper (https://arxiv.org/abs/2501.17901), introducing an alternative approach that leverages molecular fingerprints—well-established chemoinformatics techniques primarily used with smaller molecules—to predict peptide properties efficiently and accurately. Our research demonstrates that count-based variants of hashed molecular fingerprints, when paired with tree-based classifiers like LightGBM, outperform deep learning methods. We validate our approach across six benchmarks and 126 datasets, achieving state-of-the-art results in peptide function prediction. Our findings challenge the assumed necessity of long-range dependencies in peptides, showing that short-range molecular substructures capture information sufficient for accurate function prediction. Additionally, we will present performance optimizations that enhance computational efficiency, including parallel implementation and sparse representations. Our work is encapsulated in an open-source Python library, scikit-fingerprints, providing a practical tool for researchers in machine learning and computational chemistry. This presentation will offer insights into the broader applications of peptide-based drug discovery and highlight the importance of using molecular fingerprints in chemoinformatics with scalable machine learning frameworks. Attendees will gain an understanding of current chemoinformatics research on peptides and familiarize with graph vectorization methods. They will see how combining domain-specific feature extraction with tree ensembles can yield superior results compared to complex models, all at a fraction of the computational cost. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/CTCYRB/ Room 1.20 (Ground Floor, Shannon) Piotr Ludynia PUBLISH RMGD73@@pretalx.com -RMGD73 Efficient processing pipelines for large scale molecular datasets in Python en en 20250821T110500 20250821T113500 003000 Efficient processing pipelines for large scale molecular datasets in Python Problem Statement Public molecular collections are fragmented across multiple repositories, e.g. PubChem, UniChem, or COCONUT. Each has a distinct data representation, inclusion criteria, and other idiosyncrasies. This makes it difficult to assemble a single, high-quality dataset, e.g. for pretraining molecular foundation models, aimed at varied chemoinformatics applications. Software Our framework accepts one or more raw chemical datasets as inputs which are then delegated into their respective pipelines (described below). The output is a single, cleaned master dataset. Pipeline Overview Pipeline consumes raw data from each source and executes a sequence of steps: 1. Download: fetch original files 2. Preprocess: parse and normalize input formats 3. Standardize: sanitize and canonicalize SMILES via RDKit 4. Deduplicate: remove duplicates using InChI keys (per source, then globally) The final output is a merged “master” dataset ready for downstream analysis or modeling. Technologies & Tools - Dask - distributed and parallel computing framework. Pipelines require significant use of Python libraries, hence we need a tool which handles parallel Python interpreters very well. - RDKit - for chemical operations. Thanks to its core written in C++,the operations are high-performance. - scikit-fingerprints - for molecular filters Intended Uses This software enables users to: - Pretrain large ML models on unified chemical space - Compare dataset cardinality, functional-group distributions, Bemis–Murcko scaffolds, and Circles metrics - Benchmark self-supervised architectures (e.g., Mol2Vec, MolFormer) across varied dataset sizes and chemical domains - Integrate new data sources or custom filters as requirements evolve PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/RMGD73/ Room 1.20 (Ground Floor, Shannon) Franciszek Job PUBLISH 7SLETG@@pretalx.com -7SLETG EffVer: Versioning code by the effort required to upgrade en en 20250821T114000 20250821T120000 002000 EffVer: Versioning code by the effort required to upgrade Version numbers are hard to get right. Maintainers want to communicate to users what the impact of adopting a new version will be, but poor communication can lead to a lot of frustration. There are a few popular version schemes in use today including Semantic Versioning (SemVer) and Calendar Versioning (CalVer). However, projects in the Python community often don’t strictly conform to these standards which leads to confusion. In this talk we will discuss Intended Effort Versioning (EffVer), a new scheme that captures the reality of what many Python projects do today. This formalisation has been officially adopted by projects including Jupyter Hub, Matplotlib, JAX and many more. It’s very common for projects in the Python ecosystem to try to follow Semantic Versioning (SemVer), a scheme that adds specific meanings to each version segment and guarantees backward compatibility in all but major releases. In practice many projects violate the semantics laid out in SemVer because often reality is more complex than the scheme allows for. Some projects go another route and use Calendar Versioning (CalVer), a scheme that intentionally gives no meaning in the segments other than the date the software was released. This information causes challenges in other ways, such as not communicating how much effort it will be to adopt the new version or misleading users to believe that releases of different projects from the same time period will be compatible. EffVer is forward and backward compatible with SemVer, making it easy to adopt for projects already loosely following SemVer. However, instead of communicating specific semantics about what a release contains, it communicates the magnitude of the effort required to adopt those changes. This is already what many Python packages do today, but by formalising things it makes it easy for maintainers to reason about what the next version of a project should be, and allows users to more clearly understand how much effort they need to spend to adopt a new version. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/7SLETG/ Room 1.20 (Ground Floor, Shannon) Jacob Tomlinson PUBLISH GQBY9V@@pretalx.com -GQBY9V Voilà Meta-Dashboards for Streamlined Geospatial Data Visualization en en 20250821T133000 20250821T140000 003000 Voilà Meta-Dashboards for Streamlined Geospatial Data Visualization This presentation delves into the Joint Research Centre's (JRC) extensive experience in leveraging Voilà dashboards, widget libraries, and geospatial web applications for scientific data visualization. Over the past several years, the JRC has cultivated a robust ecosystem around Voilà and Python, resulting in the design and deployment of numerous real-world web applications across diverse scientific domains, including agricultural/biodiversity surveys and air quality monitoring. This effort has been accompanied by a strong commitment to building a thriving developer community, supported by dedicated training and comprehensive resource libraries, like for example the vois Python library, available as open source at https://code.europa.eu/jrc-bdap/vois. A key focus of this talk is the introduction and demonstration of our innovative meta-dashboards. These are Voilà-powered applications designed to streamline the creation of complex, multi-page dashboards by automating framework and code generation. We will showcase how meta-dashboards, such as the tool accessible at https://vois.azurewebsites.net/, can significantly accelerate the development of complex geospatial web applications. Through a live demonstration, attendees will witness the ease with which a fully functional geospatial application can be built using these visual tools. The presentation will demonstrate Voilà's efficacy in developing real-world web applications that transform complex datasets into actionable insights. Participants will gain a clear understanding of how Voilà is used at the JRC to create powerful data visualization tools, supporting comprehensive data assessment and informed decision-making across various scientific fields. The applications showcased will highlight the ability of Voilà to handle large and complex datasets, and to create interactive environments that are easy to use and understand (see https://www.daigio.it/EuroSciPy2025/ for some screenshots of the Voilà dashboards that will be presented). PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/GQBY9V/ Room 1.20 (Ground Floor, Shannon) Davide De Marchi PUBLISH SVTWWE@@pretalx.com -SVTWWE Processing Cloud-optimized data in Python (Dataplug) en en 20250821T140500 20250821T143500 003000 Processing Cloud-optimized data in Python (Dataplug) Cloud-optimized (CO) data formats are designed to efficiently store and access data directly from cloud object storage without needing to download the entire dataset. These formats enable faster data retrieval, scalability, and cost-effectiveness by allowing users to fetch only the necessary subsets of data. They also allow for efficient parallel data processing using on-the-fly partitioning, which can considerably accelerate data management operations. For example, Dask can efficiently read data in parallel from Object Storage in CO formats like ZARR. Cloud-optimized formats are now widely used in geospatial settings with entire datasets available in the AWS Registry for Open Data like [Sentinel-2 Cloud Optimized GeoTIFFs](https://registry.opendata.aws/sentinel-2-l2a-cogs/). In this line, COPC (Cloud Optimized Point Cloud) was developed to overcome the limitations of LIDAR. Likewise, Cloud Optimized GeoTIFF (COG) was developed to facilitate cloud processing of GeoTIFF files. Nevertheless, there are no cloud optimized versions of widely used formats in genomics (FASTA, FASTQ, VCF, FASTQGZIP) and metabolomics (imzML). Furthermore, a costly preprocessing from legacy formats is required (from GeoTIFF to COG, from LIDAR to COPC). In this talk, we will present a novel data processing library called [Dataplug](https://github.com/CLOUDLAB-URV/dataplug) that enables Cloud-optimized access to legacy formats without a costly preprocessing and also avoiding huge data movements. Dataplug covers legacy formats like LIDAR but also major data formats found in bioinformatics (genomics, metabolomics) that lack appropriate Cloud Optimized alternatives. In this talk, you will learn how to process scientific data formats in Python using the [Dataplug library](https://github.com/CLOUDLAB-URV/dataplug) from any Python data analytics platform like Dask or Ray. We will show different data processing pipelines in the Cloud that demonstrate the benefits of cloud-optimized data management. ## Objectives By the end of this talk, you will be able to: 1. Understand Cloud-Optimized data formats and their benefits for data processing in the Cloud 2. Learn how to process Cloud Optimized Data from Object Storage in Python using Dask 3. Use Dataplug library to enable on-the-fly partitioning of Cloud Optimized data (COG, ZARR, COPC). 4. Use Dataplug library to enable on-the-fly partitioning of non-Cloud Optimized formats (LIDAR, FASTQGZIP, FASTA, FASTQ, VCF,imzML, MS) ## Outline Introduction (10 minutes) 1. About Us 2. Understanding Cloud-Optimized data formats and Cloud Object storage 4. Processing Cloud-Optimized data in Dask Processing Cloud-optimized data in the Cloud with Python (15 minutes) 1. Processing COG (Cloud-Optimized GeoTIFFs) in Python in the [NDVI pipeline](https://github.com/cloudbutton/geospatial-usecase/tree/main/ndvi-diff) 2. On-the-fly processing of compressed genomic data (FASTQGZIP) with [Dataplug](https://github.com/CLOUDLAB-URV/dataplug/blob/master/examples/fastqgz_example.py) 3. On-the-fly processing of metabolomics data (imzML) with [Dataplug](https://github.com/CLOUDLAB-URV/dataplug/blob/master/examples/imzml_processed_example.py) 3. Commparing LIDAR and COPC processing with Dataplug library in Dask ([code](https://github.com/CLOUDLAB-URV/dataplug/blob/master/examples/lidar_example.py)) Conclusions (2 minutes) ## Audience The talk is aimed at Python developers interested in processing data in the Cloud. In particular, it may be of interest in the following domains: geospatial data (COG, COPC, LIDAR, ZARR, Kerchunk), genomics data (FASTA, FASTQ, VCF, FASTQGZIP), astronomics (MS), and metabolomics data (imzML). This talk requires basic understanding of Cloud Object Storage. PUBLIC CONFIRMED Talk (25 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/SVTWWE/ Room 1.20 (Ground Floor, Shannon) Universitat Rovira i Virgili (Pedro Garcia Lopez) Daniel Alejandro Coll Tejeda PUBLISH VELXWA@@pretalx.com -VELXWA Can Your Research Notebook Survive a Code Update? On maintaining Reproducibility with Continuous Integration en en 20250821T144000 20250821T150000 002000 Can Your Research Notebook Survive a Code Update? On maintaining Reproducibility with Continuous Integration From the user's perspective, maintenance of notebooks that reproduce paper results using up-to-date project codebase serves the purpose of documenting and exemplifying project features and applications. It enables exploratory usage with little-to-no effort needed to set up a working environment. However, this is contingent on a design embracing modularity and inversion of control - it is not uncommon in atmospheric modelling for papers to use different simulation flow control or different parameterisations. We present the inversion of control solutions from PySDM that enable the choice of formulae and constants from user code, without trade-offs in: (i) performance, (ii) ability to switch between CPU and GPU backends, and (iii) dimensional analysis of physics-relevant routines for testing unit correctness. The maintenance of notebooks within code repositories also poses challenges in terms of handling embedded visuals. The jupyter-utils project helps with embedding GitHub-renderable and Jupyter-book-compatible vector graphics and animations. From the developer's perspective, notebooks within the code repository are a source of test scenarios and constraints for the assertions that constitute a robust regression-test suite. We present the notebook_vars() function from the jupyter-utils package, designed to be used in concert with the fixture logic of the pytest framework. It enables the execution of the notebook code once in a test session, allowing us of the final notebook state in multiple automated tests. This occurs without modifications to the notebooks themselves. In notebooks pertaining to specific research studies, a direct link from test code up to subject literature is provided. Overall, we achieve an improvement in code readability and refactoring capability. The presented solutions, along with the availability of platforms such as Google Colab, mybinder.org or institutional Jupyter hubs, ensure single-click reproducibility of research-paper results, and a structure for retaining this through future releases of the code base. In addition to the benefits for software users and developers, this satisfies the scientific-method and research-journal reproducibility requirements. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/VELXWA/ Room 1.20 (Ground Floor, Shannon) Agnieszka Żaba PUBLISH MDVAHD@@pretalx.com -MDVAHD ELVA - Local-First Real-Time Collaboration Apps in Your Terminal en en 20250821T153000 20250821T155000 002000 ELVA - Local-First Real-Time Collaboration Apps in Your Terminal ### Our Motivation As scientists working with distributed groups all over the world, we have high and numerous expectations for a digital collaboration tool: - platform-independent - secure - reliable synchronization of any kind of data - works seamlessly offline - private, no remote cloud services involved - no central servers required, peer-to-peer - access to the local filesystem - easily adaptable and extensible - easily accessible - stable and lightweight - open-source Several tools already exist that can fulfill a subset of these requirements, but they come with all sort of restrictions. Some examples: - Git is great for keeping track of text files, but it requires you to manually save the current state by committing and lacks efficient native support for binary files. Also, the user has to manually deal with merge conflicts when multiple people work on the same files. While powerful extensions such as Git-LFS and git-annex do a great job with binary data, Git is still unsuitable as general purpose collaboration platform. - Cloud-based software such as Nextcloud, Seafile or the corresponding commercial alternatives are able to sync any type of data out of the box, but they implement versioning only in a rudimentary fashion, if at all. Furthermore, a central server is a vital part of the architecture. Since their synchronization is file-based, merge conflicts still arise and need to be resolved by hand. - Browser-based editors like Visual Studio Code, HedgeDoc or Etherpad feature real-time synchronization of content, but cannot sync binary data, and when they are implemented as in-browser apps, they cannot always access the local filesystem. `ELVA` aims to close these gaps. ### Our Ideas In short, we want to have the comfortable real-time synchronization of in-browser apps but in a software that runs locally for proper access to the filesystem on every operating system. Conflict-free replicating data types (CRDTs) are the backbone of `ELVA`. They provide the fundamental logic for proper integration of shared data fragments. The suite shall be written in Python, as it is relatively easy to learn and widely used in scientific research. If users of `ELVA` feel the need for special functionality, they should be able to add it by themselves without much effort. Also, as we intend `ELVA` to be also usable by non-technical people, we need to have a good user interface (UI). We decided to settle on text-based terminal apps, which are lightweight and work consistently across platforms, but graphical interfaces are likewise implementable. Versioning could still be realized with a Git integration and synchronizing bulk storage is a question of app and protocol design. ### Our Prototype We are proud to present our prototype of `ELVA` with a working text editor and a chat app. Currently, we use the [`pycrdt`](https://github.com/y-crdt/pycrdt) package providing Python bindings to the [`Yrs`](https://github.com/y-crdt/y-crdt) library, which holds the Rust implementation of CRDTs originating from the [`Yjs`](https://github.com/yjs/yjs) library. `Yjs` is used in a variety of popular software: [JupyterLab](https://jupyter.org/), Facebook's [Lexical](https://lexical.dev/) editor, [Nextcloud](https://nextcloud.com/) and [many other projects](https://github.com/yjs/yjs?tab=readme-ov-file#who-is-using-yjs). The text-UI is powered by the [`Textual`](https://github.com/Textualize/textual) framework. We intentionally kept the UI minimal to also allow pure keyboard usage. A builtin REPL widget provides the needed configuration flexibility. The project's design concept stands: We provide apps to be directly used and library code for writing own features and apps. The configuration specification as well as the synchronization protocol are also defined. We publish comprehensive documentation under [https://elva.mintgruen.tu-berlin.de](https://elva.mintgruen.tu-berlin.de) alongside a few guides and a blog. Code management with versioning and changelog has been properly set up for ease in long-term maintenance. The source code of `ELVA` is published under the AGPL-3.0 license on [https://github.com/innocampus/elva](https://github.com/innocampus/elva). ### Our Vision We hope to build up a community around this project by showing it to and improving it with other people. As a part of that, we plan to introduce `ELVA` to our students in programming classes. A dedicated app for bulk synchronization is being developed. Additionally, we would like to explore networking without a central server and implement easy to setup peer-to-peer communication protocols to also allow for spontaneous collaboration in meetings or at conferences. Other apps and integrations might be written in other languages than Python thanks to `ELVA`'s integrated networking and the `Yrs` bindings available in several other languages. Another important point is integrating Git into `ELVA` for proper versioning and publishing with services such as [GIN](https://gin.g-node.org/). ### Contact Information Jakob Zahn Head of Software Development Projekt `ELVA` Pronouns: he/his Address: Mr. Technische Universität Berlin innoCampus Room E116 Einsteinufer 19 10587 Berlin Germany Phone: +49 30 314 77006 Mobile: +49 152 06370569 E-Mail: [jakob.zahn@tu-berlin.de](mailto:jakob.zahn@tu-berlin.de) [https://elva.mintgruen.tu-berlin.de](https://elva.mintgruen.tu-berlin.de) [https://www.tu.berlin](https://www.tu.berlin) PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/MDVAHD/ Room 1.20 (Ground Floor, Shannon) Jakob Zahn Tiziano Zito PUBLISH 8U3PB3@@pretalx.com -8U3PB3 How To Accelerate Molecular Insights - Efficient Distance Calculations In Python en en 20250821T160000 20250821T162000 002000 How To Accelerate Molecular Insights - Efficient Distance Calculations In Python The ability to efficiently compute distances between vectorized molecular representations is a backbone of both cheminformatics and bioinformatics. Molecular distance metrics serve as the foundation for a wide range of applications across these fields, with one of their most critical roles being in clustering tasks. Clustering is an essential method in molecular analysis, enabling researchers to identify structural similarities, predict biological activity, and facilitate virtual screening in drug discovery. As the volume of available molecular data continues to grow exponentially, the demand for scalable and computationally efficient methods to process and analyze these vast datasets has never been greater. Traditional approaches to molecular similarity computations often rely on highly optimized, low-level implementations written in languages such as C++ to achieve maximum performance. These solutions leverage hardware-efficient operations and fine-tuned memory management to deliver exceptional computational speeds. However, despite their efficiency, such implementations can pose significant challenges for researchers who lack a strong background in computer science or software development. The complexity of writing and maintaining low-level code can create barriers to entry, making it difficult for scientists to experiment with or customize molecular analysis workflows. In contrast, Python has emerged as a dominant force in the landscape of scientific computing, offering an extensive ecosystem of numerical and data-processing libraries, such as NumPy, SciPy, and scikit-learn. The language’s simplicity, readability, and rich functionality make it an attractive alternative for researchers seeking to implement computational methods without delving into the intricacies of low-level programming. Despite historical concerns about Python’s execution speed compared to compiled languages, recent advancements in just-in-time (JIT) compilation, vectorized operations, and parallel computing have significantly narrowed the performance gap. In this work, we explore how Python’s modern computational capabilities can be harnessed to efficiently compute molecular distances while maintaining accessibility and usability. We focus on vectorized molecular representations, such as binary and count fingerprints, and incorporate sparse matrix representations to handle large molecular datasets efficiently. Sparse matrices enable us to store and process only the non-zero elements in molecular representations, dramatically reducing memory consumption and improving computation times for large-scale analyses. By leveraging bulk calculations and optimized numerical routines, we demonstrate that Python-based implementations can achieve near-C++ performance. Through careful optimization strategies, including the use of NumPy’s broadcasting, we show that Python can handle the challenges of large molecular datasets effectively, maintaining a balance between performance and accessibility. Beyond implementation, we conduct a benchmarking study to evaluate the performance of our optimized Python-based methods. We compare them against state-of-the-art C++ libraries specifically designed for molecular similarity computations, assessing key factors such as computational efficiency, memory consumption, and scalability when applied to large molecular datasets. Our results indicate that, with appropriate optimizations, Python-based approaches can serve as practical alternatives, achieving a balance between performance, usability, and accessibility. We highlight the trade-offs involved, demonstrating how Python’s versatility enables efficient molecular distance computations without sacrificing interpretability or ease of integration within broader data analysis pipelines. By showcasing the feasibility of high-performance molecular similarity computations in Python, our work lowers the barrier to entry for researchers and practitioners who may not have extensive experience with lower-level programming languages. This contribution enhances the accessibility of advanced molecular informatics tools, fostering broader adoption and enabling a wider range of scientists to leverage these computational techniques in their research. Ultimately, this work paves the way for more inclusive and reproducible computational chemistry and bioinformatics, empowering researchers across disciplines to engage with large-scale molecular data analysis using modern, user-friendly methodologies. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/8U3PB3/ Room 1.20 (Ground Floor, Shannon) Adam Staniszewski PUBLISH JACK7B@@pretalx.com -JACK7B The BrainGlobe initiative - image analysis in a common coordinate space. en en 20250821T163000 20250821T165000 002000 The BrainGlobe initiative - image analysis in a common coordinate space. The [BrainGlobe](https://brainglobe.info) initiative has three main goals: providing specific tools for analysis and visualisation, cultivating core tools to facilitate development of interoperable tools in Python, and fostering a community of neuroscientists and developers to share knowledge, build software, and engage with the scientific and open source community. The development of BrainGlobe builds upon and relies heavily on established packages from the broader scientific Python ecosystem ensuring compatibility and interoperability with other tools. Our initiative addresses the crucial need for interoperability in neuroscience research by offering a comprehensive suite of tools accessible to users across different platforms. With a focus on ease of installation and usability, our goal is to empower researchers to analyse neuroanatomical data efficiently and introduce them to the broader scientific python ecosystem. In this talk I plan to discuss the benefits of working in a common coordinate space for image analysis and visualisation. I will begin by introducing the concept of a BrainGlobe atlas and the associated [brainglobe-atlasapi](https://github.com/brainglobe/brainglobe-atlasapi) package. This standard access point has enabled the emergence of an ecosystem of BrainGlobe tools, developed both internally by the core BrainGlobe team, and externally by outside contributors. Abstracting the concept of an atlas and standardising access allows the downstream tools to be species agnostic, widening the potential user pool. I will then describe [brainreg](https://github.com/brainglobe/brainreg) and [brainglobe-registration](https://github.com/brainglobe/brainglobe-registration), the tools we use to register data into a BrainGlobe atlas. I will provide two examples of how our tools utilise BrainGlobe atlases to provide valuable context, based on the annotations of the atlas. The first example will be [brainmapper](https://github.com/brainglobe/brainglobe-workflows), a pipeline that utilises [brainreg](https://github.com/brainglobe/brainreg) and [cellfinder](https://github.com/brainglobe/cellfinder) to detect cells in large 3D volumes and output counts per anatomical region. The second example involves [brainglobe-segmentation](https://github.com/brainglobe/brainglobe-segmentation) which can be used to segment objects, and transform segmentations into sample or atlas space. Lastly, I will demonstrate visualising registered multi-modal data in 3D using [brainrender](https://github.com/brainglobe/brainrender). At present, the BrainGlobe team maintains 17 packages which have 100+ code contributors. The [BrainGlobe Atlas API](https://github.com/brainglobe/brainglobe-atlasapi) serves as a standardised framework for working with anatomical reference atlases, facilitating comparison across samples [1]. Using [brainreg](https://github.com/brainglobe/brainreg), 3D whole-brain imaging data can be registered to any BrainGlobe atlas [2]. [Cellfinder](https://github.com/brainglobe/cellfinder) automates cell detection in large 3D images in a computationally efficient manner [3], while common neuroanatomical segmentation issues are tackled with [brainglobe-segmentation](https://github.com/brainglobe/brainglobe-segmentation) [2]. [Brainrender](https://github.com/brainglobe/brainrender) uses [vedo](https://github.com/marcomusy/vedo) to enable visualisation of 3D neuroanatomical data from both public sources and user-generated data [4, 5]. Our team is continually working on addressing the needs of the community. Currently, efforts are underway to broaden the types of microscopy data registrable into the BrainGlobe ecosystem with [brainglobe-registration](https://github.com/brainglobe/brainglobe-registration), using elastix to register 2D slices, 3D sub-volumes, and whole brain data. Additionally, we are developing [brainglobe-stitch](https://github.com/brainglobe/brainglobe-stitch), a package for fusing large tiled 3D imaging datasets (300+ GB). This package will be available as a napari plugin to allow efficient previewing of the fused dataset. Lastly, we are porting the functionality of brainrender to a napari plugin, [brainrender-napari](https://github.com/brainglobe/brainrender-napari), to provide a cohesive analysis and visualisation environment for all BrainGlobe tools. Concurrently, we are streamlining and enhancing the developer experience. This involves consolidating related repositories within the BrainGlobe codebase and extracting duplicated code to [brainglobe-utils](https://github.com/brainglobe/brainglobe-utils), a shared library. We are also intensifying efforts to improve docstring coverage and providing introductory guides for new developers, along with a development roadmap outlining planned future work. Citations: [1] F. Claudi, L. Petrucco, A. Tyson, T. Branco, T. Margrie, and R. Portugues, ‘BrainGlobe Atlas API: a common interface for neuroanatomical atlases’, J. Open Source Softw., vol. 5, no. 54, p. 2668, Oct. 2020, doi: 10.21105/joss.02668. [2] A. L. Tyson et al., ‘Accurate determination of marker location within whole-brain microscopy images’, Sci. Rep., vol. 12, no. 1, p. 867, Dec. 2022, doi: 10.1038/s41598-021-04676-9. [3] A. L. Tyson et al., ‘A deep learning algorithm for 3D cell detection in whole mouse brain image datasets’, PLOS Comput. Biol., vol. 17, no. 5, p. e1009074, May 2021, doi: 10.1371/journal.pcbi.1009074. [4] F. Claudi, A. L. Tyson, L. Petrucco, T. W. Margrie, R. Portugues, and T. Branco, ‘Visualizing anatomically registered data with brainrender’, eLife, vol. 10, p. e65751, Mar. 2021, doi: 10.7554/eLife.65751. [5] M. Musy et al., "vedo, a python module for scientific analysis and visualization of 3D objects and point clouds", Zenodo, 2021, doi: 10.5281/zenodo.7019968. PUBLIC CONFIRMED Talk (15 mins + Q&A) https://pretalx.com/euroscipy-2025/talk/JACK7B/ Room 1.20 (Ground Floor, Shannon) Igor Tatarnikov PUBLISH NHKMDP@@pretalx.com -NHKMDP Standardised Quantity/Unit APIs discussion en en 20250821T160000 20250821T164500 004500 Standardised Quantity/Unit APIs discussion https://github.com/quantity-dev/metrology-apis is a cross-ecosystem effort to standardise APIs for Metrology in Python. The vision is to have libraries like Pint, `astropy.units`, and Unyt all adopt common APIs which consumers can then use in a ‘backend-agnostic’ fashion. Come along to learn about the current design prototypes and proposals, and voice your ideas for what these APIs should look like! PUBLIC CONFIRMED Maintainer track https://pretalx.com/euroscipy-2025/talk/NHKMDP/ Room 2.41 (First Floor, Turing) Lucas Colley