PyConDE & PyData Berlin 2024

There is a Better Way to Automate and Manage Your (Fluid) Simulations
04-24, 15:20–15:50 (Europe/Berlin), B09

This is a story about applying Python and the “hacker mindset” to Computer Aided Engineering (CAE), an emerging domain within the Python ecosystem. Shell scripts have traditionally been the preferred tool for automating CAE pipelines, especially in subfield of Computational Fluid Dynamics (CFD). However, this approach is brittle, severely limited and cumbersome to manage at scale. Data management is also a challenge, with tens to hundreds of GB per simulation needing to be stored and versioned in complex folder structures. One possible approach is to use Python as an automation and glue language and Data Version Control (DVC) which is a Python based tool built on top of git to track pipelines and data.


This is a story about applying Python and the “hacker mindset” to Computer Aided Engineering (CAE), an emerging domain within the Python ecosystem. Shell scripts have traditionally been the preferred tool for automating CAE pipelines, especially in subfield of Computational Fluid Dynamics (CFD). However, this approach is brittle, severely limited and cumbersome to manage at scale. Data management is also a challenge, with tens to hundreds of GB per simulation needing to be stored and versioned in complex folder structures. One possible approach is to use Python as an automation and glue language and Data Version Control (DVC) which is a Python based tool built on top of git to track pipelines and data.

This talk will show you how to use Python to automate many tasks in CAE workflows, even when the tools don’t offer a native Python interface:
- Exporting CFD simulation results from Starccm+ to a PowerPoint template with python-pptx and updating the final presentation with new simulation data
- Preparing input data for an electrical thermal simulation to improve performance 80-fold

Both examples will illustrate best practices and lessons learned in the automation of the CFD software that are applicable beyond the field.

DVC was originally designed and is broadly used for machine learning pipelines, but its flexibility allows it to be adapted to other domains. The potential benefits for engineering applications are immense. This talk will show you how easy it is to convert an existing CAE pipeline to DVC and show the benefits:
- Running hundreds of simulations, comparing them and choosing the optimal with DVC
- Managing software versions declaratively and comparing results across versions
- Creating in-depth meta studies and comparing many simulations with Jupyter notebooks

Finally, this talk will give an outlook on the changing CAE ecosystem and propose new features for DVC to better leverage it for this use case.

Audience
Either simulation engineers seeking to enhance and scale their workflows or software engineers aiming to build powerful and flexible simulation tooling.

Relevant talks or blog posts
- Sending Rovers to Mars with Jupyter
- Managing OpenFOAM Physical Simulations with DVC, CML, and Studio
- How Python enables future computer chips


Expected audience expertise: Domain

Intermediate

Expected audience expertise: Python

Novice

Abstract as a tweet (X) or toot (Mastodon)

Exploring the integration of Python into Computer Aided Engineering (CAE) workflows: While shell scripts are ubiquitous, they face challenges in CAE, particularly in Computational Fluid Dynamics (CFD). Python + DVC provides a robust alternative to manage simulations at scale.

I am a simulation engineer developing sustainable battery system platforms at Volkswagen. I have been working on battery systems since 2016, starting in Formula Student. There, I created an open source library for analyzing vehicle control unit data with a Pandas-like interface called CANdas. I hold a master's degree in mechanical engineering with a strong focus on simulation and data management. I am always on the lookout to improve simulation workflows using Python.