2026-08-13 –, Muschel — N3 All times in Europe/Berlin
While Julia’s Dagger.jl provides a productive framework for task-based parallelism using Directed Acyclic Graphs (DAGs), its default reliance on TCP-based Distributed.jl limits performance on low-latency HPC interconnects. To bridge this gap, we developed MPIAcceleration, a strategic extension that replaces standard transport with an MPI-aware backend. By leveraging MPI.jl and non-blocking communication, we enable Dagger to use specialized hardware such as InfiniBand and Slingshot while maintaining a simple, high-level API.
Dagger's MPIAcceleration works seamlessly with the current scheduler, allowing task graphs to be executed across MPI ranks with minimal modifications. It only requires a single line of code: Dagger.accelerate!(:mpi).
When this feature is enabled, each MPI rank is integrated into Dagger's Processor/Memory Space model, which ensures that tasks are executed close to where their data resides. This rank-aware placement helps to minimize communication overhead. Additionally, remote data transfers happen transparently, providing handles on the appropriate ranks.
Yan Guimarães is a Software Engineering student at the University of Brasília (UnB) and a contributor to Dagger.jl whose work focuses on high-performance computing and distributed task scheduling. As a Google Summer of Code 2025 contributor with MIT’s JuliaLab, he designed and implemented an MPI-based backend for Dagger’s DAG scheduler, improving its ability to run efficiently on HPC systems. He evaluated this work on the Aurora exascale supercomputer at Argonne National Laboratory, demonstrating the performance benefits of MPI-based communication for distributed linear algebra workloads. Yan is the first author of “Productive Scalable Distributed Task Scheduling Using an MPI-based Backend,” which he presented at the ACM Student Research Competition at SC25. His broader research interests include scalable runtime systems, distributed computing, JIT compilation, and MLIR-based compiler optimizations for high-performance computing.
Consultant at MIT's JuliaLab, Co-maintainer of Dagger. My interests span from more broad topics such as the accessibility and educational initiatives for parallel computing to Applied Physics and Numerical Linear Algebra.