2026-08-14 –, Tent — RW1
DyadAgent is an AI coding assistant for generating and debugging Dyad code across modelling and simulation workflows, enabling engineers to express complex model requirements in natural language. Evaluating the performance of such an agent requires verification of generated simulation results against standards of physical correctness and numerical accuracy. DyadAgentBench is an evaluation infrastructure designed to measure the agent's modelling and simulation capabilities in a systematic and reproducible manner. In this talk, we present the infrastructure and evaluation framework developed for DyadAgent, covering how agent performance is assessed and how the resulting insights are used to benchmark and guide iterative improvements.
We will demonstrate the infrastructure that enables the evaluation of the agent's physical modelling capabilities and the correctness of its numerical simulations. In this talk, we will also demonstrate how we track incremental updates to the agent and measure their cumulative impact on the agent's capabilities.
Software Engineer in Dyad AI team at JuliaHub.