LLM-based coding agents can navigate codebases, edit files, run tests, and iterate with little supervision. Teams have started applying them to ML projects, where the volume of repetitive experimental work makes automation appealing.
However, coding execution and ML optimization are not the same problem. In software engineering, success is usually local and binary. In ML, code correctness is only a prerequisite: progress is measured across repeated experiments, and getting there requires exploration over many trajectories rather than a single one. Coding agents, by contrast, are built to push one trajectory to completion. Over long horizons they drift. Changes get bundled into one step, so results cannot be attributed. Context is lost between sessions, and hypotheses that were already tested come back in slightly different wording.
Adding a higher-level LLM agent on top to orchestrate the others does not fix the problem. The orchestrating agent drifts too. Karpathy's autoresearch is a recent illustration: a coding agent given an .md prompt spins up its own worktrees and logs experiments to a CSV, but the trajectory it explores stays single and shallow. Stable behavior across hundreds of experiments requires deterministic code, not better prompts.
This talk shows how to build such a system. A non-LLM orchestrator runs experimentation as a tree search over hypothesis-constrained modifications, with each branch living in its own git worktree. Different node types separate proposing a change from implementing it and from debugging it. Results flow into a real experiment tracking system through a token-efficient query interface, so an agent's context can be reset between calls without losing history. The orchestrator dispatches to existing coding CLIs (e.g. Claude Code, Codex) without abstracting them away.
