Automated numerical verification of quantum physics papers using Julia and LLM agents
Tobias J. Osborne
The explosion in the number of submitted quantum physics papers is placing the scientific publication system under extreme stress. In an effort to address this challenge, I have been experimenting with end-to-end pipelines for the automated numerical verification of claims in quantum physics papers. Given an arXiv preprint, LLM coding agents extract mathematical assertions into a structured knowledge graph, then generate Julia code to numerically check each claim. I will demonstrate this pipeline across three domains: topologically ordered many-body quantum systems (using, e.g., TensorCategories.jl), quantum information theory, and quantum optics (using QuantumOptics.jl). In practice, this system routinely catches small sign errors and gaps in every paper I have studied. LLM coding agents enable automation of workflows that previously would have involved much yak shaving and frustration. As a consequence, time-poor persons with little software engineering expertise (such as myself!) can now employ the amazing cutting-edge tools the Julia community have developed. I will directly address practical challenges including LLM hallucinations in mathematical reasoning, ensuring correctness of generated code, and the gap between symbolic assertions and finite-dimensional numerics.