Agentic AI for gravitational wave data analysis: a head-to-head comparison of coding agents executing a matched filter pipeline on Einstein Telescope simulated data
TL;DR
Two state-of-the-art systems, Claude Code (Anthropic) and Codex (OpenAI), were tasked with autonomously executing the same simple end-to-end pipeline on Einstein Telescope simulated data, on a shared computing infrastructure and without human intervention, but the agents exhibited substantially different behaviors and computational costs.
Abstract
We report a methodological study of agentic AI in gravitational-wave data analysis: two state-of-the-art systems, Claude Code (Anthropic) and Codex (OpenAI), were tasked with autonomously executing the same simple end-to-end pipeline on Einstein Telescope (ET) simulated data, on a shared computing infrastructure and without human intervention. The object of study is the behavior, reliability and auditability of the agents, not the physics output of the pipeline, which serves as a controlled test case. The pipeline comprises power spectral density estimation from raw ET simulated noise, geometric template bank generation using IMRPhenomD waveforms, matched filter recovery of 100 binary black hole signal injections, automated results generation, and large language model-assisted production of a LaTeX manuscript formatted in the style of a Physical Review D paper. Both agents received identical written specifications and identical compute resources. The experiment was run twice: a first run with unrealistically loud injections, and a second run with signals rescaled to a physically motivated SNR range. The scientific results converged in both runs in the sense that both systems achieved comparable detection efficiency and template bank size. However, the agents exhibited substantially different behaviors and computational costs: Claude Code completed the pipeline in ∼3.4 min with silent deviations from the specification, while Codex required ∼16 min across explicit self-correcting restarts, including an unsolicited performance optimization of the matched filter inner loop. The autonomously generated manuscripts also diverged substantially in length, details, and quality. In the second run, a subtle difference in the interpretation of the SNR range instruction led to a genuine scientific divergence: Claude Code silently shifted the SNR floor to ρ=8 (100% efficiency), while Codex followed the specification literally, targeting SNRs down to ρ=7 and recording one missed detection. We discuss the implications of these behavioral differences, such as speed versus auditability, silent deviations versus explicit self-correction, instruction interpretation, and the criticality of intermediate data representations in multi-model pipelines, for the deployment of agentic AI in scientific computing workflows, within the limits of a single-pipeline, two-run benchmark.
