Spatial sliding-window attention for learning solution operators of partial differential equations

Abstract

This paper studies a spatial sliding-window attention mechanism for learning solution operators of partial differential equations in a data-driven setting. The mechanism is formulated as an explicit spatial-locality bias that retains only nearby interactions and avoids materializing the full spatial score matrix. We derive an output perturbation bound governed by the discarded off-window score energy and establish a separate conditional residual-stability estimate under a discrete inf–sup assumption. Experiments on the one-dimensional Burgers equation and the two-dimensional Navier–Stokes equations compare sliding-window attention with an explicit dense spatial-score control and established operator-learning baselines. Separate fixed-weight forward and gradient tests confirm that the explicit dense and standard reassociated evaluations of unmasked Galerkin attention agree to floating-point precision. A two-dimensional feasibility test further shows that the standard reassociated implementation runs at both tested resolutions. Overall, the results reveal a problem- and resolution-dependent trade-off between prediction accuracy and computational resource utilization. The sliding-window mechanism is thus best interpreted as a controllable spatial-locality mechanism rather than a uniformly more efficient replacement for standard Galerkin attention.