Efficient Attention Pruning for Resource-Constrained Super-Resolution Using Statistical Thresholds and Quantum-Guided Sparsity

Abstract

Non-local attention improves single-image super-resolution (SISR) by letting every query position aggregate information from all other positions, but the softmax normalisation at its core assigns strictly positive weight to every candidate regardless of relevance, diluting genuinely informative long-range correlations once the candidate set is large. This paper proposes Adaptive Sparse Attention (ASA), a non-local attention mechanism that replaces the fixed softmax support with a perquery threshold τ = μ(s) + α·σ(s) derived from the mean and standard deviation of each query’s own similarity-score distribution, so that only statistically above-average candidates are retained. This further proposes two extensions: SCASA-TT, which compresses the query/key/value projections with a Tensor-Train (TT) factorisation and adds a complementary spatial-coupling sparsity mask, and Quantum ASA, in which a 4-qubit variational quantum circuit (VQC) predicts the threshold coefficient α per query rather than using a fixed constant. All three variants, together with a non-attentive convolutional baseline, share a common backbone and are evaluated using Y-channel PSNR/SSIM on Set5, Set14, BSD100 and Urban100. Quantum ASA attains the best reconstruction quality of every method on every dataset and scale factor tested, with the largest gains observed on Urban100 and Manga109, the two benchmarks richest in repeated structure, consistent with the premise that non-local selfsimilarity is a more exploitable prior precisely where such redundancy is abundant. An analytical hardware-cost model projects a 73.7% reduction in multiply-accumulate operations and a 74.8% reduction in memory bandwidth for ASA-sparsified attention relative to dense attention at matched sequence length.