No-reference image quality assessment via multihead cross-scale attention fusion and contrastive ranking loss
TL;DR
Three targeted enhancements to No-Reference Image Quality Assessment show state-of-the-art results on all three synthetic benchmarks and reveal configuration-dependent patterns on authentic-distortion data.
Abstract
No-Reference Image Quality Assessment (NR-IQA) must predict perceptual quality scores directly from distorted images without access to any pristine reference, making it substantially more challenging than full-reference approaches. While Transformer-based frameworks have recently advanced the state of the art, existing methods still leave room for improvement in cross-layer multi-scale feature interaction, the density of ranking supervision extracted from each training batch, and the spatial adaptability of the attention mechanism. Taking a code-reproduced TReS as the baseline, this paper introduces three targeted enhancements. First, a Multi-Head Cross-Scale Attention Fusion (MCSAF) module applies multihead self-attention to the concatenated feature sequences from different ResNet-50 layers, allowing each scale’s contribution to adapt to the input content instead of being determined by naive concatenation. Second, an Advanced Contrastive Ranking Loss extends TReS’s triplet loss—which exploits only the extreme-quality sample pair within each batch—into a joint formulation comprising all-pair dynamic-margin ranking, hard negative mining, and temperature-scaled InfoNCE contrastive learning; this raises the effective supervision density from 𝑂(1) to 𝑂(𝐵 2 ) with no additional inference cost. Third, a Deformable Attention mechanism with Learnable Position Encoding replaces fixed sinusoidal codes with end-to-end optimizable row-column embeddings and uses a lightweight offset MLP to predict 𝐾 sampling points per query, reducing attention complexity from 𝑂(𝑁2 ) to 𝑂(𝑁𝑀𝐾). Experiments on LIVE, CSIQ, TID2013, LIVEC, and KonIQ-10K show state-of-the-art results on all three synthetic benchmarks (CSIQ SROCC/PLCC: 0.9465/0.9593, +4.0%/+2.8% over baseline) and reveal configuration-dependent patterns on authentic-distortion data. Ablation studies consistently identify the Advanced Contrastive Ranking Loss as the most universally effective component.