Skip to content
Preprint

Rate-distortion optimization for full-reference image quality metrics via stochastic Hessian estimates

Sep 2026 · 0 citations · 31 references
Engineering

TL;DR

This work approximate a broad class of FR-IQA metrics by an input-dependent quadratic distortion (IDQD), whose quadratic form matrix is derived from the Hessian of the metric evaluated at the source video.

Abstract

Block-based video codecs select coding parameters based on the input by optimizing a rate-distortion trade-off. The conventional distortion choice, the sum of squared errors (SSE), simplifies parameter selection: the SSE is the sum of block-wise SSEs, so rate-distortion optimization (RDO) can treat blocks independently. Alternatively, full-reference image quality assessment (FR-IQA) metrics such as MS-SSIM or LPIPS often align better with the human visual system than SSE, but they cannot be used in-loop: they do not decompose block-wise and typically require the fully decoded image as input. Building on existing results in metric quadratization, we approximate a broad class of FR-IQA metrics by an input-dependent quadratic distortion (IDQD), whose quadratic form matrix is derived from the Hessian of the metric evaluated at the source video. To make the distortion computable block-wise, we propose two approximations of the Hessian matrix: 1) keeping the block-diagonal, and 2) keeping only its diagonal. We propose estimators for both that require only matrix-vector products with the Hessian obtained by automatic differentiation. Across five metrics for Kodak and CLIC in VVC, IDQD-RDO achieves 14.2-36.7 % BD-rate savings under the target metric with no decoder changes and incurs 10-30 % encoding complexity overhead.

View source

Similar papers

Review Open access 2025

A Comprehensive Framework for Perceptual Quality Assessment in AI-Based Image Compression

Perceptual quality assessment has become the central bottleneck in the design, evaluation, and deployment of artificial-intelligence-based image compression systems. Classical signal-fidelity measures such as mean squared error (MSE) and peak signal-to-noise ratio (PSNR) correlate poorly with human judgment. Learned co...

Mahesh Boddu, Soumitra Kumar Mandal · 0 citations
Oct 2026

Optimizing UVQ for scaling video quality assessment

UVQ (Universal Video Quality) is an ML-based metric that offers state-of-the-art accuracy for no-reference video quality assessment. This work outlines the enhancements made to the UVQ-stack enabling quality assessment in a large scale. Our improvements span three complementary axes: system efficiency, model efficiency...

Joong Gon Yim, Armen Abnousi, Qi Cai et al. · 0 citations
Preprint Oct 2026

PerQ: Inverse Generative Modeling for Neural Image Compression via Quantization Error Compensation

Neural image codecs face a fundamental trade-off between pixel fidelity and perceptual quality. Distortion-optimized codecs preserve pixel-wise fidelity but produce blurred reconstructions at low bitrates, whereas generative codecs leverage learned priors to improve perceptual quality but tend to exhibit saturating PSN...

Tian-Hao Peng, Ho-Man Kwan, Fan Zhang et al. · 0 citations
Sep 2026

Only What Was Seen: Observation-Gram Compaction of View-Dependent Appearance in 3D Gaussian Splatting

Most of the memory of a 3D Gaussian Splatting model holds spherical-harmonic colour coefficients, yet each Gaussian is seen only from the narrow cone of directions of the training cameras. We turn this into a distortion metric that other compressors can adopt: a per-Gaussian observation Gram matrix, accumulated from vi...

Krzysztof Pietroszek · 0 citations
Preprint Nov 2026

Towards High-Fidelity Learned Video Compression

Professional video workflows operate at qualities that learned video codecs do not reach. We measure how much of a professional quality range a codec covers, from the lowest quality of the standard test conditions, VVC at QP 37, to the highest 4:2:2 grade of the production codec ProRes. To compare inter-frame coding al...

Bumyoon Kim, Yongseong Kim, Inhyuk Jeong et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.