ISER: Instance-Specific Early Stopping with Dynamic Low-Rank Adaptation for Learned Image Compression

Abstract

Image compression has evolved from human-centric perceptual coding toward support for diverse machine vision applications, requiring modern codecs to serve both human viewing and downstream tasks in closed-set settings (where target tasks are incorporated during training) and open-set settings (where previously unseen tasks arise at test time). While recent learned compression methods jointly optimize perceptual quality and closed-set task performance, they often fail to generalize to unseen open-set tasks due to fixed training assumptions and objectives. Our prior work, LoRA-comp (Low-Rank Adaptation Compression), effectively addresses open-set challenges via instance-specific test-time fine-tuning (TTFT) without requiring task-specific pre-training. Nevertheless, its fixed LoRA architecture, which assigns a uniform rank across all layers, often leads to suboptimal instance-level performance. Moreover, allocating the same number of training epochs to every instance introduces unnecessary encoding-time overhead. To address these challenges, we propose Instance-Specific Early Stopping with Dynamic Rank Adaptation (ISER), which extends LoRA-comp. Building upon the LoRA-comp–based instance-specific adaptation framework, ISER introduces (i) instance-specific early stopping (ISES) combined with a multi-scale training strategy (MSTS) to reduce TTFT overhead and (ii) instance-specific dynamic rank adaptation (ISRA) to tailor the LoRA architecture per instance. Experiments demonstrate that ISER consistently outperforms competing methods. Compared to LoRA-comp, ISER achieves up to a 7% BD-Rate improvement and up to a 44% reduction in encoding time. Moreover, ISER achieves up to a 98% relative improvement in BD-Rate gain and up to a 19.2% reduction in decoding time over TransTIC.