Nov 2026· Proceedings of the 34th ACM International Conference on Multimedia· pp. 1861-1869· 0 citations· 41 references
TL;DR
PAIR (Prototype-Aware Inertia Rectification) is proposed, a unified framework that reduces seen-category bias from both local and global perspectives and provides a more balanced basis for open-vocabulary prediction.
Abstract
Open-vocabulary multi-label recognition (OV-MLR) aims to predict multiple labels beyond seen training categories by transferring knowledge from vision-language models. Existing methods typically rely on either raw patch-level matching or a single global token, which makes them vulnerable to seen-category bias under seen-only supervision. In particular, patch features tend to collapse onto a few competitive seen categories, causing predictions to be dominated by seen-category responses, and limiting generalization to unseen labels. To address this issue, we propose PAIR (Prototype-Aware Inertia Rectification), a unified framework that reduces seen-category bias from both local and global perspectives. First, PURE (Prototype Recovery with Uncertainty-aware Region Enhancement) reorganizes patch tokens into teacher-guided latent prototypes, and refines them via transport-based prototype recovery, yielding more transferable local representations. Second, AIR (Alignment Inertia Rectification) builds a prototype-conditioned global representation, and rectifies global alignment through hard-negative shaping and local-to-global consistency regularization. By jointly rectifying local prototype learning and global representation alignment, PAIR provides a more balanced basis for open-vocabulary prediction. Extensive experiments on MS-COCO and NUS-WIDE show that PAIR consistently outperforms the strong OV-MLR baseline, demonstrating its effectiveness across both benchmarks.
Existing vision-language models (VLMs) exhibit strong zero-shot generalization, yet remain fundamentally limited in Unsupervised Multi-label Image Classification (UMIC), primarily due to their inadequate multi-object perception and the intrinsic modality gap between vision and language. In particular, global representa...
Xi-Li Deng, Yi Zhao, Nan-Qing Liu et al.· Neural Networks· 0 citations
Open-Vocabulary Multi-Label Recognition (OV-MLR) aims to identify all queried semantic concepts present in an image. Existing methods mainly improve global image-text matching or text-side representations, leaving local prediction largely under-explored. In this paper, we revisit local prediction and reveal that it can...
Zhuo-Ming Li, Bo Han, Xiao-Yu Wang et al.· 0 citations
Zero-Shot Learning (ZSL) has achieved remarkable progress, yet most existing methods still rely heavily on predefined semantic embeddings as supervisory signals. Such semantics are often incomplete, inaccurate, and insufficiently discriminative, resulting in ambiguous decision boundaries and limited generalization to u...
Bo-Cheng Ren, Yuan-Yuan Yi, Qing-Chen Zhang et al.· IEEE Transactions on Pattern...· 0 citations
This paper proposes VISRA, a semantic-driven framework for LT-GCD built upon a parametric GCD pipeline that leverages Vision-Language Models to provide additional semantic supervision for representative novel-class instances and guide visual representation learning.
Yi-Zhong Zhang, Hong-Zhe Shou, Guan-Yu Lu et al.· Proceedings of the 35th ACM...· 0 citations
Open-vocabulary object detection (OVD) expects the model trained on base classes to recognize both base and novel classes at inference. Typical methods rely on CLIP-style vision-language embeddings, yet the learned space is often biased toward base categories, with limited effective capacity preserved for novel concept...
Wei Zhang, Zhu Teng, Zhixiang Chi et al.· IEEE transactions on multime...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.