Skip to content
Book Open access

PAIR: Prototype-Aware Inertia Rectification for Open-Vocabulary Multi-Label Recognition

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · pp. 1861-1869 · 0 citations · 41 references

TL;DR

PAIR (Prototype-Aware Inertia Rectification) is proposed, a unified framework that reduces seen-category bias from both local and global perspectives and provides a more balanced basis for open-vocabulary prediction.

Abstract

Open-vocabulary multi-label recognition (OV-MLR) aims to predict multiple labels beyond seen training categories by transferring knowledge from vision-language models. Existing methods typically rely on either raw patch-level matching or a single global token, which makes them vulnerable to seen-category bias under seen-only supervision. In particular, patch features tend to collapse onto a few competitive seen categories, causing predictions to be dominated by seen-category responses, and limiting generalization to unseen labels. To address this issue, we propose PAIR (Prototype-Aware Inertia Rectification), a unified framework that reduces seen-category bias from both local and global perspectives. First, PURE (Prototype Recovery with Uncertainty-aware Region Enhancement) reorganizes patch tokens into teacher-guided latent prototypes, and refines them via transport-based prototype recovery, yielding more transferable local representations. Second, AIR (Alignment Inertia Rectification) builds a prototype-conditioned global representation, and rectifies global alignment through hard-negative shaping and local-to-global consistency regularization. By jointly rectifying local prototype learning and global representation alignment, PAIR provides a more balanced basis for open-vocabulary prediction. Extensive experiments on MS-COCO and NUS-WIDE show that PAIR consistently outperforms the strong OV-MLR baseline, demonstrating its effectiveness across both benchmarks.

Read PDF

Similar papers

Nov 2026

ClassifierAlign: Unsupervised multi-label image classification via training-free vision-language classifier alignment.

Existing vision-language models (VLMs) exhibit strong zero-shot generalization, yet remain fundamentally limited in Unsupervised Multi-label Image Classification (UMIC), primarily due to their inadequate multi-object perception and the intrinsic modality gap between vision and language. In particular, global representa...

Xi-Li Deng, Yi Zhao, Nan-Qing Liu et al. · 0 citations
#artificial intelligence Preprint Nov 2026

LoCo: Selective Local Competition for Discriminative Open-Vocabulary Multi-Label Recognition

Open-Vocabulary Multi-Label Recognition (OV-MLR) aims to identify all queried semantic concepts present in an image. Existing methods mainly improve global image-text matching or text-side representations, leaving local prediction largely under-explored. In this paper, we revisit local prediction and reveal that it can...

Zhuo-Ming Li, Bo Han, Xiao-Yu Wang et al. · 0 citations
Oct 2026

Prototype Matters: Domain-Wise Prototype Induction for Zero-Shot Recognition.

Zero-Shot Learning (ZSL) has achieved remarkable progress, yet most existing methods still rely heavily on predefined semantic embeddings as supervisory signals. Such semantics are often incomplete, inaccurate, and insufficiently discriminative, resulting in ambiguous decision boundaries and limited generalization to u...

Bo-Cheng Ren, Yuan-Yuan Yi, Qing-Chen Zhang et al. · 0 citations
Book Open access Nov 2026

VISRA: Vision-Language Induced Supervision and Representation Alignment for Long-Tailed Generalized Category Discovery

This paper proposes VISRA, a semantic-driven framework for LT-GCD built upon a parametric GCD pipeline that leverages Vision-Language Models to provide additional semantic supervision for representative novel-class instances and guide visual representation learning.

Yi-Zhong Zhang, Hong-Zhe Shou, Guan-Yu Lu et al. · 0 citations
Open access Nov 2026

Partition, Refine, and Detect: Preserving Novel Category Capacity in Open-Vocabulary Object Detection

Open-vocabulary object detection (OVD) expects the model trained on base classes to recognize both base and novel classes at inference. Typical methods rely on CLIP-style vision-language embeddings, yet the learned space is often biased toward base categories, with limited effective capacity preserved for novel concept...

Wei Zhang, Zhu Teng, Zhixiang Chi et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.