Test-Time Generalized Category Discovery (TT-GCD), a unified and more realistic scenario where a vision-language model must adapt to distribution shifts, classify known categories using only textual supervision, and discover novel categories, all during test time and without access to labeled data is introduced.
Abstract
Test-Time Adaptation (TTA) and Generalized Category Discovery (GCD) are traditionally treated as disjoint problems: the former adapts models to domain shift assuming all test classes are known, while the latter discovers novel categories assuming labeled training data for known classes. However, real-world deployment rarely fits either setting. Motivated by this gap, we introduce Test-Time Generalized Category Discovery (TT-GCD), a unified and more realistic scenario where a vision-language model must adapt to distribution shifts, classify known categories using only textual supervision, and discover novel categories, all during test time and without access to labeled data. To address this challenging scenario, we propose PACT (Prototype Assignment for Category discovery at Test time), a fully unsupervised framework that casts known-class recognition and novel-class discovery via prototype assignment. PACT first re-aligns shifted visual features with the text-derived class representations of the VLM using confident zero-shot predictions. Known and novel categories are then both represented by prototypes in the visual embedding space, estimated from the unlabeled test stream, and each test image is assigned to the category whose prototype is most similar to its visual feature. Extensive experiments across corruption and domain-shift benchmarks demonstrate that PACT outperforms adapted state-of-the-art TTA and GCD methods, effectively bridging the gap between adaptation and discovery.
Test-Time Discovery (TTD) aims to recognize known classes and detect novel ones from a continuous unlabeled test stream after deployment. Existing TTD methods mainly rely on visual clustering or prototype comparison, and can only assign anonymous labels to discovered classes, leaving them difficult to interpret in open...
Tian-Le Liu, Gong-Bo Cheng, Cheng-Gong Ni et al.· 0 citations
This paper proposes VISRA, a semantic-driven framework for LT-GCD built upon a parametric GCD pipeline that leverages Vision-Language Models to provide additional semantic supervision for representative novel-class instances and guide visual representation learning.
Yi-Zhong Zhang, Hong-Zhe Shou, Guan-Yu Lu et al.· Proceedings of the 35th ACM...· 0 citations
Generalized Category Discovery (GCD) aims to perform old-category recognition and novel-category discovery in the same unlabeled set. Early methods address both objectives through clustering and evaluate the exchangeable predicted indices with global Hungarian (GH) matching. Recent methods adopt trainable parametric he...
Zi-Tai Wang, Qian-Qian Xu, Cong Hua et al.· 0 citations