Skip to content
Preprint

Test-Time Logit Prompting for Source-Free Missing Modality Adaptation

Sep 2026 · 0 citations · 31 references
Computer Science

TL;DR

Test-Time Logit Prompting (TLP), a lightweight source-free test-time adaptation framework for visual recognition with missing modalities, is proposed, which optimizes logit prompts with uncertainty-aware adjustment and modality-complete consistency regularization, adaptively adjusting prediction confidence while preserving semantic consistency.

Abstract

Vision-language models (VLMs) have achieved remarkable performance by leveraging complementary information from large-scale image-text pairs. However, missing-modality inputs are commonly encountered during real-world deployment, often leading to significant performance degradation. Existing methods primarily enhance model robustness by learning modality compensation strategies from source training data. However, their reliance on source training data makes them difficult to apply when original data are unavailable due to privacy, storage, or accessibility constraints, such as clinical applications and personalized AI services. This raises an important yet underexplored question: can VLMs be efficiently adapted at test time for visual recognition with missing modalities without accessing source training data? To this end, we propose Test-Time Logit Prompting (TLP), a lightweight source-free test-time adaptation framework for visual recognition with missing modalities. To address missing-induced prediction shifts, TLP optimizes logit prompts with uncertainty-aware adjustment and modality-complete consistency regularization, adaptively adjusting prediction confidence while preserving semantic consistency. Extensive experiments across diverse vision-language benchmarks demonstrate that TLP consistently enhances recognition performance under missing-modality scenarios, achieving up to 8\% improvements while requiring only hundreds of tunable parameters and a few test-time optimization steps.

View source

Similar papers

Preprint Aug 2026

Towards Purified Multi-Label Test-Time Adaptation of Vision-Language Models

PuRF is introduced, a novel PuRiFication-driven cache-based method for multi-label test-time adaptation of vision-language models that consistently outperforms state-of-the-art methods on ViT-B/32 across five datasets.

Yiwen Liang, Hui Chen, Yizhe Xiong et al. · 0 citations
#machine learning Preprint Oct 2026

Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series

This work proposes CARAT, which decouples model reliance from runtime corruption detection to guide omission or attenuation, amortizing reliance estimation through source training and position CARAT as a practical default to wearable TTA, offering competitive robustness with modest computational requirements and benefi...

Payal Mohapatra, Yue-Yuan Sui, Hao Yang et al. · 0 citations
#machine learning Preprint Sep 2026

Harnessing Image Question Dependence for Better VLM Test-time Reinforcement Learning

The proposed TTIQ, a test-time reinforcement learning framework that harnesses image-question dependence for better vlm adaptation, combines image and question dependence with calibrated confidence to construct a response-level reward that favors jointly grounded responses.

Xinrui He, Ting-Wei Li, Jun-Ting Wang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Do More Modalities Always Help? A Geometric Perspective on Missing-Modality Robustness

Geodesic Unlearning (GU), a lightweight parameter-editing method that leverages Grassmannian subspace geometry for structured subspace correction to improve missing-modality robustness, is proposed and suggested as a practical route for robustness correction.

Song-Yuan Sui, Zhen Tan, Mohan Zhang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Measure Less, Know More: Self-Supervised Test-Time Feature Acquisition

ECHO-$k$ is introduced, a task-agnostic and self-supervised learning principle for modality acquisition that uses a deep model's internal pretrained representations as proxy targets that summarize cross-modal information and consistently improves budgeted downstream performance across diverse foundation-model backends.

Eeshaan Jain, L. Bleistein, B. Deplancke et al. · 0 citations
#machine learning Preprint Sep 2026

Representation Editing for Multimodal Test-Time Adaptation

This work proposes FourIer Representation Editor (FIRE), a novel multimodal TTA approach that directly edits semantically rich intermediate representations, and demonstrates the superiority of FIRE over existing multimodal TTA methods.

Long-Fei Huang, Xiang-Yu Wu, Yang Yang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.