Adaptive Token-Aware Query Reformulation for Text-to-Image Retrieval
TL;DR
This work proposes a retrieval-time token-aware query reformulation method that emphasizes informative query tokens without retraining or changes to the retriever architecture and constructs a focused query embedding by pooling salient tokens while preserving global semantic context.
Abstract
Text-to-image retrieval (T2IR) aims to retrieve semantically relevant images given natural language queries. However, existing systems typically adopt static retrieval pipelines that process all queries uniformly, despite differences in the informativeness of individual query tokens. To address this limitation, we propose a retrieval-time token-aware query reformulation method that emphasizes informative query tokens without retraining or changes to the retriever architecture. Our method uses token-level routing to determine when query adaptation should be applied and constructs a focused query embedding by pooling salient tokens while preserving global semantic context. The reformulated query representation is used for first-stage retrieval, and the retrieved candidates are subsequently reranked. Experiments on MS COCO and Flickr30k show modest overall gains over the corresponding base pipeline, while also providing interpretable token-level evidence and region-level visualizations. Overall, our results show that retrieval-time token-aware query reformulation improves existing T2IR pipelines while making the role of individual query tokens more explicit.