Beyond One Granularity: Spectral Ensemble and Consensus Expansion for Training-Free Composed Image Retrieval
Abstract
Composed image retrieval (CIR) enables expressive multimodal search: given a reference image and a text modifier, the goal is to retrieve images that match the visual content as altered by the text. Building effective CIR systems has historically required expensive triplet-annotated training data, limiting scalability. Training-free CIR sidesteps this bottleneck by repurposing pre-trained vision-language models to compose queries without any task-specific supervision. However, existing training-free methods commit to a single, globally fixed projection granularity when aligning visual and textual features. This creates an inherent trade-off: coarse granularities capture broad categorical distinctions but lose fine-grained detail, while fine granularities recover instance-level attributes but amplify noise. The problem is especially acute in instance-level CIR, where per-query databases vary from fewer than 10 to over 10,000 images, making any fixed setting suboptimal across the evaluation. We therefore propose a unified retrieval method with two complementary components that fuse scores across multiple granularities to capture complementary evidence and yield more robust rankings. MACE (Multi-granularity Adaptive Composition with Expansion) constructs a spectral ensemble from multiple nested truncation levels and fuses their scores without any additional eigendecomposition or database modification. CGCQE (Cross-Granularity Consensus Query Expansion) integrates into MACE to strengthen pseudo-relevance feedback, selecting expansion candidates that rank consistently well across all granularities and preventing granularity-specific false positives from corrupting the expanded query. Experiments on five CIR benchmarks, including the challenging instance-level i-CIR, show that MACE with CGCQE achieves state-of-the-art performance, outperforming both training-free and supervised zero-shot methods.