From Dialogue to Evidence: Retrieval-State-Conditioned Interaction for Text-Based Person Retrieval

Abstract

Interactive text-based person retrieval aims to resolve ambiguity in an underspecified initial query through iterative clarification. However, existing methods largely rely on free-form dialogue, which makes interaction depend on unstructured language generation rather than directly selecting the most discriminative visual evidence among competing candidates. As a result, acquired evidence is only implicitly encoded in dialogue history, remaining uncertainty is not explicitly represented, and generated questions are often weakly grounded in the visual differences that determine ranking. This limitation is further amplified by the long-tailed distribution of visual attributes: frequent attributes are easy to ask about but quickly become weakly discriminative, whereas long-tail attributes are often more informative for disambiguation yet underutilized. To address this, we reformulate interactive retrieval as a retrieval-state-conditioned evidence acquisition problem. We introduce a grounded discrete concept space composed of reusable concept atoms, which provides a shared representation for the initial query, candidate-side evidence, and user feedback. At each interaction round, the system selects the next concept atom conditioned on the current candidate set using a redundancy-aware utility that jointly favors ambiguity reduction, candidate discrimination, and high-quality partitioning of the candidate pool. In our framework, natural language serves only as an interaction interface, while decision-making is performed over structured concept-level evidence. This formulation replaces unconstrained dialogue generation with structured, utility-driven clarification, enabling targeted and interpretable interaction without training additional question-generation or interaction-policy modules. Experiments on Interactive-PEDES under a unified multi-round protocol show that our method consistently outperforms language-centric baselines, delivering faster early-stage refinement and larger cumulative improvements across interaction rounds, with R@1 improving by 31.01% over five rounds.