Sep 2026· Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence· 0 citations· 23 references
TL;DR
This work proposes Snap2Review, a novel framework that harmonizes cross-modal retrieval with fine-grained preference learning, and introduces Tri-DPO, which employs a stratified negative sampling strategy with varying difficulty levels to force the model to discern fine-grained user preferences.
Abstract
Personalized review generation is vital for e-commerce engagement, yet integrating user-provided images while maintaining distinct user personas remains a critical challenge. Existing methods often struggle to bridge the semantic gap between objective visual signals and subjective linguistic patterns, frequently resulting in hallucinations or homogenized, impersonal content. To address these limitations, we propose Snap2Review, a novel framework that harmonizes cross-modal retrieval with fine-grained preference learning. First, we optimize a cross-modal retriever that maps visual features directly to semantically related historical reviews, using these interactions as precise anchors to strictly ground the generation. Second, we introduce Tri-DPO, which employs a stratified negative sampling strategy with varying difficulty levels, ranging from obvious errors to subtle stylistic mismatches, to force the model to discern fine-grained user preferences. By distinguishing these fine-grained nuances, our model moves beyond generic praise to accurately mimic the user's authentic writing style. Extensive experiments on real-world datasets demonstrate that Snap2Review significantly outperforms strong baselines in both relevance and personalization metrics.
Conversational Image Retrieval (CIR) refines image search through multi-turn interaction, where the Questioner plays a central role in eliciting information about the user’s target. However, existing CIR systems are commonly evaluated in end-to-end settings, making it difficult to determine whether performance gains originate from questioning strategies, retrieval backbones, answering behavior, or interaction protocols. We introduce a controlled benchmark for studying questioning strategies in CIR. The proposed benchmark isolates the Questioner module by keeping the retriever, answerer, dialogue budget, decoding setting, and evaluation pipeline fixed across all methods. Under this unified protocol, we compare representative strategies including Blind, Text-based, Text-based Recon, Top-K Guided, and Hybrid. Experimental analysis reveals a clear stage-dependent trade-off between efficiency and discriminative capability. Text-based strategies remain computationally lightweight but become increasingly vulnerable to contextual drift over extended interaction, whereas Top-K Guided improves retrieval refinement at the cost of higher inference latency. Overall, the proposed benchmark provides a reproducible framework for analyzing conversational questioning behavior in CIR systems.
Bui Tay, Son Nguyen Thanh, Phuc Nguyen Vu et al.· International Conference on...· 0 citations
Traditional conversational recommenders entangle retrieval and response generation within a single text interface, so exact entity cues fade as the dialogue’s intent evolves, which compromises explanation credibility. We address this within the ACM RecSys Challenge 2026, which mandates both top-20 ranking and evidence-grounded response generation. This paper presents the third-place solution by team “swyoo” for the Blind-B industry track. We decouple retrieval and response into separate pipelines connected strictly via ranked tracks and metadata. Retrieval combines a hybrid lexical-dense pool for exact matching with a task-adapted pool driven by fine-tuned Qwen 8B adapters. Candidates are calibrated via LightGBM, then routed to an evidence-grounded propose-assign-select (PAS) framework to structure responses. This system also ranked second on the explanation-quality leaderboard in the final blind evaluation. Our findings demonstrate that: (i) isolating retrieval and response preserves both catalog cues and fluid intent; (ii) structuring generation via explicit evidence assignment supports this near-best-in-class explanation quality under the official LLM judge.
Sungwook Yoo, Se-Wook Yoo· Proceedings of the Workshop...· 1 citation
Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence. However, standard retrievers and rerankers optimize for semantic similarity rather than answer utility, creating a preference gap: documents that appear relevant may not help the generator produce a correct answer. Motivated by this, we propose a two-stage generator-in-the-loop alignment framework that closes this gap without human document-level relevance annotations. Our framework consists of two stages: in Stage 1, a VLM generates a hypothetical text passage from the image-query pair, which is used as the retrieval query for dense text search, bridging the image-to-text modality gap. In Stage 2, a cross-encoder reranker adapted with low-rank adaptation (LoRA) is fine-tuned using answer-supervised preference pairs mined from the frozen VLM: given the dataset answer label, a candidate document is labeled positive if the VLM produces the correct answer when given that document as context, and negative otherwise. This generator-guided signal is compatible with multiple alignment loss functions, including contrastive (triplet) loss, pairwise direct preference optimization (DPO), and supervised fine-tuning (SFT), and supports periodic re-mining to refresh preference pairs as the reranker improves. Experiments on VQA-X and A-OKVQA with Qwen3.5-2B and Qwen3-VL-4B-Instruct show that our proposed framework consistently outperforms rank-order, random, and REPLUG-style likelihood baselines under various alignment losses and pool size settings, suggesting that answer-level generator feedback is an effective supervision signal for preference alignment.
Zhang-Yu Chang, Dong-Jun Han, Seyyedali Hosseinalipour et al.· 0 citations
Experiments on the LongLaMP dataset show that PrefReward outperforms non-personalized and retrieval-based baselines in both generation quality and personalization interpretability.
Yue Wu, Chengbing Wang, Yimeng Bai et al.· arXiv.org· 0 citations
Real-world image search queries are multimodal and compositional: ``find this shirt in pink''specifies an entity to retain, an attribute to modify, and context to ignore. Yet existing re-rankers either compress such multifaceted relevance into an opaque embedding or rely on free-form chain-of-thought that easily omits or hallucinates fine-grained constraints. Drawing on rubric- and checklist-based evaluation from NLP, we recast multimodal image re-ranking as a semantic constraint satisfaction problem and propose EviRank, which parses any query - text-only, image-only, or composed - into a unified evidence package: typed criteria across six semantic slots (e.g., entities, attributes, relations), each labelled required, forbidden, or ignorable. Re-ranking then reduces to evidence-conditioned verification, combining deterministic rubric scoring and evidence-grounded listwise comparison in a single training-free procedure. The explicit evidence can further serve as structured supervision for optionally distilling a lightweight student. Across five benchmarks spanning text-to-image, image-to-image, and composed image retrieval, EviRank achieves state-of-the-art performance, and the distilled student preserves over 90% of the teacher's capability at substantially lower cost.
Enjun Du, Siyi Liu, Zi-Rong Chen et al.· 1 citation
Composed video retrieval (CoVR) searches a gallery for the target video that realizes a natural-language modification of a source clip. However, at gallery scale, this creates a fundamental tension: compact embeddings enable efficient, reusable search but can miss the transient actions, state changes, and subtle constraints that demand fine-grained video reasoning, whereas applying large multimodal models uniformly sacrifices scalability. To address these limitations, we propose that frozen foundation models should instead occupy complementary roles, with inference depth adapted to query difficulty. Based on this premise, we introduce \methodname{}, a framework for training-free \methodexpansion{}. Specifically, a composed-query embedding first searches reusable video-only gallery representations; uncertain queries undergo bounded reranking and candidate expansion; ambiguous edits trigger target-description generation; and only close leading candidates reach multimodal verification. To support these roles, frame selection, spatial resolution, and time cues are adapted to each stage. Across complete target-gallery evaluations, our method reaches state-of-the-art performance among training-free approaches, with 89.55 and 93.43 R@1 on Dense-WebVid-CoVR and CoVR-R, respectively (with more than +35\% and +25\% absolute margins to the closest counterpart). These results show that adaptively orchestrating foundation-model capabilities can combine scalable retrieval with fine-grained reasoning without task-specific training. The source code and all relevant guidelines are available on https://github.com/demidovd98/CoVRAGE.
Dmitry Demidov, Muhammad Zaigham Zaheer, Omkar Thawakar et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
