Jul 2026· Annual International ACM SIGIR Conference on Research and Development in Information Retrieval· pp. 5382-5385· 0 citations· 8 references
Computer Science
TL;DR
This workshop aims to provide a forum to analyze implications of AI-generated content implications with an emphasis on reflection, evaluation, and human-centered system design, and to foster community-driven discussion that may inform future evaluation efforts, including potential shared tasks or tracks in venues such as TREC, CLEF, FIRE, or NTCIR.
Abstract
The proliferation of AI-generated content is fundamentally altering the information ecosystems in which retrieval systems operate. Search engines, recommender systems, and retrieval-augmented generation pipelines increasingly function in mixed information environments where synthetic and human-authored content are tightly interwoven, raising system-level challenges for information retrieval. Key issues include limitations in evaluation validity, as traditional metrics designed for human-authored corpora fail to capture the distinctive properties of AI-generated content; shifts in retrieval behavior and ranking dynamics, as systems may inadvertently favor procedurally generated but weakly grounded information; and challenges to user trust, as assumptions about the provenance and reliability of retrieved human content become more difficult to distinguish from generated content. Rather than focusing only on model-centric performance comparisons, this workshop aims to provide a forum to analyze these implications with an emphasis on reflection, evaluation, and human-centered system design, and to foster community-driven discussion that may inform future evaluation efforts, including potential shared tasks or tracks in venues such as TREC, CLEF, FIRE, or NTCIR.
For decades, search and recommendation systems have been optimized as distinct components within large-scale discovery platforms. The rise of generative AI is beginning to blur this boundary. At Spotify, we are exploring how large language models can evolve from tools that retrieve content into systems that reason over users, catalogs, and intent, while remaining steerable through natural language and user interaction. This talk presents lessons from deploying and studying generative retrieval and recommendation systems across Spotify's content ecosystem. I will describe how semantic identifiers enable language models to operate directly over large, heterogeneous catalogs, allowing search, recommendation, retrieval, explanation, and user understanding to be expressed within a common generative framework. I will discuss recent work on production-scale podcast discovery, language-steerable recommendation, and the NEO framework for unifying search, recommendation, and reasoning across multiple content types. These systems demonstrate how grounding language models in catalog entities and user behavior can improve discovery while preserving the flexibility of natural-language interaction. More broadly, they suggest a path toward discovery systems in which retrieval, recommendation, and reasoning are no longer separate stages, but capabilities of a shared generative model. Beyond model frameworks, I will discuss the emerging challenges of alignment and evaluation in discovery systems. Unlike traditional retrieval problems, generative recommendation often has many valid answers. I will present approaches for learning from large-scale behavioral signals, preference-aware optimization, and profile-aware LLM-as-a-judge evaluation, along with lessons from online experimentation at Spotify. These experiences suggest that future discovery systems will require new forms of personalization, controllability, and evaluation that extend beyond conventional ranking metrics. I will conclude with a research agenda for generative discovery systems, including language-steerable interfaces, unified retrieval-and-reasoning models, preference-aligned generation, and evaluation frameworks designed to measure user-specific relevance at scale. As search, recommendation, and conversational AI continue to converge, these directions point toward a new generation of discovery systems that can understand intent, reason over large catalogs, and help users navigate increasingly complex information spaces.
Paul N. Bennett· Proceedings of the 32nd ACM...· 0 citations
This article presents an ongoing experiment on the use of Retrieval-Augmented Generation (RAG) architectures in library contexts, considering them not as straightforward extensions of information retrieval techniques but as document mediation devices that require explicit methodological reflection. The contribution focuses in particular on the design and analysis of an integrated evaluation framework, conceived as a structural component of system development rather than as an ex post performance check. Using the personal archive of Emanuele Artom as a case study, characterized by bibliographic and archival heterogeneity, the article examines how the quality of generated responses emerges from the interaction between knowledge base modeling, retrieval strategies and evaluation criteria. The proposed framework combines automatic metrics, LLM-as-a-judge approaches, and human-in-the-loop processes, treating divergences between automated evaluation and expert judgment as diagnostic tools for analyzing system behaviour. Preliminary results indicate that core categories of document mediation, such as relevance, completeness, citability and transparency, cannot be fully reduced to computational parameters but, instead, require continuous negotiation between automated models and disciplinary expertise. From this perspective, RAG is framed as an epistemically unstable research object, whose reliability and governability depend on the robustness and reflexivity of the evaluation processes embedded in its development.
Angelo La Gorga, Lorenzo Verna· JLIS.it· 0 citations
This work introduces a trust based adaptive reranking model- ATM (Adaptive Trust Model) that allocates computational resources according to file level uncertainty, instead of assigning a fixed number of reranker calls per query, which focuses computation only where ranking confidence is low.
Jenny Kalaiarasi.S· Journal of Intelligent Decis...· 0 citations
In modern enterprise applications, Retrieval-Augmented Generation (RAG) has emerged as a powerful mechanism for knowledge delivery. However, conventional RAG frameworks operate in a behavioral vacuum, relying solely on text-based queries while ignoring the user's operational context, navigation history, and implicit intent. This lack of behavioral awareness frequently results in fragmented user journeys, information overload, and suboptimal decision support. To address these limitations, this study introduces a novel Behavior-Aware Retrieval-Augmented Generation (BA-RAG) framework specifically engineered for enterprise ecosystems. The proposed architecture captures real-time user telemetry—including navigation pathways, query velocity, and interaction history—and transforms these signals into behavioral context vectors. During the retrieval phase, these context vectors are mathematically fused with standard semantic search embeddings to rerank knowledge inputs prior to Large Language Model (LLM) generation. The framework was evaluated via a controlled user study (N=45) within a simulated enterprise Resource Planning (ERP) environment. The experimental results demonstrate that the behavior-aware framework yielded a 25% reduction in user navigation steps, a drop in average task completion time from 8.4 to 6.3 minutes, and enhanced retrieval precision, raising the Normalized Discounted Cumulative Gain (NDCG@5) score from 0.71 to 0.88. These findings demonstrate that incorporating real-time user behavioral context is critical for transforming generative AI utilities into proactive workflow accelerators within complex, data-dense corporate environments.
Sri Charan Chowdary Konidina· International Journal of Adv...· 0 citations
Recommender systems are essential to real-world online services due to their substantial business value. Increasingly, large language models (LLMs) have been adopted for recommendation tasks, owing to their strong reasoning capabilities and their effectiveness in handling cold-start items. A large body of work has sought to adapt retrieval-augmented generation (RAG) for LLM-based recommender systems by retrieving information from external databases and integrating it into the LLM's input to improve recommendation quality. Despite the rapid growth of the field, existing tutorials lack a clear taxonomy and a dedicated review of RAG methods for LLM-based recommender systems. To fill this gap, we present the first tutorial on this topic. First, we discuss the types of information used in LLM-based recommender systems, including target-user information and external databases for retrieval. Second, we present the design space of RAG for LLM-based recommender systems, outlining the key design choices at each stage of the pipeline: (1) retrieval, (2) augmentation, and (3) generation. Lastly, we introduce open questions for future research on this topic. Tutorial materials, including slides, are available at https://recsys-rag-tutorial.github.io/.
Sunwoo Kim, Geon Lee, Kyungho Kim et al.· Proceedings of the 32nd ACM...· 0 citations
Recommender systems are among the most impactful applications of artificial intelligence, serving as critical infrastructure connecting users, merchants, and platforms. However, most industrial systems remain heavily reliant on historical co-occurrence patterns and log-fitting objectives, i.e., optimizing for past user interactions without explicitly modeling user intent. This log-fitting approach often leads to overfitting to narrow historical preferences, failing to capture users’ potential interests. As a result, it reinforces filter bubbles and long-tail phenomena, ultimately harming user experience and threatening the sustainability of the recommendation ecosystem. To address these challenges, we rethink the overall design paradigm of recommender systems and propose RecGPT, a fully integrated, production-ready framework that places user intent at the center of the recommendation pipeline. By integrating large language models (LLMs) into key stages of user interest mining, item retrieval, and explanation generation, RecGPT transforms log-fitting recommendation into an intent-centric process. To effectively align general-purpose LLMs to the above domain-specific recommendation tasks at scale, RecGPT incorporates a multi-stage training paradigm, which integrates reasoning-enhanced pre-alignment and self-training evolution, guided by a Human-LLM cooperative judge system. Currently, RecGPT has been fully deployed on the Taobao App. Online experiments demonstrate that RecGPT achieves consistent performance gains across stakeholders: users benefit from increased content diversity and satisfaction (e.g., CICD +5.53%, DT +4.35%), merchants and the platform gain greater exposure and conversions (e.g., CTR +5.65%, IPV +7.55%, DCAU +2.46%), and sustained long-term retention improvements (LT-30 +1.63%). These comprehensive improvement results across all stakeholders validate that our intent-centric design can foster a more sustainable and mutually beneficial ecosystem.
Jiakai Tang, Wen Chen, Dian Chen et al.· ACM Transactions on Informat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
