A Design-Space Approach to Robust Agentic Conversational Recommendation

Abstract

Agentic Conversational Recommender Systems (ACRSs), in which a large language model orchestrates dialogue, item retrieval, and tool use (such as querying item attributes or ranking candidate items), are advancing quickly, yet two problems prevent us from fully understanding their potential and limitations. First, progress is reported at the level of whole systems, so gains cannot be traced to specific design choices. Second, evaluation relies on simulated users who arrive with clear, well-defined needs, leaving untested the users a conversational system should help most—those who cannot yet put their preferences into words. The goal of my PhD project is to address these problems. I take a design-space approach, in three stages: (1) a characterization of the design space of an ACRS: the individual design choices, and a controlled method that studies each in isolation; (2) an analysis of which of these design choices most determine how well an ACRS elicits and satisfies a user’s preferences, and of which effects are invariant across architecture, domain, and model family versus contingent on the conditions of deployment; and (3) an evaluation that spans simulated users across the full range of user behavior and tests the resulting conclusions against real users, mapping when simulation-based evaluation can be trusted. The first stage is complete; the other two are under way and form the focus of the rest of my PhD. Ultimately, I aim to build conversational recommenders that help users move from an uncertain, ill-formed need to a satisfying choice—adapting across the full spectrum of preference-awareness and initiative, robust to that diversity, and grounded in design principles whose conditions of validity are known.