Are You Sure This is What You Really Want? Conversational Recommender Systems for Enhancing Human Autonomy

Abstract

Recommender systems are widely used to help users navigate information overload in digital environments. While they are often portrayed as tools that enhance autonomy by optimizing choice and personalizing content, this article argues that many current recommender systems designs in fact undermine user autonomy. Drawing on a conception of autonomy in which the formation and configuration of preferences, particularly higher-level desires in the Frankfurtian sense, play a prominent role, we examine how dominant recommendation techniques shape users’ desires and, to a greater extent, their identities. We first analyze what we call conjectural recommenders, systems that infer preferences from behavioral data such as clicks or viewing histories. These systems conflate observed choices with genuine preferences, reinforcing first-order desires, narrowing the diversity of recommendations, and trapping users in homogeneous digital environments that impede identity development and degrade deliberative capacities. We then assess interrogative recommenders, which incorporate explicit user feedback as a form of positive friction in the user interface. Although these recommenders prompt users to articulate evaluations, we argue that they remain insufficient for capturing second-order desires and raise new design challenges concerning paternalism and usability. In response, we propose the Socratic conversational recommender system, a model that combines conversational recommendation with a second layer of Socratic questioning aimed at eliciting users’ metapreferences. Rather than prescribing substantive values, the system guides reflection through formal criteria such as coherence, empirical awareness, and cognitive pluralism. The objective is not for the system to discover users’ true preferences but to foster their reflective engagement with what they want to want. We conclude that embedding deliberative dialogue within recommender design offers a promising pathway for aligning algorithmic recommendation with the preservation and development of human autonomy.