The Social Lure of Generative AI

Abstract

The central claim of this paper is that current generative AI systems are not social agents despite their increasingly human-like language abilities where “social” is intended to refer not merely to agents being able to produce socially appropriate behaviors, but to possessing the affective, motivational, and interpersonal mechanisms that underlie human social interaction and cognition. Yet, humans inevitably respond to conversational AI as though it were a social partner despite the absence of the cognitive and affective mechanisms that make genuine social interaction possible. The problem therefore is not that AI is or is not social, but that humans cannot help responding socially. Recognizing this distinction is essential for both AI research and AI governance. Rather than striving to make conversational systems appear ever more human, future work should seek to align their outward behavior with their actual capabilities while developing appropriate safeguards for applications in which emotional or social interactions are unavoidable.