Beyond the Prompt: Why Physical Embodiment Still Matters for LLM-Powered Conversational Agents

Abstract

Large Language Models (LLMs) have transformed conversational AI, but an unresolved human-robot interaction question remains: “With conversational agents, does physical embodiment matter?”. This paper compares an embodied Pepper robot with a non-embodied laptop interface driven by the same LLM - Llama 3.2 7B parameter model in a within-subjects, counterbalanced study $(\mathrm{N}=32)$. Participants engaged in open-ended dialogue with both agents and completed post-interaction questionnaires assessing interaction quality, social response, comfort, satisfaction, and overall preference. The embodied condition was rated significantly higher on naturalness, flow, relevance, understood responses, engagement, perceived social entity, connection, and comfort, with the largest paired effect on engagement $\left(d \_z=1.12\right)$. Self-rated tech-savviness did not significantly correlate with the core outcomes reported for either condition. We argue that the contribution of the study lies in demonstrating that embodiment remains consequential even when linguistic capability is held constant at an LLM level.