Understanding Human-AI Interaction Through Chat Logs: Linguistic Patterns, Breakdowns, and User Expectations in a RAG-Based Sales Assistant

Abstract

Retrieval-augmented generation (RAG) is increasingly used across knowledge-intensive enterprise contexts, yet little is known about real-world interaction with such systems. We present a qualitative field study of a RAG-based assistant in B2B sales, analyzing 190 chat sessions and a complementary user survey. Drawing on conversational and HCI perspectives, we examine interaction patterns, breakdowns, and repair practices in authentic user-AI dialogues. Results show that interactions are short, directive, and shaped by search-like mental models. Breakdowns frequently occur due to underspecified input and incomplete system responses, with repair largely initiated by users. Survey findings indicate generally positive perceptions of usefulness and trust, though expectations of complete answers contrast with limited clarification behavior. We contribute an analysis of real-world RAG interaction and derive implications for proactive clarification and hybrid interaction.