Large-Language-Model-Agent-Enabled Analysis of Semi-Structured Interviews: Comparing Driver Experience Across Partial Automation Systems
TL;DR
It is suggested that driver satisfaction with partial automation can coexist with uncertainty about system limits and automation boundaries, and that driver satisfaction with partial automation can coexist with uncertainty about system limits and automation boundaries.
Abstract
SAE Level 2 (L2) partial automation changes the driving task, shifting the driver’s role from active operator to supervisor, raising questions about trust, satisfaction, perceived limitations, and system understanding. This study presents a locally hosted, auditable large language model (LLM) agent pipeline for analyzing 161 semi-structured post-study interviews from a 30-day naturalistic driving study of Tesla Autopilot, Cadillac Super Cruise, and Volvo Pilot Assist. The pipeline segmented transcripts into semantic units and coded each unit by question theme, answer theme, subtheme, and sentiment, enabling structured comparison across systems while preserving traceability to source evidence. Results showed significant vehicle-level differences in sentiment, with satisfaction-related responses strongly positive but performance-limitation narratives strongly negative. Validation against manually coded interview data showed high accuracy for answer subthemes and sentiment. Findings suggest that driver satisfaction with partial automation can coexist with uncertainty about system limits and automation boundaries.