Debugging Requirements Interview Scripts: A Framework for Quality Assessment

Abstract

Requirements elicitation interviews are a fundamental technique for identifying stakeholders' needs in software projects. Interview scripts, which are lists of questions prepared by the analyst to guide discussions with stakeholders, often exhibit issues or “smells”, such as unclear phrasing, use of technical jargon, omission of quality requirements, or failure to address prioritization. Providing structured, high-quality feedback on such scripts remains costly and difficult to scale, particularly when expert review is required. This paper introduces a framework for systematically assessing and “debugging” requirements interview scripts, based on a coding scheme capturing interview smells and missing topics. We further investigate the extent to which such assessment can be supported by automated feedback generated by Large Language Models (LLMs). We compare automatic feedback with that of human reviewers using a ground truth defined by requirements engineering experts. Our results reveal a clear trade-off: LLMs provide broad coverage when identifying potential interview issues but tend to generate overly inclusive feedback, while human reviewers produce more precise, context-sensitive assessments. These findings show that interviewscript quality can be systematically operationalized and partially automated. LLMs enable scalable first-pass screening, while human reviewers provide validation and contextual judgment, together supporting hybrid human-AI review workflows for requirements elicitation in both professional and training contexts.