Annotation Light Requirements-to-Test Traceability with Hybrid Retrieval and LLM Validation

Abstract

Software projects often maintain large requirement and test repositories, yet explicit trace links between them are incomplete or inconsistently maintained. This paper presents a requirements-to-test traceability pipeline that ranks candidate links for natural-language requirements using hybrid retrieval and optional LLM-based validation. Requirements and test cases are chunked, scored with normalized dense semantic and BM25 lexical similarity, blended, and aggregated into test level rankings. Because industrial repositories rarely provide curated relevance judgments, we evaluate with naturally occurring weak labels: deterministic REQ-keyword links used as high precision, pipeline independent anchors. On an industrial dataset with 331 requirements, 567 test cases, and 470 keyword derived links, the base hybrid scorer achieves 85.4% Hit@10 and 0.645 MRR. Tuning the dense–sparse blend and using asymmetric query/document embeddings improves performance to 89.3% Hit@10 and 0.695 MRR without adding a reranking stage. In an exploratory single run setting, LLM validation over the candidate pool increases measured keyword-gold F1 from 0.429 to 0.491 when prompted to accept tests exercising any aspect of a requirement. The same configuration transfers to EasyClinic, a public manually curated benchmark, achieving 0.929 MRR and improving top-10 candidate-pool F1 from 0.264 to 0.686 at constant recall. These results show that weak traceability anchors can support practical, annotation-light evaluation of requirements-to-test traceability systems.