Traceable RAG for Legacy COBOL Comprehension: Integrating Architectural and Semantic Analysis

Abstract

Understanding undocumented COBOL applications is a major challenge in legacy system modernization. We present a locally deployed open-source RAG pipeline that combines architectural and semantic program analysis to support traceable LLM-based question answering about the full legacy codebase. The extracted information is organized into structured artifacts covering program metadata, source code, business rules, and dataflow, while preserving links to the underlying analysis results. Relevant evidence is retrieved through a hybrid strategy and provided to a local LLM, which generates answers accompanied by inspectable supporting artifacts. A preliminary expert evaluation classified 31 of 42 question formulations as satisfactory. At the information need level, the system obtained satisfactory results for 11 of 14 needs, while eight were answered satisfactorily across all three paraphrased formulations.