Towards Fully Automated COBOL-to-Java Translation

Abstract

COBOL remains one of the most widely deployed programming languages in mission-critical business systems, yet the pool of developers who can maintain it is shrinking. Automated translation to modern languages like Java is an increasingly attractive modernisation strategy, but systematic evaluation of available approaches remains limited. Here, we evaluate state-of-the-art open source COBOL-to-Java translation approaches, across a benchmark of 326 small open source programs, and investigate LLM-based strategies for repair and maintainability improvement of resultant translations. The evaluated tools present a clear correctness-maintainability trade-off: COBOL4J, a rule-based COBOL-to-Java translator, achieves the highest functional correctness, based on test-pass rates, at 86.4%, but generates verbose, unmaintainable output. XMAiNframe-7B, a COBOL fine-tuned LLM, strikes the best balance across both dimensions with 55.2% functional accuracy; and Qwen2.5-Coder-7B, a general-purpose model, performs weakest on functional correctness at 22.4%. We find that the post-hoc repair pipeline, with the best-performing iterative prompting strategies, eliminates most compilation errors across all three tools on the tested dataset. Post-hoc refactoring reduces COBOL4J’s verbose output from an average of 15 functions per file, to only 1.2, however, half the time at the cost of correctness. Our findings show that production-grade COBOL modernisation is best approached as a hybrid pipeline: using rule-based translation to ensure baseline functional correctness, followed by domain-specialised LLM refinement to improve structure and maintainability. The derived bug taxonomy analysis can be used to guide targeted prompt design and improve future COBOL-to-JAVA translator development.