UML Class Diagram Evaluation and Repair Strategies based on LLMs

Abstract

UML class diagrams are a crucial tool for defining the structure of software systems, but designing accurate and comprehensive class diagrams is a challenging task. Traditionally, creating UML models relies on the expertise and experience of professionals. However, with the development of AI technologies, particularly LLMs, new opportunities for software modeling have emerged. Despite this, there has been limited research on the application of LLMs in software modeling, especially in UML class diagram modeling. This study conducts experiments on several typical software system cases. Combining SDMetrics with expert manual review, this paper comprehensively evaluates the practical performance of mainstream LLMs in UML class diagram modeling from multiple dimensions, including size and completeness, relationship correctness, inheritance hierarchy, and design rule compliance. Focusing on typical defects in LLM-generated UML class diagrams, this study reveals that LLMs exhibit uncertainties analogous to human memory. Accordingly, three targeted repair strategies are proposed, including memory reinforcement, external knowledge injection, and detection-guided automated targeted repair. Experimental results obtained from the case studies indicate that (1) compared to expert-crafted class diagrams, LLM-generated UML class diagrams exhibit several issues, such as incomplete identification of key classes, confusion or omissions in relationships, insufficient or absent inheritance relationships, unused classes, and circular dependencies, and (2) after applying the repair methods, all the LLMs show varying degrees of improvement in addressing these issues. The average repair rate for key class identification reaches 85%, the coupling relationship repair rate is 46%, the inheritance relationship repair rate is 69%, while repair rates for unused classes and circular dependencies both reach 100%.