Integrating large language models and knowledge graphs for adaptive design review

Abstract

Automated Compliance Checking (ACC) systems are fundamentally static, unable to easily adapt to new regulations, project constraints, organizational, or practitioner-defined rules. This paper presents a framework integrating Knowledge Graphs (KGs) and Large Language Models (LLMs) to support a more extensible design review environment. In this framework, the KG acts as a structured repository for rules and executable logic, while the LLM serves as an intelligent interface. The central innovation is the human-in-the-loop feedback mechanism, where new logic generated by the LLM is validated, executed, and permanently stored in the KG, transforming it into an active, evolving validation engine. Following a Design Science Research (DSR) methodology, we implement and evaluate the framework as a prototype embedded as an Autodesk Revit add-in, demonstrating its ability to retrieve and execute existing rules from the KG, capture new requests during design, and maintain a verifiable, adaptive compliance checking system. Across a two-experiment evaluation, the system achieved 100% mapping accuracy for six existing rules, while generating new executable rules from natural language succeeded in 70% of 20 trials. Performance was strong on parameter-based checks (100%) but dropped on rules involving spatial reasoning (20–60%), where the LLM still struggles to produce reliable logic.