Integrating large language models and knowledge graphs for adaptive design review
TL;DR
This paper presents a framework integrating Knowledge Graphs and Large Language Models to support a more extensible design review environment, and demonstrates its ability to retrieve and execute existing rules from the KG, capture new requests during design, and maintain a verifiable, adaptive compliance checking system.
Abstract
Automated Compliance Checking (ACC) systems are fundamentally static, unable to easily adapt to new regulations, project constraints, organizational, or practitioner-defined rules. This paper presents a framework integrating Knowledge Graphs (KGs) and Large Language Models (LLMs) to support a more extensible design review environment. In this framework, the KG acts as a structured repository for rules and executable logic, while the LLM serves as an intelligent interface. The central innovation is the human-in-the-loop feedback mechanism, where new logic generated by the LLM is validated, executed, and permanently stored in the KG, transforming it into an active, evolving validation engine. Following a Design Science Research (DSR) methodology, we implement and evaluate the framework as a prototype embedded as an Autodesk Revit add-in, demonstrating its ability to retrieve and execute existing rules from the KG, capture new requests during design, and maintain a verifiable, adaptive compliance checking system. Across a two-experiment evaluation, the system achieved 100% mapping accuracy for six existing rules, while generating new executable rules from natural language succeeded in 70% of 20 trials. Performance was strong on parameter-based checks (100%) but dropped on rules involving spatial reasoning (20–60%), where the LLM still struggles to produce reliable logic.