A controllable AI framework for certification technical review via knowledge graph and rule engine

Abstract

Technical review is a critical control stage in management system certification because it verifies whether audit conclusions are appropriate, complete, and compliant with accreditation requirements. In current practice, this work is still largely manual and therefore suffers from low efficiency, reviewer inconsistency, weak evidence traceability, and dossier accumulation during peak periods. Although large language models (LLMs) have shown strong capabilities in semantic understanding and long-document analysis, certification technical review is a high-constraint task in which clause legality, evidence chain integrity, and accountability are more important than fluent text generation. Pure LLM-based solutions therefore face practical risks such as hallucinated clause references, unstable reasoning, opaque decision grounds, and limited cross-dossier consistency. To address these limitations, this paper proposes a controllability-oriented AI framework for certification technical review by integrating LLMs with retrieval-augmented generation (RAG), a certification knowledge graph, and a rule engine. The framework decomposes review into dossier structuring, clause matching, logical consistency verification, risk identification, and review recommendation generation. The knowledge graph provides structured relationships among clauses, audit activities, processes, risk points, and nonconformity patterns, while the rule engine enforces deterministic constraints on clause legality, audit duration, qualification matching, and risk-level triggers. A human-in-the-loop review workstation is further introduced to ensure editable outputs, visible evidence chains, and full revision traceability. We define controllability in certification review through five dimensions: legality, consistency, evidence completeness, explainability, and human acceptability. Based on these dimensions, a comparative experimental protocol is designed on real-world desensitized audit dossiers. The results indicate that the proposed hybrid framework substantially improves clause matching accuracy, consistency defect detection, risk coverage, and reviewer acceptance compared with LLM-only and LLM+RAG baselines. The study offers an interpretable and deployable pathway for AI-assisted technical review in certification and other high responsibility document review scenarios.