Jul 2026· Journal of Information Technology and Digital World· Vol 8, pp. 195-209· 0 citations· 11 references
TL;DR
CompVault, an Enhanced Retrieval-Augmented Generation (ERAG)-based Artificial Intelligence Compliance Monitoring and Report Generation System for intelligent regulatory compliance assessment, and results indicate that the ERAG-based framework can be used as an efficient, scalable, and explainable solution for regulatory compliance monitoring and automated report generation.
Abstract
Compliance with regulatory rules becomes an increasingly complex task because of the constantly changing legal regulations and policy regulations. Currently, the existing approach to compliance verification is mostly based on the manual analysis of documents and is characterized by high time consumption, inconsistency, and high levels of operational risks. To solve the problem, this paper introduces CompVault, an Enhanced Retrieval-Augmented Generation (ERAG)-based Artificial Intelligence Compliance Monitoring and Report Generation System for intelligent regulatory compliance assessment. The introduced approach combines the semantic document retrieval, vector-based knowledge representation with ChromaDB, and contextual reasoning by means of the large language model to check compliance of organizational policies with regulatory requirements. It also determines the compliance gaps and compliance risk level and generates structured reports with recommendations to decision-makers. Web-based implementation was implemented using FastAPI, all-MiniLM-L6-v2 embeddings, and LexGLUE legal benchmark as a knowledge source about regulations. As a result, the following metrics were achieved: 97.42% accuracy, 96.88% precision, 97.15% recall, 97.01% F1-score, and 98.80% AUC-ROC. Therefore, the achieved results indicate that the ERAG-based framework can be used as an efficient, scalable, and explainable solution for regulatory compliance monitoring and automated report generation.
Large-scale engineering projects generate continuous streams of compliance-critical documents, including material submissions, method statements, inspection and test plans, safety data sheets, and contractor certificates. Each must be verified against project specifications, regulatory codes, and contractual requirements. Current manual expert review is slow, inconsistent, and provides limited audit depth. Existing AI approaches typically use Retrieval-Augmented Generation to retrieve relevant clauses but do not conduct structured, rule-bound verification. This paper presents an agentic document verification framework that moves beyond passive retrieval to active, rule-aware compliance checking. The system uses a semantically indexed knowledge base built from project specifications, regulatory standards, and historical approval records. A dedicated Verification Agent decomposes documents into structured claim units, including numerical parameters, referenced standards, tabular test results, and graphical certificates. These are evaluated against dynamically constructed project rule sets using a Chain-of-Thought inference pattern. The framework generates compliance reports with pass, fail, or query verdicts, confidence scores, and traceable evidence bindings for each decision point. Material submissions are the primary validation domain because their dense technical content, cross-referenced tables, graphs, and third-party certificates rigorously test multi-modal parsing and verification capabilities. Validation used an industrial pilot across 10 live projects in the Electrical and Mechanical engineering sector. Results from 63 processed submissions show a 70.9% reduction in average review time, from 52.5 minutes under the existing digital workflow to 15.3 minutes with the AI-assisted system. Results also show an 88% system agreement rate, with human overwrites required in only 12% of verdicts. By incorporating a Propose-Decide-Evidence governance model, the system retains the human engineer as final decision-maker while establishing an efficient, auditable, continuously improving compliance workflow.
Ka Tai Lau, Man Chit, Jovian Cheung et al.· AHFE International· 0 citations
Trust is fundamental in modern regulatory ecosystems, and compliance checking plays a critical role in fostering that trust. Regulatory compliance verification is essential for businesses operating in highly controlled environments, as it ensures alignment with sector-specific guidelines across domains such as financial reporting, data privacy, and cybersecurity. Manual compliance testing, however, is often time-intensive and prone to inconsistencies, particularly when compliance depends indirectly on third-party services such as cloud providers, where vendors rely on external providers to meet regulatory standards. In this paper, we present CTRAG, a novel Retrieval-Augmented Generation (RAG) pipeline designed for automated compliance checking. CTRAG employs advanced strategies, including adaptive chunking, dynamic retrieval configurations, and in-context learning, to improve the precision and relevance of compliance assessments. By extracting control questions from regulatory texts and cross-referencing them with unstructured company documentation, CTRAG achieves highly accurate, document-informed compliance verification, even in cases of indirect compliance through third-party services. Empirical evaluations demonstrate significant improvements, with CTRAG achieving an F1-score of 78% and a recall of 85% in the final deployed configuration, ensuring minimal missed non-compliance cases while reducing manual reviewer effort in a real-world deployment. To validate CTRAG value, we developed and deployed a POC within a Big Four professional services firm, applying it to real-world cases and cross-checking results against manual compliance reports. These findings highlight CTRAG potential to streamline compliance workflows, mitigate risks, and enhance regulatory trust in complex, high-stakes environments.
Muhammad Roman, Karen Rafferty, Barry Devereux· 0 citations
Code reviews are a central component of collaborative software development, yet they often require considerable manual effort in practice. Ensuring consistent compliance with project-specific standards can be challenging, particularly when these standards require contextual interpretation of code, naming conventions, or documentation.
This paper presents an LLM-based prototype designed to support the application of a documented rule catalog within GitLab merge requests. The agent is integrated into the existing workflow and is triggered automatically by merge request events. It analyzes the source code contained in a merge request and evaluates it against explicitly defined project rules.
To ensure structured and consistent output, a predefined JSON schema guides the model’s response and enables validation before publication. Invalid or non-parseable outputs are detected and not posted to the merge request.
The approach is not intended to replace human reviewers, but to support them in the systematic application of documented project standards. The implementation demonstrates the technical feasibility of integrating rule-based governance with LLM-supported contextual interpretation in a practical GitLab workflow.
Sehyang Na, D. Kostunin· TH Wildau Engineering and Na...· 0 citations
The operations of life insurance are among the most document-heavy and heavily regulated financial services. Such conditions have motivated growing interest in generative AI for document automation. The probabilistic nature of large language models brings unacceptable risks into regulated workflows. Hallucinations, omitted mandatory disclosures, and non-compliant phrasing could expose insurers to regulatory scrutiny, legal liability, and consumer harm. The paper presents a compliance-aware reference architecture that reconfigures the design problem by decoupling probabilistic generation and deterministic compliance enforcement. The architecture employs retrieval-augmented grounding over a continuously versioned corpus of authoritative sources, along with automated validators that check for the completeness of disclosures, traceability of evidence, detection of prohibited phrases and a human-in-the-loop review gate which captures audit trails for regulatory scrutiny. Our architecture is evaluated against four widely used LLMs: Claude 3.5 Sonnet, Llama 3.170 Billion, Amazon Nova Pro, and Pixtral Large. For this, we create 50 test cases, which span five document types, yielding 200 baseline and 200 compliance-aware drafts. After excluding 3 failed generations from Pixtral Large, a total of 197 baseline-compliance matched comparisons were used in the final analysis. The compliance-aware pipeline reduces unsupported claims by $\mathbf{8 0} \boldsymbol{\%}$, from $\mathbf{2 6. 5 \%}$ in prompt-only outputs to 5.3%. After human review, all models achieve 100% final approval, with pre-review ready-to-send rates ranging from 78% to 90.7% across models. Results provide statistically significant evidence that the model-agnostic architecture substantially reduces unsupported claims while remaining operationally feasible, providing insurers with a promising proof-of-concept architecture for deploying generative AI in regulated document workflows, motivating further validation at production scale.
Ensuring software compliance with regulations such as the General Data Protection Regulation (GDPR) and the Artificial Intelligence Act (EU AI Act) poses a significant challenge, as requirements engineers must translate complex legal text into actionable software requirements - a process that remains largely manual and error-prone in practice. We present an automated regulation-to-requirements pipeline that identifies requirement-bearing clauses in regulatory documents and derives system-agnostic software requirements, accompanied by plain-language explanations, traceable to their legal sources. We evaluate the pipeline on the full clause sets of the GDPR (398 clauses) and the EU AI Act (574 clauses). For requirement-bearing clause identification, the approach achieves macro-averaged F1 scores of 0.82 and 0.78, respectively, outperforming a SetFit-based baseline. Human evaluation shows high completeness (4.60 and 4.45) and correctness (3.74 and 3.54) of derived requirements, while explanation clarity scores are near-ceiling (4.92 and 4.94) on a 1-5 scale. We implement the approach in Reg2Req, a publicly released tool that further supports requirement classification, use case seeding, cross-reference analysis, definition indexing, and a traceability matrix to operationalize regulatory compliance in practice. A user study with 25 practitioners shows that the plain-language explanations significantly improve comprehension of derived requirements and confidence in acting on them (p<0.001), and that all participants would use Reg2Req as a starting point for deriving software requirements from a regulation.
Automated Compliance Checking (ACC) systems are fundamentally static, unable to easily adapt to new regulations, project constraints, organizational, or practitioner-defined rules. This paper presents a framework integrating Knowledge Graphs (KGs) and Large Language Models (LLMs) to support a more extensible design review environment. In this framework, the KG acts as a structured repository for rules and executable logic, while the LLM serves as an intelligent interface. The central innovation is the human-in-the-loop feedback mechanism, where new logic generated by the LLM is validated, executed, and permanently stored in the KG, transforming it into an active, evolving validation engine. Following a Design Science Research (DSR) methodology, we implement and evaluate the framework as a prototype embedded as an Autodesk Revit add-in, demonstrating its ability to retrieve and execute existing rules from the KG, capture new requests during design, and maintain a verifiable, adaptive compliance checking system. Across a two-experiment evaluation, the system achieved 100% mapping accuracy for six existing rules, while generating new executable rules from natural language succeeded in 70% of 20 trials. Performance was strong on parameter-based checks (100%) but dropped on rules involving spatial reasoning (20–60%), where the LLM still struggles to produce reliable logic.
Maen Alnuzha, Tanya Bloch· Journal of Information Techn...· 1 citation