Research on Dynamic Judicial Hallucination Risk Identification Mechanism for Legal Domain Large Language Models

Abstract

Legal domain large language models are increasingly used in legal consultation, precedent retrieval, document generation, and AI-assisted adjudication. While these models improve the efficiency of legal information processing, legal hallucinations such as fabricated cases, misquoted statutes, distorted holdings, and broken reasoning chains may undermine the reliability of judicial grounds, the clarity of liability allocation, and procedural legitimacy. To identify hallucination risks in AI-generated legal opinions, this study constructs a dynamic risk identification mechanism based on verifiable legal corpora collected from public statutes, judicial interpretations, judicial documents, and typical cases. The mechanism integrates retrieval-augmented generation, semantic consistency detection, citation validity assessment, and reasoning-chain completeness scoring to classify model outputs into different risk levels. The experimental results show that the proposed model outperforms ordinary prompting, RAG prompting, and self-consistency detection in accuracy, recall, F1-score, and AUC, achieving an overall accuracy of 0.86 and a high-risk recall of 0.89. The mechanism transforms legal hallucination from an opaque generation error into a computable, explainable, and reviewable judicial risk, providing technical support for algorithmic transparency, human oversight, and responsibility allocation in AI-assisted justice.