An Evidence-Constrained RAG Learning Support System for Interdisciplinary Smart Water Engineering Education: Multi-Source Knowledge Governance and Reproducible Technical Evaluation

Abstract

Smart water engineering is inherently interdisciplinary, requiring students to synthesize hydrological and hydraulic principles, observational data, domain-specific models, engineering standards, and computational tools to support evidence-based reasoning and decision-making. Yet, the knowledge required for such learning is fragmented across textbooks, technical standards, model manuals, and software documentation, while general-purpose large language models (LLMs) provide limited guarantees regarding the relevance, provenance, and traceability of generated information. To address this challenge, this study adopted a design science research approach to develop and technically evaluate an evidence-constrained retrieval-augmented generation (RAG) framework for interdisciplinary smart water engineering education. In this article, trustworthiness is used in a deliberately operational sense: provenance and version traceability, evidence support, citation-scope control, and boundary-aware refusal. It does not denote demonstrated user trust, professional correctness, or learning improvement. The framework integrates multi-source knowledge governance, hybrid retrieval, reranking, citation constraints, evidence-gap detection, and structured refusal into an auditable evidence-support pipeline. The frozen knowledge base comprised 159 registered sources, 17,295 retrievable chunks, and 826 aligned Chinese–English technical terms. Evaluation on a frozen 150-question benchmark showed that the full framework achieved a Recall@5 of 0.8256 and an nDCG@10 of 0.7717, exceeding the lexical-retrieval baseline by 0.3233 and 0.3081, respectively. On a separate 180-question refusal benchmark, it achieved an F1 score of 0.8451 while maintaining an acceptance rate of 0.9267 for answerable questions. Under a 20-question cross-domain holdout, evidence-gap cascading increased Recall@5 from 0.5750 to 0.7000; this small-sample result is exploratory and does not establish generalizable cross-domain performance. The author-retained frozen records, analysis scripts, prompt versions, and hashes can support independent verification when legally shareable materials are provided upon reasonable request, whereas calls to hosted model services and access to restricted or third-party source documents cannot be reproduced exactly without the same provider state and permissions. No students or domain experts participated in the evaluation. The findings therefore establish technical feasibility and evidence-auditability, not educational effectiveness or expert-validated professional correctness.