Retrieval-Augmented Large Language Model for Institutional Knowledge Management and Decision Assistance in Public Organizations

Abstract

The increasing volume and complexity of institutional documents in public organizations create challenges in accessing reliable knowledge for administrative processes and evidence-based decision-making. Conventional knowledge management systems often rely on keyword-based retrieval, while standalone Large Language Models (LLMs) may generate inaccurate responses when processing domain-specific institutional information. This study proposes a domain-specific Retrieval-Augmented Generation (RAG) framework to enhance institutional knowledge management and AI-assisted decision support in public-sector organizations. The framework was developed using a Design Science Research approach with Universitas Malikussaleh as a case study. The proposed architecture integrates institutional knowledge base construction, semantic retrieval, grounded language generation, and source attribution mechanisms. A knowledge base comprising 416 official institutional documents was developed through document preprocessing, semantic chunking, embedding generation, and vector database indexing. The framework was evaluated using 200 institutional queries based on retrieval performance, response quality, explainability, and system efficiency metrics. The results demonstrate effective retrieval capability, achieving Precision@5 of 0.884, Recall@5 of 0.921, and Mean Reciprocal Rank of 0.895. Generated responses achieved 94.6% factual accuracy, 91.8% contextual relevance, and 96.5% source attribution accuracy, while the hallucination rate was reduced to 3.2%. Furthermore, the framework achieved an average response latency of 1.18 seconds, indicating practical feasibility for institutional applications. These findings demonstrate that integrating semantic retrieval with grounded LLM generation can improve knowledge accessibility, transparency, and reliability for AI-assisted decision support in public organizations. The proposed framework provides a practical foundation for trustworthy institutional knowledge services and supports more efficient, explainable, and evidence-based administrative decision-making across diverse institutional contexts