A multi-agent GraphRAG framework for pharmacotherapy safety verification in clinical decision support systems

Abstract

Background The management of pharmacotherapy in patients with multimorbidity and polypharmacy is a difficult task in routine clinical practice. Clinicians must consider diagnoses, prescribed drugs, contraindications, drug-drug interactions, renal and hepatic function, laboratory values, dose limits and individual risk factors. Clinical decision support systems in traditional settings are based on static rules, and hard to update. Standalone large language models are capable of processing clinical language but their outputs may be unsupported by evidence, incomplete or out of date. In this study, we developed and evaluated a hybrid multi-agent GraphRAG framework for personalised pharmacotherapy safety verification. Methods We developed a system that integrates graph based retrieval with large language models. The framework was composed of three parts: (1) a schema for a pharmacotherapy knowledge graph that encoded indications, contraindications, interactions, laboratory thresholds, dose limitations and relevant patient conditions; (2) an Extractor–Critic workflow that transformed unstructured medical text into graph elements with quality checks prior to their incorporation; and (3) a retrieval and re-ranking module that leveraged semantic search, graph traversal and safety-directed scoring. The system was evaluated on 12 benchmark cases extraction and 30 clinical safety questions answering. Comparison of conventional RAG and GraphRAG was done using linear mixed effects models, permutation testing, and paired case-level analysis. Results The multi-agent extraction workflow beat the zero-shot baseline. Strict F1 improved by 0.156 points (p < 0.01), mainly due to higher ecall (+0.172, p < 0.001). For clinical question answering, we found that GraphRAG outperformed conventional RAG in terms of medical accuracy (0.545 vs. 0.443, absolute difference of 0.102, p = 0.04). GraphRAG provided the better answer 66.7% of the time. The advantage was greatest where safety was dependent on contra-indications, laboratory cut-offs or patient specific factors. Smaller models performed as well as the larger models with the same GraphRAG configuration. Conclusion The evaluation revealed that the integration of LLM-based extraction with a pharmacotherapy knowledge graph enhanced safety-oriented clinical decision support. GraphRAG provided more accurate and traceable answers than traditional RAG. Routine adoption will require validation on larger graphs, a wider range of datasets and real clinical populations, but this approach may be appropriate for cost sensitive and privacy conscious workflows.