Findings support RAG as a promising approach to improve the accuracy, consistency and safety of LLM-driven DMHIs through a controlled comparison of RAG-enabled versus RAG-disabled modes.
Abstract
Digital mental health interventions (DMHIs) offer scalable support, but ensuring they accurately detect users'intent during volatile situations can be challenging. Pure parametric Large Language models (LLMs) do not contain specific safety critical architecture, and can miss critical cues, or hallucinate, undermining reliability. Retrieval Augmented Generation (RAG), which supplements an LLM with retrieved context, could enhance intent detection during volatile situations. Commercially available DMHIs typically combine multiple independent safety layers like rule-based filters, symbolic escalation protocols, and neural classification. The incremental contribution of any single layer, however, remains unquantified. This paper evaluates six LLM models within a DMHI called Wysa, via a controlled comparison of RAG-enabled versus RAG-disabled modes. Anonymized real and synthetic user-chatbot exchanges were annotated by a qualified clinical team against multi-class intent categories (e.g. self-harm, abuse, panic). The study computed classification accuracy, recall, precision and F1 scores against ground truth labels and tested differences for statistical significance. Performance was also examined by risk category and inter-model agreement. While RAG caused a rise in false alarms, the trade-off is consistent with safety-critical design principles that prioritize sensitivity, where flagged cases are routed to additional review rather than acted on directly. Overall, these findings support RAG as a promising approach to improve the accuracy, consistency and safety of LLM-driven DMHIs. Keywords: Digital Mental Health Intervention, Large Language Model, Retrieval Augmented Generation, Accuracy, Recall, Precision
Background/Objectives: Large language models (LLMs) show promise for clinical decision support, yet their accuracy in interpreting specialized medical guidelines remains uncertain. Retrieval-augmented generation (RAG) may enhance performance by grounding responses in authoritative knowledge bases. This study aimed to compare the accuracy, comprehensiveness, and safety of RAG-enhanced versus standard LLMs for answering clinical questions derived from the German S3 guideline for oral cavity carcinoma. Methods: We conducted a prospective, single-blind benchmark study evaluating six LLMs: one RAG-enhanced model (Custom GPT with guideline access), one consensus-based model (ConsensusGPT), and four standard models (DeepSeek-V3.2, Mistral Small 3.2, Qwen3-Next-80B, GPT-OSS-120B). Fifty clinical questions covering 17 guideline domains were presented to each model three times, yielding 900 evaluations. Three expert reviewers assessed responses using 5-point Likert scales for accuracy, comprehensiveness, and clarity, under a single-blind procedure, the effectiveness of which was tested by a pre-specified manipulation check. We then ran a paired within-model experiment in which each base model was queried with and without guideline access through a transparent, openly released retrieval pipeline, and scored every response with a condition-blind automated judge alongside deterministic retrieval metrics computed from the logs. Secondary outcomes included hallucination rates and guideline citation behavior. Inter-rater reliability was assessed using intraclass correlation coefficients (ICCs). Results: In a paired within-model design that held each base model fixed, adding transparent guideline retrieval improved accuracy—significantly in the three weaker open-weight models (Mistral, Qwen3, and GPT-OSS) and directionally in the already-strong DeepSeek and GPT-5 bases. Because a pre-specified blinding check found that experts could still identify retrieval-augmented answers with 98.5% accuracy, we anchored causal interpretation on measures that do not depend on the human raters, ranked by their independence: deterministic, log-derived retrieval metrics first, and then an automated, condition-blind LLM judge, whose agreement with the experts (Spearman ρ = 0.81, 95.7% within-one agreement) establishes shared calibration rather than independence from their bias. Deterministically from the retrieval logs, citation groundedness rose from 0% to 51–89% and retrieval recall@5 was 92%. On the judge, content-level hallucination fell from 42% to 4% and accuracy rose by a pooled +0.64 points (95% CI 0.47–0.80); the accuracy gain persisted after adjustment for response length (+0.48, 95% CI 0.22–0.73), which retrieval shortened rather than lengthened. The accuracy gain was large for weaker base models and small or non-significant for already-strong ones, whereas the hallucination and auditability gains were consistent across all models. The human ratings reproduced the judge’s accuracy effect (+0.61, 95% CI 0.49–0.74), and GPT-5 run through the transparent pipeline showed no significant difference from the proprietary Custom GPT (judge accuracy 4.48 vs. 4.58). Conclusions: Guideline retrieval yields a reproducible, largely base-independent improvement in the safety and auditability of LLM answers to clinical guideline questions, with accuracy gains concentrated in weaker base models. Because retrieval-augmented answers are recognizable to experts, rigorous evaluation should rely on rater-independent measures, and residual hallucination continues to require human oversight.
Andreas Vollmer, Lara Schorn, Felix Schrader et al.· Diagnostics· 0 citations
Current Information Retrieval (IR) systems, including conversational IR paradigms enhanced by Retrieval-Augmented Generation (RAG), assume that users can formulate coherent queries and reliably interpret retrieved information. In dementia care contexts, these assumptions fail: queries are unstable, feedback signals are unreliable, relevance fluctuates with cognitive state, and multi-user interaction is essential. Dementia impacts over 50 million people globally, yet current RAG research has yet to consider use cases where users have dementia or Mild Cognitive Impairment (MCI), and so RAG systems continue to be designed without considering people with dementia, MCI, or their caregivers as users, creating a fundamental mismatch between system capabilities and this population's needs. Our perspective outlines functional requirements, not optional features, for RAG systems serving cognitively diverse populations, including temporal user modelling that tracks cognitive trajectories over time, caregiver-in-the-loop retrieval that enables verification and oversight, consent-aware evidence access that respects fluctuating capacity, and cognitive-load-aware presentation that adapts complexity to comprehension level. These requirements would collectively transform RAG from a cognitively-stable user paradigm into a cognitively-adaptive one. Drawing on participatory workshops with 41 members of the public and dementia-care community, our perspective was shaped through empirical evidence from those with lived experience of dementia or MCI. We advocate for inclusive IR through our cognitively-adaptive RAG that supports cognitive independence in dementia care, and call on the IR community to advance accessibility and equity through inclusive retrieval system design.
Claire Rogers, Asmaa Z. A. M. Alqadri, Fiona A. Beaton et al.· Annual International ACM SIG...· 0 citations
The rapid growth of digital platforms has accelerated public access to online health information, but it has also become a primary channel for the spread of health misinformation. Large Language Models (LLMs) have been widely adopted to address this problem through pure generative approaches; however, they frequently produce convincing yet inaccurate information, a phenomenon known as hallucination. This study implements a Retrieval-Augmentation Generation approach for automatic fact-checking of health claims using the PubHealth dataset, and compares its performance against a pure generative (Non-RAG) model. The proposed pipeline combines a FAISS-based semantic retriever, CrossEncoder re-ranking, and a FLAN-T5 generator to classify each claim as Fact or Myth while supplying relevant supporting evidence, and is integrated into a Gradio-based interface for claim verification and dataset expansion. The models were evaluated using accuracy, precision, recall, F1-Score, Cohen’s Kappa, faithfulness, context precision, and answer relevancy across four scenarios of question and claim normalization. The experimental results show that the RAG model outperformed the Non-RAG model on the classification metrics, with the best configuration achieving an F1-score of 0.7823, an accuracy of 0.7857, and a Cohen's Kappa of 0.5532, whereas the Non-RAG model suffered from extreme bias toward the Fact label and failed to detect the Myth label, reaching an accuracy of only 0.4211. Claim normalization was found to substantially improve classification accuracy, while the low faithfulness scores of the RAG model reflect a limitation of the metric on short binary answers rather than hallucination. These findings confirm that the retrieval and augmentation stages contribute significantly to producing more accurate, transparent, and evidence-based verification of health claims.
ReFrame is a training-free multimodal input reframing framework where two agents share a lightweight locally deployed MLLM: the evidence-generation agent constructs complementary risk and utility evidence, and the rewrite-and-routing agent converts it into a safe proxy prompt and image-routing decision before calling the downstream MLLM, without modifying it or accessing its internal information.
Wenzheng Jiang, Xuankun Rong, Yuanzhao Zhai et al.· 0 citations
Diabetes management requires continuous, personalized guidance for medication use, nutrition, glucose monitoring, and daily lifestyle decisions. However, many digital health tools cannot provide comprehensive support tailored to each patient’s medical condition and changing needs. Rule-based conversational systems can be safe but are often rigid and limited in natural interaction, whereas standalone large language models (LLMs) can produce fluent responses but may generate unsupported or clinically unsafe information, lack clear source attribution, and cannot reliably use patient-specific information or updated medical knowledge. Other challenges include limited personalization and fragmented diabetes knowledge distributed across clinical guidelines, patient records, nutritional databases, glucose-monitoring data, and medication safety resources. To address these limitations, this paper presents a Retrieval-Augmented Large Language Model (RA-LLM) architecture for trustworthy conversational AI in diabetes care which is a part of our AI-based Diabetes Care (AIDCare) mHealth solution. The proposed framework integrates clinical guidelines, structured electronic health record attributes, nutrition knowledge, optional glucose-monitoring context, and pharmaceutical safety rules. It combines clinical entity recognition, hybrid dense–sparse retrieval, patient-context filtering, cross-encoder reranking, evidence-constrained prompt construction, and multi-layer safety verification to generate personalized, evidence-supported conversational responses. The proposed system was evaluated on a controlled benchmark of 10,000 synthetic diabetes interaction scenarios covering medication guidance, nutrition, glucose monitoring, complication awareness, and lifestyle management. Compared with a vanilla LLM baseline, the RA-LLM improved evidence-grounded response correctness from 78.3% to 94.7%, increased source attribution reliability from 34.0% to 94.0%, and reduced the predefined safety-violation rate by approximately 89%. These findings show the potential of multi-source retrieval, patient-context integration, and safety-aware conversational generation for improving AI-assisted diabetes self-management. As the evaluation is based on synthetic scenarios, real-world clinical validation remains necessary.
Muhammad Jamil, A. Kavak, Sema Bayraktar et al.· IEEE Access· 0 citations
It is suggested that multimodal safety cannot be enforced reliably without modeling the requested operation, the visual target to which it applies, and the confidence of that grounding.