Skip to content

HEALTHASSIST AI: A MULTIMODAL LARGE LANGUAGE MODEL FRAMEWORK FOR INTELLIGENT HEALTHCARE ASSISTANCE

2026 · European Journal of Computer Sciences and Informatics · Vol 3, pp. 181 · 0 citations

TL;DR

The secure authentication and regulatory compliance make HealthAssist a scalable and implementable system for real-world clinical deployment, with demonstrated clinical acceptability across expert panel evaluation.

Abstract

Aim/Background: The HealthAssist architecture is a complex MLLM which seeks to provide a dependable and reliable healthcare assistance through the use of artificial intelligence, particularly in underserved areas. It uses the Mistral Large 3 MoE model of 675 billion parameters while using only 41 billion at any point during execution. The healthassist system has an independent 2.5 billion parameter vision encoder used in the interpretation of medical imagery, alongside a large context of 256,000 tokens. Methods: This system incorporates seven different clinical services that include: medication consultation, disease identification, lab results interpretation, prescription decryption, symptom-based diagnosis, emergency services, and recommending hospitals locally. The architecture is made to ensure security through the use of SHA-256 authentication and compliance with the FDA SaMD and EU AI Act framework. Results: The results showed that the system attained an overall accuracy of 90.54% (CI=89.21-91.87), 94.64% semantic similarity, 86.54% METEOR, 88.12% ROUGE-L, and 92.89% anti-hallucination rate, performing better than other existing state-of-the-art baseline models on three different benchmark databases. Conclusion: The secure authentication and regulatory compliance make HealthAssist a scalable and implementable system for real-world clinical deployment, with demonstrated clinical acceptability across expert panel evaluation.

View source

Similar papers

Review Open access Aug 2026

Large Language Models and Medical AI Systems for Healthcare Diagnosis: A Systematic Review

The rapid growth of artificial intelligence systems (AI systems) has increased interest in the use of patient care and clinical decision-making processes. There is some uncertainty regarding their reliability and safety in clinical practice. A more detailed systematic review of literature examining LLMs applied to healthcare diagnosis was conducted. A PRISMA-based systematic review has been carried out of relevant literature published in the major databases for the years 2022–2025. Key findings include a growing trend to develop multimodal models based on diverse input modalities, combining LLM models with other models as part of clinical workflows. The Usage of complementary methodologies such as retrieval-augmented generation, knowledge graphs, and federated learning is highly expanding, particularly in enhancing the efficiency and accuracy of clinical decision-making processes. Significant challenges such as hallucinations, bias, prompt sensitivity, limited explainability, and inadequate clinical validation continue to pose major obstacles. Although promising, LLM-based systems are not yet reliable enough for autonomous medical diagnosis. Overall, this review contains multiple recommendations for future research in many areas (e.g., LLMs) to ensure a high level of safety, transparency, and clinical applicability for LLMs and other AI/ML-related technologies and devices.

M. U. K. Gunawardhna, Pirunthavi Wijikumar, D. Weerasinghe · 0 citations
#explainable ai Open access Nov 2026

AI health assistant combining transformers and XGBoost for multilingual care

CIMAS HealthMate is proposed, a hybrid multilingual VHA that integrates transformer-based natural language processing (NLP) with an explainable extreme gradient boosting (XGBoost) decision model to provide accurate and transparent symptom triage.

Shamiso Simango, M. Mutandavari · 0 citations
Conference Jul 2026

Med Care AI: A Multimodal Artificial Intelligence-based Healthcare Information and Medical Scan Analysis System

Digital health care platforms continue to cause an increase in global access to medical information, but they haven been fully able to facilitate access through continued fragmentation and lack of accessibility, as the majority of these systems use text-based descriptions of disease alone, with no capability of intelligent interpretation or interactivity and do not support multimodal forms of data (e.g., medical imaging). In this paper, we present a multimodal artificial intelligence-based platform for health care information retrieval and analysis called MedCare AI, which consists of three components: A curated knowledge base of 610 different diseases across 22 different categories from the World Health Organization (WHO) and the Centers for Disease Control and Prevention (CDC); An image analysis module that uses artificial intelligence (AI) to analyze scans for six different types of imaging technology, specifically: X-ray; computed tomography (CT); magnetic resonance imaging (MRI); ultrasound; positron emission tomography (PET); and electrocardiogram (ECG) imaging. Our analysis module utilizes a fine-tuned version of the ResNet-50 convolutional neural network (CNN) built on a defined preprocessing pipeline; A health care assistant that uses natural language processing to drive conversational interaction and uses a bi-directional long-short-term memory (BiLSTM)-based named entity recognition system. MedCare AI's performance outcomes were assessed on a dataset comprised of 500 queries, 300 conversations and 200 scans, achieving an average of 92.6%, 91.4%, 90.8%, and 91.1% on accuracy, precision, recall and F1 score respectively, when compared to current chat-based applications, demonstrating up to a 6.2% improvement over standalone chat-based applications. Robustness evaluation results identified less than 2.1% degradation of F1 score as a result of three differing levels of noise; demonstrating that the platform's capabilities as a multimodal integrative system meet the current gaps identified within existing literature, by providing access to disease knowledge retrieval, scan-based diagnostic and conversational interaction from a single easy to use Internet-based platform.

Parumanchala Bhaskar, A.Nageswari, B.Anjani Pranitha et al. · 0 citations
Review Open access Aug 2026

Explainable Artificial Intelligence for Tabular Data in Healthcare: A Systematic Review of Methods, Evaluation, and Applications

Explainable Artificial Intelligence (XAI) has emerged as a critical enabler for the adoption of machine learning models in high-stakes domains such as healthcare. While significant progress has been made in XAI for computer vision and natural language processing, tabular data—the predominant format of electronic health records—presents unique challenges and opportunities. This systematic review provides a comprehensive analysis of XAI methods specifically applied to tabular healthcare data for classification tasks. We examine 21 primary studies published between 2020 and the first half of 2026, covering three complementary perspectives: (1) intrinsically interpretable models, (2) post-hoc methods including LIME, SHAP, and their variants, and (3) evaluation frameworks that assess both model-centered fidelity and human-centered clinical alignment. Our analysis reveals that SHAP remains the dominant post-hoc method, achieving strong model fidelity but showing inconsistent alignment with clinical expert reasoning. Key findings include the significant impact of class imbalance on explanation consistency, the importance of clinician-centered evaluation, and the emergence of hybrid approaches integrating XAI with generative AI and transfer learning. We identify critical gaps, including limited adoption of XAI in AutoML pipelines, lack of standardized evaluation metrics, and predominance of single-institution validation studies.

Angelower Santana-Velásquez, M. B. Salazar-Sánchez · 0 citations
Open access Jul 2026

A Locally Executable AI System for Improving Preoperative Patient Communication: Multidomain Clinical Evaluation

By decoupling clinical information retrieval from generative chitchat, LENOHA enhances safety, preserves privacy, and markedly reduces energy use, offering a practical blueprint for sustainable and equitable medical AI deployment across diverse care settings.

Motoki Sato, Sou Nagata, Mizuho Ohnuma et al. · 0 citations
Open access Aug 2026

Ethical considerations for multimodal artificial intelligence in healthcare

Multimodal artificial intelligence (MMAI) is transforming biomedicine by integrating heterogeneous data, e.g., images, speech, behavior, physiological signals, and text, into unified representational spaces. This enables powerful cross-modal inference and data synthesis, with potential gains in diagnostic accuracy, early detection, and patient support. However, these capabilities introduce ethical challenges that exceed existing AI governance frameworks. MMAI can infer sensitive information without patient awareness, and can convert such inferences into new data objects (e.g., images, clinical text) that enter medical records without clear provenance, acquiring the practical status of observed clinical facts. This raises ethical concerns around the infrastructural emedding of inference-based data objects as durable, reusable clinical and research data. The procedures and technical pipelines that govern how such data are classified and integrated into clinical and research infrastructures embed consequential decisions about provenance, attribution, and contestability, often made in advance of adequate governance. In this Perspective, we characterize what distinguishes MMAI-generated data from other forms of algorithmic inference and argue that MMAI is ethically novel in part because it renders cross-modal inferences as recordable data objects, thereby blurring the boundary between observation and generation. We therefore argue for a shift from data-centric protection toward governance of inference and infrastructuring. We propose a four-part agenda: (1) provenance labeling as a prerequisite for accountability; (2) evidence-building to track emergent inference capacities; (3) dynamic consent models responsive to evolving capabilities; and (4) privacy-preserving techniques to limit unjustified or unconsented inferences. These steps aim to support innovation while safeguarding individual rights and expectations.

K. Kostick-Quenet, Jennifer K. Wagner, Laura Y. Cabrera et al. · 0 citations