Large Language Models as Clinical Support Tools in Drug Information Services: Performance Comparison With Pharmacists

Nuntapong Boonrit Najwa Bin-Useng Aphichaya Sirijariyawat Tanawan Chaithong Suppakun Rattayarungsri Sutima Yodsri A. Hopkins Warit Ruanglertboon

Aug 2026 · Journal of the American College of Clinical Pharmacy · Vol 9 · 0 citations · 33 references

Medicine

TL;DR

LLMs show potential as tools for preliminary drug information retrieval and rapid responses generation in drug information services, however, variable concordance and persistent limitations in citation credibility indicate the need for continued pharmacist oversight.

Abstract

Large language models (LLMs) are increasingly explored for drug information support, yet their reliability and clinical applicability remain uncertain. This study evaluated multiple LLMs in responding to real‐world drug information questions retrieved from a university hospital in Thailand, focusing on clarity in Thai, concordance with pharmacist responses, relevance, context awareness, and citation credibility.

DOI PubMed Semantic Scholar

View source

Similar papers

Open access Aug 2026

Evaluating search-enabled large language model interfaces for mpox public health consultation: a guideline-based comparative study

The evaluated search-enabled LLM interfaces showed heterogeneous performance across safety, accuracy, empathy, reliability/information quality, and readability, which support the need for guideline-based evaluation, source transparency, readability optimization, and robust safety safeguards when such interfaces are eva...

Qi-Qi Zheng, Ru Chen, Ming-Ming Cai et al. · Frontiers in Public Health · 0 citations

Open access Aug 2026

Benchmarking large language models for HIV medical decision support

HIVMedQA is developed, a clinician-curated benchmark of HIV-related open-ended medical question-answer pairs spanning basic knowledge, clinical reasoning, complex patient vignettes, and bias-modified scenarios that provides a structured benchmark for evaluating LLMs in HIV clinical decision support.

Gonzalo Cardenal-Antolin, J. Fellay, Bashkim Jaha et al. · Communications Medicine · 1 citation

Aug 2026

Benchmarking large language models for multilingual health information delivery: a mixed-methods data analytics framework applied to vaccine communication in a middle-income setting

This framework provides a replicable approach to evaluating AI-generated health content across languages and contexts and identified five influential factors: local contextualisation and regulatory anchoring, structured presentation and readability, balance between completeness and accessibility, source credibility, an...

Jazlan Jamaluddin, Karleen Chong, Jin Ian Choong et al. · International Journal of Dat... · 0 citations

Review Jul 2026

Uncertainty-aware extraction of clinical findings from Finnish EHRs using open large language models

Open LLMs can extract clinical findings from Finnish pediatric records with accuracy comparable to published English benchmarks, and uncertainty-based triage substantially reduces required expert workload.

J. Leinonen, J. Knuutila, S. Kurki et al. · medRxiv · 0 citations

#small language model Preprint Aug 2026

Performance of a domain-specific large language model in answering patient questions in psychiatry

MIND was able to answer many questions about escitalopram in a manner deemed accurate, complete, and safe by psychiatrists the majority of the time, and represents a step towards building safe LLM systems to enhance patient education in psychiatry.

Alexander J. Hish, A. Nagendran, S. Compton · 0 citations

Review Open access Aug 2026

The Reliability of Human Evaluation of Large Language Models in Health Care Settings: Scoping Review

It is suggested that the reliability of health care LLM reliability is difficult to evaluate adequately using a single universal standard, and future evaluations of health care LLM reliability need to be guided by standardized evaluation frameworks that reflect domain-specific contexts.

Euijun Yang, S. Ko, Hyekyung Woo · Journal of Medical Internet... · 0 citations