Large language model chatbots for patient-facing erectile dysfunction information: a cross-sectional evaluation of safety, accuracy, empathy, information quality and readability.
Nov 2026· Sexual Health· Vol 23 6· 0 citations· 46 references
Medicine
Abstract
Background
Generative artificial intelligence chatbots are increasingly used for health information, including sensitive conditions, such as erectile dysfunction (ED), but their safety and quality remain uncertain. To compare six large language model chatbot interfaces for ED information across safety, accuracy, empathy, information quality, source-related transparency, readability and response length.
Methods
Thirty-eight standardized ED questions were submitted once to six chatbot interfaces, yielding 228 responses. Five independent raters assessed Safety, Accuracy, Empathy, DISCERN, Ensuring Quality Information for Patients, the Journal of the American Medical Association benchmark and Global Quality Score. Readability and response length were also evaluated. Between-interface differences were tested using Cochran's Q or Friedman tests.
Results
Twenty-six responses (11.4%) were safety-flagged. Safety-flag frequency differed across interfaces (Q = 11.765, P = 0.038), but the effect size was small (0.062; 95% CI, 0.026-0.148), and no pairwise difference remained significant after multiplicity correction. Accuracy was generally high (median 4.00/5 for all interfaces), but differed slightly overall (P = 0.010, Kendall's W = 0.080). Empathy (P < 0.001, W = 0.160), DISCERN (P = 0.003, W = 0.508), Ensuring Quality Information for Patients (P < 0.001, W = 0.576), and Journal of the American Medical Association benchmark scores (P < 0.001, W = 0.888) differed significantly, whereas Global Quality Score did not (P = 0.146). Readability and response length also differed (all P < 0.001).
Conclusion
Large language model chatbots generally provided accurate ED-related information, but performance varied across clinically relevant dimensions. They may support information seeking, but should not replace individualized clinical assessment.
Supporting data, adapters, predictions and code for the article *Low-Cost LoRA Fine-Tuning of Small Language Models for Multi-Step Arithmetic Reasoning* by Jake O'Grady, Asena Isik Gürhan, Chee Fong Ting and Effirul Ramlan (University of Galway). We generated 20,000 GSM8K-derived arithmetic problems with step-by-step s...
O'Grady, Jake, Gürhan, Asena Isik, Chee, Fong Ting et al.· Zenodo (CERN European Organi...· 465 citations
The results are packaged in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible, and the need to shorten time-to-market, by speeding up the development through low-precision engineering activities.
Carmine Giardino, Nicolò Paternoster, M. Unterkalmsteiner et al.· IEEE Transactions on Softwar...· 178 citations· ⚡14
Software startup companies develop innovative, software-intensive products within limited timeframes and with few resources, searching for sustainable and scalable business models.
M. Unterkalmsteiner, P. Abrahamsson, Xiaofeng Wang et al.· e-Informatica Software Engin...· 157 citations· ⚡17
This study conducts a case survey study based on the secondary data of the major pivots happened in 49 software startups, and demonstrates that customer need pivot is the most common among all pivot types.
Sohaib Shahid Bajwa, Xiaofeng Wang, Anh Nguyen-Duc et al.· Empirical Software Engineeri...· 127 citations· ⚡15
The comparison of adopter and non-adopter sample reveals three potential adoption inhibitor, security, data privacy, and portability, which underlines the importance of the technical and security perspectives for research investigating the adoption of technology.
Nattakarn Phaphoom, Xiaofeng Wang, S. Samuel et al.· Journal of Systems and Softw...· 111 citations· ⚡8
The ongoing work building a Raspberry Pi cluster consisting of 300 nodes is presented, with potential use cases being an inexpensive and green test bed for cloud computing research and a robust and mobile data center for operating in adverse environments.
P. Abrahamsson, S. Helmer, Nattakarn Phaphoom et al.· IEEE International Conferenc...· 110 citations· ⚡7
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Senior Research Scientist Jim Glass discusses how his MIT-IBM Computing Research Lab collaboration connects perception and language, and addresses the shortcoming of current models.
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Enforcing physics-based constraints prevents AI models from generating unreliable predictions for complex nonlinear systems in weather, climate, and turbulent flows.
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.