Practitioner-derived taxonomies of conversational artificial intelligence (AI) workflow errors show low inter-rater agreement among human coders, leaving open whether the instrument is ill-specified or the judgments are inherently difficult. We delivered a locked eight-category workflow-error taxonomy verbatim, under s...
D. Austria, B. McCollister, J. Lindsey et al.· medRxiv· 0 citations
TRACE (Tracking Reliability of AI-generated Conversational Evidence), a practitioner-audit framework for evaluating the downstream workflow reliability of conversational AI, is presented, suggesting taxonomy legibility under standardized conditions even where human judgment diverged.
D. Austria, B. McCollister, J. Lindsey et al.· medRxiv· 0 citations
An evaluation framework that pairs the Technology Acceptance Model (TAM) with the Medical Condition Regard Scale (MCRS) and is called the TAM-MCRS LLM Evaluation Framework, a novel clinician-centered approach for comparing which LLMs produce the highest-quality post-care patient education across accuracy, appropriatene...
D. Austria, Grace Williams, Christopher Girardo et al.· JMIR Formative Research· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.