This single-annotator pilot documents concrete failure modes rather than population rates and supports evaluating surface accuracy, term-script rendering, chunk-level script consistency, and medical-meaning preservation separately.
Abstract
AI_LectureNote is a historical, readability-oriented post-ASR workflow for Korean-English medical lectures. It rewrites speech-to-text output into study transcripts while restoring Latin-script medical terms rather than Korean phonetic transliterations. We retrospectively evaluate the workflow on four author-recorded lectures across five conditions. In this pilot, post-processing raised the macro English-script rendering rate from 0.39 to 0.71 on the whisper-1 path and from 0.26 to 0.65 when applied to 3-minute chunked gpt-4o-transcribe output. However, English-script rendering did not imply semantic faithfulness: the two post-processed conditions showed semantic drift in 34 and 36 of 282 reference sentences and polarity failures in 11 and 13 of 101 polarity-cue rows. A descriptive cross-input comparison suggested different candidate failure patterns: polarity-failure sets overlapped more strongly across front-ends (Jaccard 0.60; 9 shared of 15 unioned failures) than general semantic-drift sets (Jaccard 0.23; 13 shared of 57 unioned drifts). This single-annotator pilot documents concrete failure modes rather than population rates and supports evaluating surface accuracy, term-script rendering, chunk-level script consistency, and medical-meaning preservation separately.
INTRODUCTION
Locally deployed speech-to-text systems such as Whisper enable privacy-preserving transcription of medical encounters. However, the resulting transcripts are often lengthy, noisy, and insufficiently structured for direct integration into clinical documentation workflows. Large language models (LLMs) can tr...
Ahmad Wattar, Jan Christoph, Christoph Demus et al.· Studies in Health Technology...· 0 citations
A budgeted workflow is proposed, combining source novelty, peer disagreement, and stronger candidate-aware judging to allocate human review through Pali-to-English translation, indicating that evaluator strength matters beyond the prompt alone.
This paper reviews three research streams that have developed largely in isolation, namely LLM translation quality and automatic evaluation, human–AI collaboration and knowledge-worker productivity, and audiovisual translation, and integrates them through an operations-management input–process–output–outcome framework...
We submit M\'eTRON-FR, a 125M GPT-2 pretrained on 92.47M words of French, to the BabyLM 2026 Strict track. It scores 85.97 +/- 0.17% on QFrBLiMP (a native Quebec-French benchmark of grammatical minimal pairs) and 62.80% on the BabyLM-weighted leaderboard. A cross-lingual GLUE (General Language Understanding Evaluation)...
Vocabulary learning research increasingly calls for curriculum-aligned evidence and methods that reflect how vocabulary is selected, sequenced, and assessed in real educational systems. However, stream-differentiated curricular pathways may entail distinct lexical demands that are rarely quantified for morphologically...
K. Palthamburaj, Mercedes Premalatha Ramesh· Education sciences· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.