Aug 2026· Advanced Electromagnetics· Vol 15, pp. 758-768· 0 citations
TL;DR
This study proposes a multi-stage intelligent documentation framework that integrates a 12-layer Conformer architecture, BERT-BiLSTM-CRF semantic modeling, and BART-based structured text generation that illustrates the effectiveness of deep time-frequency feature extraction and contextual sequence modeling for complex noisy signals.
Abstract
Accurate clinical speech recognition remains challenging because rapid pronunciation, domain-specific terminology, and background noise often degrade automatic speech recognition and subsequent medical record generation. This study proposes a multi-stage intelligent documentation framework that integrates a 12-layer Conformer architecture, BERT-BiLSTM-CRF semantic modeling, and BART-based structured text generation. The Conformer encoder captures both local acoustic characteristics and long-range contextual dependencies, while the semantic module performs medical entity recognition and normalization to enhance terminology consistency. The extracted information is subsequently incorporated into a BART generator with clinical knowledge prompts to produce standardized SOAP-compliant medical records. Experimental results demonstrate a word error rate of 6.3%, medical term accuracy of 95.8%, low response latency of approximately 940–960 ms, and generation quality approaching physician-written records. Beyond clinical documentation, the proposed framework illustrates the effectiveness of deep time-frequency feature extraction and contextual sequence modeling for complex noisy signals, offering methodological insights for electromagnetic signal interpretation, antenna measurement data processing, and intelligent information extraction in propagation-related applications.
In natural language understanding, intent recognition plays a central role in human–computer interaction. However, in elderly-care scenarios, acoustic signals are often affected by atypical speech patterns, slower speaking rates, and environmental noise, making audio information less reliable and reducing the effectiveness of conventional text–audio fusion methods. To address this problem, we propose a prompt-guided and quality-aware text–audio intent recognition framework. Specifically, a χ2-based intent prototype soft prompt is introduced to enhance the semantic representation of text. Then, a residual-free text-guided cross-attention module is designed to refine degraded acoustic features using textual semantics as reliable guidance. In addition, a dynamic fusion gate is developed to adjust the contributions of text and audio based on modality reliability and intent-related information. Experiments on the MIntRec dataset with simulated acoustic degradation show that the proposed model achieves 60.90% accuracy, 60.80% weighted F1, and 57.90% macro-F1, outperforming several competitive baselines. These results indicate that the proposed framework can improve the robustness of intent recognition under challenging acoustic conditions in elderly-oriented interaction scenarios.
Zhi-Min Wei, Shu-Hao Tian, Yanzhen Wang et al.· Italian National Conference...· 0 citations
Abstract Background Clinical documentation burden contributes significantly to physician burnout, with health care professionals spending much of their time on electronic health record interactions. Automatic speech recognition (ASR) systems offer a promising solution; however, their application in Korean medical settings faces unique challenges due to widespread Korean-English code-switching, where clinicians routinely alternate between Korean conversational language and English medical terminology within single utterances. Objective This study aimed to develop and evaluate a hybrid postprocessing approach combining medical terminology dictionary normalization with large language model (LLM)–based postprocessing to improve ASR accuracy for Korean-English code-switched medical speech. Methods We constructed a speech dataset from 23,652 nursing progress notes, with a linguistic composition of 67.73% (512,626/756,866) Korean, 23.54% (178,166/756,866) English, and 8.73% (66,074/756,866) numerals or special symbols. Four Korean nurses recorded the notes using 5 microphone types in an acoustically isolated environment. Speech recognition was performed using OpenAI’s gpt-4o-transcribe model. For postprocessing, a medical terminology dictionary containing 1070 mapping entries was constructed from 1000 nursing progress notes to normalize Korean phonetic renderings of English medical terms. Six LLMs (2 GPT and 4 Claude variants) were then evaluated across 5 temperature settings (0.0‐0.8). Performance was assessed using BERTScore (bidirectional encoder representations from transformers score; F1), Sentence-BERT cosine similarity, word error rate, and character error rate (CER), comparing postprocessed outputs against the original written notes. Statistical significance was assessed using paired Wilcoxon signed-rank tests with Holm correction (α=.05). Results Temperature optimization showed that all postprocessing models had small temperature-related effect sizes (all |Cohen dz| ≤0.15), with GPT-4o exhibiting the largest dependency and statistically significant improvement at temperature 0.6 (Holm-adjusted P<.001 for both BERTScore and CER) and the 4 Claude variants and GPT-4.1 exhibiting practically consistent performance across all settings. Baseline ASR achieved a BERTScore of 0.9131 and CER of 0.2336. Dictionary-based normalization performed 43,507 word-level substitutions in 70.8% (16,754/23,652) of transcribed sentences. LLM-only postprocessing reduced CER by 36.09% (Claude Sonnet 4) and 32.53% (GPT-4o) compared to baseline. The combined dictionary-LLM approach achieved the best performance: Claude Sonnet 4 attained a BERTScore of 0.9638 and CER of 0.0820, representing a 64.9% reduction in CER from baseline (P<.001). Conclusions The hybrid pipeline integrating rule-based dictionary normalization with LLM postprocessing significantly improved Korean-English code-switched medical ASR accuracy. Dictionary-based normalization yielded consistent additional improvements over LLM-only postprocessing for both GPT-4o and Claude Sonnet 4. As this modular framework requires no model retraining, it offers a practical means of mitigating multilingual challenges in medical ASR.
Chanryeong Oh, Y. Hwangbo, Wonjoong Cheon et al.· Journal of Medical Internet...· 0 citations
This paper investigates how multilingual medical adaptation reshapes the internal representations of Whisper models through layer-wise encoder analysis, and shows that English medical fine-tuning produces the dominant encoder shift, whereas multilingual continuation largely preserves the adapted representation space.
Souranil Kahali, Rituparna Bose, Abner Hernandez et al.· 0 citations
Automatic speech recognition (ASR), or speech-to-text (STT), is becoming an important interface for AI systems in diagnostic workflows, but general-purpose ASR models often degrade in specialized technical domains. In diagnostic applications such as fault identification, root cause analysis, and repair recommendation, general-purpose ASR systems struggle with domain-specific terminology, abbreviations, part identifiers, and measurement expressions, leading to elevated transcription errors. This work presents a domain adaptation pipeline that unifies three components: a synthetic benchmarking framework in which domain-specific technical text is converted to speech via text-to-speech~(TTS) synthesis and transcribed by open-source ASR models to establish baseline performance; Low-Rank Adaptation~(LoRA)-based fine-tuning of Whisper Large-v3 using those synthetic audio-text pairs; and transfer validation on curated real-world automotive YouTube recordings to assess generalization beyond synthetic conditions. Using automotive technical language as a representative diagnostic domain, a data-scaling study employing progressively larger subsets of in-domain training data evaluates performance on a held-out test set via word error rate~(WER), character error rate~(CER), normalized error metrics, alphanumeric error rate, semantic similarity, and Bidirectional Encoder Representations from Transformers Score~(BERTScore). Results show consistent gains from lightweight domain adaptation on both held-out synthetic data and real-world recordings, confirming that synthetic data generation combined with LoRA-based fine-tuning is an effective and computationally practical strategy for improving ASR accuracy in specialized technical domains where labeled speech is scarce.
Aman Kumar, Ahmed K. Farahat, Huimin Zhuge et al.· PHM Society European Confere...· 0 citations
Cognitive impairment (CI) is a growing public health concern. Early and accurate diagnosis is critical for enabling timely intervention and improving patient outcomes. Speech-based CI detection has emerged as a promising non-invasive approach, as speech signals encode both linguistic and acoustic markers associated with cognitive decline. Recent advances in large language models (LLMs) further strengthen the potential of speech-based assessment by enabling more expressive representation learning and improved generalization across diverse speakers, recording devices, and clinical environments. Moreover, multimodal learning by jointly modeling linguistic and acoustic features allows for a more comprehensive characterization of cognitive and behavioral changes related to CI, leading to more reliable detection. In this work, we propose a multimodal CI detection framework based on open-source LLMs that integrates speech audio and corresponding transcripts while preserving patient privacy. Acoustic embeddings are extracted directly from speech signals, while textual embeddings are generated from automatically transcribed speech. These modality-specific embeddings are then concatenated to create a combined feature vector and used for downstream classification, without requiring access to raw or sensitive patient data. The proposed approach is evaluated on the ADReSS20 and ADReSSo21 benchmark datasets. Experimental results show that the proposed multimodal framework achieves an CI classification accuracy of 92.4% and consistently outperforms single-modality baselines. Our work establishes a new state-of-the-art for CI identification, with the proposed method demonstrating superior cross-dataset generalization. This advance highlights the power of an LLM-based multimodal framework that fuses linguistic and acoustic data to enable robust, scalable, and non-invasive screening.
Ying-lei Huang, Xin Wang, Yuhan Su et al.· 0 citations
Developing Automatic Speech Recognition (ASR) systems for specialized medical domains is challenging in lowresource settings due to the limited availability of annotated speech data. Synthetic speech generated using text to speech (TTS) systems is often used to augment training data, but directly mixing synthetic and real speech can introduce distribution mismatch that degrades encoder representations. This work introduces Real-Governed Representation-Calibrated Training (RG-RCT), a training strategy that regulates the influence of synthetic speech at the representation level. The proposed framework combines reliability-based confidence estimation, representation alignment, and layer-wise representation governance to stabilize the encoder feature space while preserving acoustic diversity from multi-speaker synthetic data. Experiments on medical-domain speech datasets using Wav2Vec2 and Whisper demonstrate that RG-RCT consistently outperforms conventional strategies such as direct mixing and confidence-aware training. The proposed method achieves the lowest Word Error Rate (WER), reaching 17.1% and 16.6% for Telugu and 23.3% and 25.6% for Kannada. These results indicate that regulating encoder representation geometry enables more effective utilization of synthetic speech for low-resource medical ASR.
S. Devarakonda, Ravi Sastry Kolluru, Srihari Bandarupalli et al.· International Conference on...· 0 citations