Large Language Models (LLMs) achieve strong reasoning performance, but their robustness to realistic lexical corruption remains poorly understood. We evaluate four open-weight instruction-tuned models and frontier models across four reasoning benchmarks under keyboard noise, character swaps, and filler insertion. Character-level perturbations substantially degrade accuracy, especially on multi-step reasoning tasks, while filler insertion has little effect. We trace this asymmetry to Attention Diversion: lexical corruption fragments subword tokenization, and the resulting fragments attract disproportionate attention mass, concentrated in middle and final transformer layers. Length-matched controls confirm that fragmentation, not prompt length, drives the loss. A factorial intervention then shows why the damage is hard to undo: fragmentation corrupts token content and attention allocation together, and the two are coupled. Restoring clean attention while the content remains corrupted is actively harmful, restoring content alone is insufficient, and only restoring both recovers a substantial share of the gap. This coupling explains why inference-time strategies, including chain-of-thought prompting, spell-checking, self-repair, and stronger repair models, fail to consistently recover performance: each addresses one channel at a time. Code and data are available at https://github.com/Jiaqian-Janelle/Attention-Diversion
Jiaqi Zhu, Yang Zhang, Junhua Ding et al.· 0 citations
OBJECTIVE
Medical Concept Normalization (MCN) maps informal health phrases to formal clinical concepts. It is a critical task for pharmacovigilance, patient record analysis, and health-related text mining. Extensive MCN research has mainly relied on single-source datasets and overlooked data quality (DQ) issues. This study aims to develop a quality-aware data fusion framework for MCN using Large Language Models (LLMs).
METHODS
The methods consist of DQ evaluation and enhancement, LLM-based data augmentation, and multi-source MCN dataset fusion using enriched concept-phrase pairs. We analyze six widely used MCN datasets-AskAPatient, CADEC, COMETA, PsyTAR, TwADR-S, and TwiMed-each collected from social media and mapped to SNOMED-CT. Our evaluation metrics include correctness, concept validity, coverage, semantic variety, and class imbalance. For DQ enhancement and data augmentation, we use Gemini for zero-shot and few-shot learning to increase the semantic variety and phrase count for rare concepts. We then perform data fusion based on shared medical concepts.
RESULTS
The DQ evaluation reveals substantial issues, including incorrect mappings, invalid concepts, low-variety redundant phrases, and long-tail concept-phrase distribution. After augmentation, phrase counts increase by 172.1%, and by 450.0% after fusion. To directly investigate model performance improvement on rare cases, we introduce concept-level macro metrics. SapBERT, KNN-BioEL, and KRISSBERT trained on the augmented and fused dataset achieve significant gains in accuracy, recall, precision, and F1 over single-source baselines, up to 38%.
CONCLUSION
Our study finds that existing MCN benchmarks present data quality issues and underexplored data fusion potential. Data quality enhancement and LLM-based controlled-variety data augmentation help alleviate overlapping phrases and long-tail issues. Moreover, quality-aware data fusion can expand conceptual coverage, improving MCN performance. This work highlights data quality evaluation and fusion strategies are effective in advancing MCN. We hope these contributions could support quality-guided large-scale MCN data generation with minimal label costs, strengthen reliable biomedical text mining and downstream applications.
AVAILABILITY
https://github.com/yhZHOU515/DataFusion4MCN.
Yuhan Zhou, Ruochi Li, Ana Cleveland et al.· Journal of Biomedical Inform...· 0 citations