Quality-aware multi-source data fusion and enhancement for Medical Concept Normalization using Large Language Models.
TL;DR
This study finds that existing MCN benchmarks present data quality issues and underexplored data fusion potential, and data quality enhancement and LLM-based controlled-variety data augmentation help alleviate overlapping phrases and long-tail issues.
Abstract
Objective
Medical Concept Normalization (MCN) maps informal health phrases to formal clinical concepts. It is a critical task for pharmacovigilance, patient record analysis, and health-related text mining. Extensive MCN research has mainly relied on single-source datasets and overlooked data quality (DQ) issues. This study aims to develop a quality-aware data fusion framework for MCN using Large Language Models (LLMs).
Methods
The methods consist of DQ evaluation and enhancement, LLM-based data augmentation, and multi-source MCN dataset fusion using enriched concept-phrase pairs. We analyze six widely used MCN datasets-AskAPatient, CADEC, COMETA, PsyTAR, TwADR-S, and TwiMed-each collected from social media and mapped to SNOMED-CT. Our evaluation metrics include correctness, concept validity, coverage, semantic variety, and class imbalance. For DQ enhancement and data augmentation, we use Gemini for zero-shot and few-shot learning to increase the semantic variety and phrase count for rare concepts. We then perform data fusion based on shared medical concepts.
Results
The DQ evaluation reveals substantial issues, including incorrect mappings, invalid concepts, low-variety redundant phrases, and long-tail concept-phrase distribution. After augmentation, phrase counts increase by 172.1%, and by 450.0% after fusion. To directly investigate model performance improvement on rare cases, we introduce concept-level macro metrics. SapBERT, KNN-BioEL, and KRISSBERT trained on the augmented and fused dataset achieve significant gains in accuracy, recall, precision, and F1 over single-source baselines, up to 38%.
Conclusion
Our study finds that existing MCN benchmarks present data quality issues and underexplored data fusion potential. Data quality enhancement and LLM-based controlled-variety data augmentation help alleviate overlapping phrases and long-tail issues. Moreover, quality-aware data fusion can expand conceptual coverage, improving MCN performance. This work highlights data quality evaluation and fusion strategies are effective in advancing MCN. We hope these contributions could support quality-guided large-scale MCN data generation with minimal label costs, strengthen reliable biomedical text mining and downstream applications. AVAILABILITY https://github.com/yhZHOU515/DataFusion4MCN.