Transformers and BiLSTM-Based Ensemble Modeling for Entity and Relation Aware Biomedical Extractive Question Answering

Abstract

Extractive Question Answering (EQA) in the biomedical domain presents significant challenges due to complex medical terminology and intricate relationships between entities. Existing systems often perform suboptimally compared to their counterparts in general domains. Current biomedical QA models struggle with accurately identifying named entities and extracting meaningful relationships within medical contexts, thereby limiting their practical utility. This study introduces a novel transformer-based approach for biomedical EQA that incorporates Named Entity Recognition (NER) to identify key medical terms, Relation Extraction (RE) to understand their interconnections, and a BiLSTM layer to enhance contextual comprehension. These components are integrated within an ensemble framework that leverages pretrained biomedical language models. Our methodology systematically evaluated six transformer encoder models and identified Bio+ClinicalBERT as the optimal foundation for developing three progressively advanced biomedical EQA models. The final model—Bio+ClinicalBERT_NER_RE_BiLSTM—incorporating ensemble learning techniques, demonstrated substantial performance improvements: an F1 score of 91.69%, exact match of 88.35%, and lenient accuracy of 0.84, marking a 5.45% performance gain over baseline methods. These results highlight the model's superior ability to navigate complex biomedical language and extract accurate information from medical texts. The developed system directly benefits biomedical researchers, clinical practitioners, medical students, and healthcare professionals by delivering accurate and rapid access to precise information within extensive medical literature. By bridging the gap between intricate medical texts and effective information retrieval, this work significantly advances automated biomedical knowledge extraction and supports evidence-based medical decision-making.