This paper investigates the effectiveness of specializing ultra-compact language models for clinical embedding generation, utilizing the EmbeddingGemma 300M as the base model, and demonstrates the feasibility of performing fine-tuning on entry-level hardware.
Findings demonstrate that compact models can achieve strong biomedical classification performance through KD under compatible teacher–student pairings, while also highlighting that KD effectiveness varies substantially depending on the specific model combination.
It is suggested that domain-adapted encoder models may be preferable for similar structured clinical NER settings, although larger and externally validated benchmarks are needed before generalizing to other languages, clinical corpora, model families, or deployment environments.
L. Elvas, Carolina Carvalho· Scientific Reports· 0 citations
The findings indicate that domain-specialized models improve in-domain retrieval relative to generic models, and that systematic optimization through the multi-stage pipeline yields measurable gains in retrieval precision.
Savaş Yıldırım, Mucahit Cevik, Ayşe Başar· SN Computer Science· 0 citations
This paper presents pre-trained models (PTMs) for the automatic extraction of relations from biomedical text, specifically targeting the variant-phenotype domain and demonstrates that fine-tuning small BERT-based models, particularly DeBERTa, yields strong performance, approaching the current state-of-the-art (SOTA).
Claudiu Creanga, L. Dinu, Daniela Gîfu· 0 citations
The proposed framework offers a practical and scalable approach to mitigating hallucinations without requiring task-specific fine-tuning, highlighting the potential of retrieval-augmented approaches for trustworthy artificial intelligence (AI)-assisted healthcare applications.
Findings show that model size alone is an insufficient selection criterion for OL and provide empirical guidance for reproducible LLM-assisted ontology engineering and indicate that architecture and model lineage can outweigh nominal parameter count.
Hamed Babaei Giglou, S. Auer, Jennifer D'Souza· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
