Modular Expert Merging for Biomedical Retrieval
Synthesize-Train-Merge (STM) is introduced, a modular framework that synthesizes hard negatives with a top-tier LLM and fine-tunes domain-specialized experts via LoRA before merging them, without continual pre-training, to maximize expert quality.
S. Khattab, Jean-Philippe Corbeil, Osman Alperen Çinar-Koras et al.
· 1 citation