Jun 2026
DNA Language Models: An Assessment of Pre-Training for Fine-Tuning Tasks
This work investigates three key questions: do transformer-based models provide sufficient improvements on fine-tuning tasks upon heavy pretraining, what is the actual contribution of pretraining in this setting, and how does BPE tokenization impact performance on genomics-related tasks.
R. Karpinsky, J. Mozziconacci, Mickaël Delcey
· arXiv.org · 0 citations