Climate-ModernBERT: Revisiting Corpus Composition for Domain-Adaptive Continued Pretraining
Climate-ModernBERT is introduced, a family of climate-adapted encoder models obtained through continued pretraining of ModernBERT-Base on three climate corpora: academic climate text, climate-filtered web data, and synthetic climate documents.
Yongan Yu, S. Raj, Jingwei Ni et al.
· 0 citations