Kenichi Maeda

Domain-Specific Reranking: When is Adaptation Worth the Cost?

It is found that when pretraining data is limited, task-specific fine-tuning—using only a few hundred labeled examples in the domain—can match or surpass the gains of domain-adaptive pretraining, suggesting that domain-adaptive pretraining may not always be necessary for strong domain performance.