Dan-Qi Chen

Where RL Breaks: Localizing and Repairing Failures via Targeted Continued Pretraining

The use of the failure $n$-grams to retrieve text from the pretraining corpus and continue pretraining the base model on it before RL suggests that where RL fails is a cheap and effective signal for selecting pretraining data.