Manas Mehta

Randomized YaRN Improves Length Generalization for Long-Context Reasoning

Randomized YaRN is proposed, a training method that improves length generalization by combining YaRN-based positional extrapolation with randomized positional encoding and a length curriculum, and suggests that progressively exposing models to OOD positional distributions provides an effective recipe for generalizable...