ESSA (Evolutionary Strategies for Scalable Alignment), a gradient-free online alignment stage that follows supervised fine-tuning (SFT) and replaces the gradient loop with inference-only black-box search, is presented.
Daria Korotyshova, B. Shaposhnikov, A. Malakhov et al.· arXiv.org· 3 citations
A difficulty-aware scaling coefficient is proposed, inspired by Focal loss, that down-weights updates on high-success sampled groups and improves average math pass@256 in all three cases, without increasing group size or computational cost.
Daniil Plyusov, Alexey Gorbatovski, B. Shaposhnikov et al.· arXiv.org· 9 citations· ⚡1
Layer-Integrated Memory (LIMe) is introduced, a lightweight extension that leverages existing key-value buffers and learns per-head, per-layer routing weights to integrate representations from previous layers to improve perplexity per FLOP and yield strong gains on synthetic tasks while preserving higher value-vector e...