ESSA (Evolutionary Strategies for Scalable Alignment), a gradient-free online alignment stage that follows supervised fine-tuning (SFT) and replaces the gradient loop with inference-only black-box search, is presented.
Daria Korotyshova, B. Shaposhnikov, A. Malakhov et al.· arXiv.org· 3 citations
A difficulty-aware scaling coefficient is proposed, inspired by Focal loss, that down-weights updates on high-success sampled groups and improves average math pass@256 in all three cases, without increasing group size or computational cost.
Daniil Plyusov, Alexey Gorbatovski, B. Shaposhnikov et al.· arXiv.org· 9 citations· ⚡1
KronSAE is proposed, a design that factorizes the latent space into heads and forms post-latent features as pairwise compositions of lower-dimensional pre-latents using mAND, a differentiable AND-like interaction that imposes a compositional co-activation prior while remaining compatible with standard SAE objectives an...
Vadim Kurochkin, Yaroslav Aksenov, Daniil Laptev et al.· 1 citation
Layer-Integrated Memory (LIMe) is introduced, a lightweight extension that leverages existing key-value buffers and learns per-head, per-layer routing weights to integrate representations from previous layers to improve perplexity per FLOP and yield strong gains on synthetic tasks while preserving higher value-vector e...
This work introduces Soft Latent Thinking, a method that replaces the LM head during reasoning with a lightweight projector, enabling autoregressive rollout in embedding space where reasoning steps remain continuous rather than tokenized.
N. Koriagin, Yaroslav Aksenov, George Bredis et al.· 0 citations
Qantara is presented, an end-to-end JEPA whose joint training objective pairs a Brownian-bridge interpolant between consecutive clean latents on the state axis with noise-to-data flow matching on the action axis, and these results move JEPA world models from single-paradigm planners to multi-paradigm controllers.
Ruslan Rakhimov, George Bredis, Yuriy Maksyuta et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.