Author

Jonas Knupp

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#artificial intelligence Preprint Jan 2026

Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it Deserves

A modular framework of depth-recurrent attention mixtures (Dreamer), combining sequence attention, depth attention, and sparse expert attention is introduced, which alleviates the hidden-size bottleneck through attention along depth, decouples scaling dimensions, and allows depth-recurrent models to scale efficiently a...

Jonas Knupp, J. Metzen, Jeremias Bohn et al. · 15 citations

#artificial intelligence Preprint Sep 2026

Good Pretraining, Bad SFT: Checkpoint Quality Across the Training Stack

It is shown that this assumption can fail in a full 30B mixture-of-experts training pipeline, and the checkpoints that perform better after the full downstream training stack also have higher solution density, i.e., retain downstream performance under local weight perturbations.

Sohir Maskey, Philipp Scholl, Jonas Knupp et al. · 0 citations