2 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
This suite provides a new efficient lens on generalization: select intermediate pre-training checkpoints that strongly generalize reasoning and alignment, better than the final pre- or mid-training checkpoints, and select pre-training data that controls and stabilizes generalization dynamics.
A novel method is introduced that localizes the model's few-shot learning ability to only a few attention heads through dimensionality reduction and decomposition of the heads'output spaces, and derives a mathematical identity relating the''aggregator''and''extractor''subspaces of attention heads, allowing us to track...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
