We have 2 of 67 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
TRACK: TRajectory-Aware Capacity routing via top-K selection is presented, a heterogeneous denoising strategy that switches between compatible large and small models at selected steps, reducing the average cost per denoising evaluation.
This work adapt and enhance preconditioned gradient methods to overcome the practical challenges of large-scale LLM pretraining and proposes algorithmic modifications including per-step QR orthogonalization and improved preconditioning strategies to enable stable training in these regimes.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
