This work constructs a family with quadratic arithmetic in the matrix dimension when the physical block size remains fixed, and derive finite-shape constraints for GPU execution, and treats these results as a feasibility and trainability check for the proposed approach at small scale, leaving further investigation to f...
Zarya is introduced, a family of hybrid language models that jointly optimizes an autoregressive (AR) objective and a masked-diffusion objective within a single architecture, demonstrating performance on standard benchmarks while offering a principled integration of autoregressive and diffusion paradigms.
Leonid S. Sinev, Ilya Koziev, Vladislav Leshchuk· 0 citations
A gap between algorithmic losslessness and its implementation under finite-precision arithmetic is demonstrated and motivated, to motivate evaluating lossless speculative decoding at the level of exact generation trajectories as well as downstream task performance.
Ilya Koziev, Leonid S. Sinev, I. Oseledets· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.