We have 6 of 21 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
It is argued that numerical format design is itself a first-class lever for stable FP4 training, and the first systematic study of FP4 LLM pretraining on energy-efficient Huawei Ascend NPUs is presented.
DigenRL is introduced, a disaggregated RL framework for diffusion-based generative LLMs that supports flexible resource allocation, accommodates heterogeneous GPUs, and facilitates efficient task scheduling and an elastic trainer-assisted generation approach to enable the trainer GPU resources to dynamically assist in...
Autonomous agents powered by large language models (LLMs) continuously accumulate experience through interaction, creating an opportunity to improve future behavior through self-evolution. A fundamental challenge is how to transform abundant, task-specific interaction experience into reusable model competence without s...
HiF4 is established as the enabling format for end-to-end FP4 RL post-training, and Rollout Residual Quantization (Rollout-ResQ) is established as the activation-side mechanism that makes the gap to BF16 closable.
This work proposes a low-precision training framework based on 2D block FP4 quantization, which enforces transposition-invariant scaling and preserves consistency between forward and backward computations, and combines this with truncation-free scaling and stochastic rounding to control quantization error and maintain...
This work proposes Residual Fallback Quantization (RFQ), a lightweight activation reconstruction framework that supplements the primary ulta-low-bit activation representation with an auxiliary quantized residual pathway that improves activation fidelity while preserving the efficiency advantages of ultra-low-bit comput...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
