We have 4 of 11 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
SCALE (Selective Control of Adaptation via Local Entropy), an entropy-guided adaptation-strength-control method that freezes the pretrained model and the SFT delta and learns bounded token- and module-specific gates by minimizing predictive entropy alone is proposed.
This paper introduces a lightweight probe for multi-step gradients of pretrained weights that incurs no additional GPU memory cost and only marginal time overhead, and employs a spectrum-aware, importance-based rank allocation and optimal initialization derived from multi-step gradients.
This paper makes the first attempt to fine-tune MoE models with MoE-style low-rank adaptation via a dual-channel Routing-Conditioned Projection module, which reuses base router activations to inform LoRA routing and introduces a single global LoRA expert pool shared across all layers.
This work introduces a student reference KL loss and mask the advantages of special termination tokens to mitigate the problem of excessive generation length and frequent truncation, and improves on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
