Skip to content

Author

Song-De Rao

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#machine learning Preprint Sep 2026

SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining

Across autoregressive language-model pretraining, SOLAR improves final perplexity over tuned static schedules and automatic LR tuners for dense models from 60M to 1B, AdamW and Muon, and two MoE settings up to 3B.

Qiu-Lin Shang, Bin-Yu Wang, Yong-Qi Qiao et al. · 0 citations
#artificial intelligence Preprint Aug 2026

On-policy Distillation with Verifiable Reward

This work reformulates the implicit reward of sampled-token OPD based on trajectory correctness, then applies a ReLU gating mechanism to ensure that correct trajectories receive non-negative rewards and incorrect ones receive non-positive rewards, making it readily combinable with any policy gradient algorithm, such as...

Wen-Ze Lin, Jia-Le Zhao, Xi-Tai Jiang et al. · 14 citations · ⚡4

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.