Skip to content

Author

Mao Zheng

We have 15 of 32 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Oct 2026

Rethinking LLM Pruning via Test-Time Scaling

Structured pruning reduces the size of large language models, often at the cost of lower accuracy. Whether this decline reflects diminished reasoning capacity or greater difficulty in sampling correct solutions is less clear. We examine this question across expert, FFN and layer pruning on four mathematical and coding...

Ming-Yang Song, Mao Zheng · 1 citation
#natural language process... Preprint Oct 2026

Understanding Jev-Style System One Models: Does Structure Reduce Selection Bias?

Selection bias in large language models (LLMs) can persist despite instructions to ignore presentation order or prompts that elicit reasoning. Recent efforts to reproduce Jev-style System One behavior have produced a growing family of models that predict probabilities directly over supplied candidates. Yet systematic c...

Ming-Yang Song, Mao Zheng, Zhi-Yuan Ma et al. · 0 citations
#natural language process... Preprint Oct 2026

Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing

External-memory systems extend language-model applications beyond a context window, but their quality depends critically on the model executing the write path: deciding what to retain, how precisely to represent it, and how new evidence should update persistent state. Existing frameworks typically place a general-purpo...

Tian-Xiang Fei, Jia-Zheng Kang, Mao Zheng et al. · 1 citation · ⚡1
Preprint Sep 2026

Data-Free On-Policy Distillation: How Far Can We Go Without External Data?

On-policy distillation (OPD) is increasingly applied to frontier foundation model post-training. Prior work in this area has largely focused on algorithmic advances, yet it remains unclear how much OPD depends on its training questions and, in particular, how far this dependence can be reduced. Across two representativ...

Gengsheng Li, Mao Zheng, Ming-Yang Song et al. · 0 citations
#natural language process... Preprint Oct 2026

ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery

Structured pruning reduces the deployment cost of reasoning language models, but the resulting capability degradation can hinder subsequent on-policy distillation (OPD) recovery. Because OPD relies on student-generated trajectories, pruning damage that persists after offline distillation can limit its effectiveness. We...

Hou-Cheng Jiang, Mao Zheng, Ming-Yang Song et al. · 2 citations

MAS-OPD: On-Policy Distillation for Multi-agent Systems

MAS-OPD is presented, where Role-Advantage Specialization defines the role advantage as the difference between the teacher signals under target and non-target role conditions, and Privileged Attribution for Coordination attributes an interaction conflict to its source and supplies it to the teacher alone as privileged...

Qi-Yong Zhong, Mao Zheng, Ming-Yang Song et al. · 0 citations

RAZOR: Pruning Replaceable Experts in LLMs

RAZOR, a training-free pruning method based on consensus residuals, the deviations of expert outputs from their original weighted mixture, gives the exact output change for a single deletion under survivor renormalization and router refill.

Ming-Yang Song, Mao Zheng · 1 citation

On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents

Guided On-Policy Distillation (Guided-OPD), a simple yet effective algorithm that mixes teacher- and student-generated turns within each rollout and schedules the teacher's intervention probability along a curriculum that decays to zero is proposed.

Gengsheng Li, Mao Zheng, Ming-Yang Song et al. · 24 citations · ⚡3
#machine learning Preprint Sep 2026

Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation

TrustMOPD replaces example-level teacher selection with label-free, token-level supervision allocation and closes the overall-score gap between the initial student and oracle-routed teachers when trained on single- and multi-teacher on-policy distillation.

Jie Sun, Mao Zheng, Ming-Yang Song et al. · 6 citations
#artificial intelligence Preprint Sep 2026

Data-free On-policy Distillation

DF-OPD matches and even surpasses real data, and the questions it produces track the teacher's own post-training data on three key diagnostics of training dynamics, which other real datasets do not.

Gengsheng Li, Mao Zheng, Ming-Yang Song et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.