Skip to content

Author

Ziyang Cai

We have 3 of 13 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#artificial intelligence Preprint Nov 2026

Perturbing Inputs Improves Pretraining Distillation

Knowledge distillation trains a student language model to match a teacher's predictions on the training text. We ask whether learning the teacher's behavior beyond the training text can further improve the student. We investigate this using three input perturbations: additive token noise, blending two documents, and bl...

Zi-Yang Cai, Yi-He Dong, Dimitris Papailiopoulos et al. · 0 citations
Preprint Aug 2026

Full-bandwidth transformer

This work trains 1B-parameter full-bandwidth transformers on up to 400B tokens and finds that latent feedback improves validation loss, 5-shot language-model evaluation, math and coding generation, and instruction-tuned performance.

Xi Wang, Zi-Yang Cai, Zheng Zhan et al. · 6 citations
Jul 2026

T^2MLR: Transformer with Temporal Middle-Layer Recurrence

Transformers with Temporal Middle-Layer Recurrence (T2MLR), a transformers-based latent reasoning architecture that fuses a cached middle layer representation from the previous token directly into an earlier layer of the current token position, enabling abstract intermediate computation to persist across decoding steps...

Ziyang Cai, Xingyu Zhu, Yihe Dong et al. · 6 citations · ⚡1

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.