Author

Cheng Luo

h-index 2 View on Semantic Scholar

We have 1 of 6 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Jul 2026

Multi-Head Attention Residuals

Multi-Head Attention Residuals (MHAR) is introduced: the routing query is reshaped into H per-subspace heads, each with its own softmax over the depth history, and a direct probe of the trained queries confirms that learned subspace disagreement is the underlying driver.

Cheng Luo, Zefan Cai, Jun-Jie Hu · arXiv.org · 1 citation