The Token Before the Value Is the Key: How Hybrid Architectures Organize Induction Circuits
Hybrid language models can improve capability as well as efficiency, raising the question of how architectural complementarity becomes learned computation. We examine the established induction roles of Carrying predecessor information, Matching a source by content, and Copying its value. How are these position-sensitiv...
Ke Cheng, Xin Xu, Yi-Xiao Chen et al.
· 0 citations