Hui-Jun Xing

TEMPER: Tensorized Efficient Manifold-constrained Parameterization for Expressive Residual Routing

Residual connections rely on a static residual pathway, and are essential for training deep neural networks. Hyper-connections (HC) increase the expressivity of residual routing by incorporating multiple residual streams and learning dynamic information flow, while manifold-constrained (mHC) variants stabilize training...