HyperTransfer: Understanding the Equivalence between Base Optimizer and Hyperball
Hyperball optimizers constrain parameter norms and update only their directions, establishing a distinct paradigm for neural network optimization. Although this geometry appears fundamentally different from that of conventional Base Optimizers, which update both parameter norms and directions, we show that the two para...
Jing-Hui Yuan, Hong-Tao Zhang, Jade Zou et al.
· 0 citations