Controlling Action-Head Co-Adaptation to VLM Representation Anisotropy
AdaIE, a parameter-free mechanism that reweights pretrained representations according to the evolving downstream sensitivity during training, is introduced, highlighting interface co-adaptation as an important optimization problem when coupling pretrained representations with downstream models.
Rui-Jie Zhang, Jia-Yin Zheng, Ye-Quan Zhao et al.
· 0 citations