Some attention heads learn similar patterns across inputs. Reusing these patterns could reduce training cost by avoiding repeated query-key score computation and softmax. Through controlled pretraining comparisons, we identify Selective Attention Freezing (SAF), which selects heads with low attention-pattern variance a...
Weixian Waylon Li, Yin-Tao Tai, Marcio Fonseca et al.· 0 citations
This work proposes SAVLA, an end-to-end symmetry-aware VLA model for robust and data-efficient policy learning, which keeps the pretrained vision-language backbone entirely frozen while combining it with an equivariant flow-matching action head and a learned canonicalizer.
FinCAD is proposed, an inference-time adaptation of Context-Aware Decoding that attenuates contributions from memorised historical outcomes without retraining and raises the subset-averaged in-sample/out-of-sample Spearman correlation on an eleven-model leaderboard.