Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning
We study sparse and continual reinforcement learning fine-tuning of language models through attention-head ranking and partitioning. For sparse learning, we rank attention heads by gradient norm to identify a small subset of heads that can be trained effectively. For continual learning, we partition heads across tasks...
Rui-Zhe Shi, Zai-Wen Yang, Qiu-Bo Wang et al.
· 0 citations