Representation-Aware Modularity: Efficient Cross-Task Generalization for LLMs
RaMod, a Representation-Aware Modularity framework, is proposed, a Representation-Aware Modularity framework to extend the ReFT paradigm to CTG through two novel components: Dual-Modular Representation & Parameter Fine-tuning, which manipulates only a strategically chosen subset of hidden representations with modular interventions to guide the model toward solving unseen tasks.
Zheng Gong, Ying Sun, Chao Wang et al.
· 0 citations