Model merging aims to build a multi-task model cheaply by combining the weights of individual task-specific models. To perform well across multiple tasks, most existing merging methods use an additional dataset to find the coefficients for the best linear combination of task-specific weight updates. However, we identif...
Sin-Han Yang, Shih-Cheng Huang, Chieh-Yen Lin et al.· 0 citations
Model merging offers a promising solution for combining multiple fine-tuned checkpoints into a single model through parameter arithmetic. However, finding optimal merging coefficients requires an extensive search that becomes prohibitively expensive as models scale in both size and number, due to high memory requiremen...
Shih-Cheng Huang, Zhi Rui Tam, Chieh-Yen Lin et al.· 0 citations
Experiments show that the proposed Preference Vector framework improves helpfulness without excessive conservatism, allows smooth control over preference trade-offs, and supports scalable multi-preference alignment.