A Proxy-Guided Multi-Objective Framework for Configuration Optimization in Merging Large Language Models

Abstract

Model merging aims to combine multiple task-specialized models into a single large language model (LLM). However, existing approaches lack effective strategies for optimizing high-dimensional merging configurations under multiple conflicting objectives and expensive evaluation budgets. In this work, we propose OptiMerge, a novel multi-objective evolutionary framework tailored for LLM model merging. OptiMerge optimizes merging configurations using a multi-objective formulation, where task accuracies serve as objectives and layer-wise merging parameters as decision variables. To improve evaluation efficiency, we propose a proxy-guided screening operator that integrates lightweight proxy signals into the selection process to eliminate low-quality candidates before expensive task evaluations. Integrated into the evolutionary cycle, this operator enables evaluation-efficient population evolution while preserving Pareto diversity. Experiments using Task Arithmetic and TIES-Merging on Mistral-7B models show that OptiMerge discovers well-structured Pareto fronts and achieves improved performance trade-offs compared to base models and grid-search baselines.