Sungyeop Jung

Quantized Model Soup Shake-Up: Weight Perturbation for Enhanced Ensemble Diversity

Model soup, averaging the weights of multiple fine-tuned models, delivers ensemble-level accuracy at single-model inference cost, but its success requires both linear mode connectivity (LMC) and sufficient diversity among candidates. We study these two requirements under quantization-aware training (QAT). First, we sho...