Weighted Proportional Conflict Redistribution for Heterogeneous LLM Ensembles

Abstract

Smaller language models often exhibit high confidence in erroneous predictions, undermining traditional ensemble methods. In heterogeneous ensembles, simple averaging can grant disproportionate influence to overconfident but incorrect outputs. This paper introduces a reliability-weighted extension of the Proportional Conflict Redistribution rule (PCR6), where model-specific weights are derived from validation accuracy. We provide a formal treatment of the conflict measure, prove that the weighted rule avoids Zadeh's paradox in the high-conflict regime, and establish a quasi-associativity bound of order O(K_max²). We further justify linear weighting from both Bayesian (conjugate Dirichlet) and minimax perspectives, and compare the approach with logarithmic pooling and Dirichlet calibration. Empirical evaluation on a multi-model MMLU setting shows that the primary practical benefit of weighted PCR6 in our experiments lies in improved calibration (substantially lower Expected Calibration Error) rather than large accuracy gains over simple averaging. Accuracy improvements appear most plausible when ensemble members differ markedly in reliability; when models are comparably strong, the method remains competitive while producing better-calibrated probabilities. The fusion step itself is computationally lightweight (tens of microseconds per query for small ensembles).