Uncertainty-Aware Trust Estimation for Multi-LLM Systems via Structured Expert Judgement
This work adapt structured expert judgment from decision theory, using context-aware calibration questions to estimate expert reliability based on the quality of its probabilistic predictions, and employs Cooke-style log weighting, which penalises overconfident incorrect predictions and favours well-calibrated experts.