A calibrated Bayesian mixture-of-experts is proposed that trains parameter-efficient per-task adapters, learns routing by directly maximizing expected VQA utility, and marginalizes expert identity at inference via Bayesian aggregation in a unified answer space; an entropy penalty prevents the utility objective from collapsing to one-hot routing.
The proposed approach combines interpretability, adaptive inductive bias selection, and probabilistic coherence within a unified mixture-of-experts framework to achieve interpretable expert assignments while achieving predictive performance competitive with homogeneous MoDT and Random Forests.
It is found that perplexity and downstream accuracy favor different MoE experts, cautioning against selecting MoE compression settings using unlabeled text alone, and showing that expert identity matters substantially more than expert weighting, while balanced and domain-specialized routing leaves limited room for expe...
Bayesian Pairwise Merging (BPM) enables personalization from pairwise feedback without per-user policy training, while characterizing the coefficient ambiguity left by limited feedback.
Ya-Ling Shen, Tong-Tong Wu, Si-Yuan Yan et al.· arXiv.org· 0 citations
Learning to defer asks a predictive system when to act autonomously and when to defer to a human expert. Population-adaptive deferral extends this problem to unseen experts using a small context set of expert behavior. Neural context encoders such as L2D-Pop can be query-dependent, but may learn routing shortcuts tied...
Joshua Strong, Emma Sun, A. Capstick et al.· arXiv.org· 0 citations
Layer-wise Distribution Alignment (LDA), a lightweight inference-time correction that uses layer-wise calibration statistics to align reduced-routing representations with the default configuration, is proposed, recovering much of the performance lost induced by the distributional shift under reduced routing while prese...
Dohyeon Kim, Bedionita Soro, Sung Ju Hwang· Conference on Empirical Meth...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.