Skip to content

Calibrated Knowledge Aggregation in Bayesian Mixture-of-Experts for Continual VQA

· 1 citation · 70 references

TL;DR

A calibrated Bayesian mixture-of-experts is proposed that trains parameter-efficient per-task adapters, learns routing by directly maximizing expected VQA utility, and marginalizes expert identity at inference via Bayesian aggregation in a unified answer space; an entropy penalty prevents the utility objective from collapsing to one-hot routing.

View source

Similar papers

Preprint Aug 2026

A Heterogeneous Mixture of Experts Framework for Interpretable Machine Learning

The proposed approach combines interpretability, adaptive inductive bias selection, and probabilistic coherence within a unified mixture-of-experts framework to achieve interpretable expert assignments while achieving predictive performance competitive with homogeneous MoDT and Random Forests.

Soham Chatterjee, Rwitobroto Dey, Smarajit Bose · 0 citations
#artificial intelligence Preprint Sep 2026

Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models

It is found that perplexity and downstream accuracy favor different MoE experts, cautioning against selecting MoE compression settings using unlabeled text alone, and showing that expert identity matters substantially more than expert weighting, while balanced and domain-specialized routing leaves limited room for expe...

Xing Chen, Heng-Shuai Yao · 0 citations

RACER: Role-Aligned Competence Estimation for Human-AI Routing

Learning to defer asks a predictive system when to act autonomously and when to defer to a human expert. Population-adaptive deferral extends this problem to unseen experts using a small context set of expert behavior. Neural context encoders such as L2D-Pop can be query-dependent, but may learn routing shortcuts tied...

Joshua Strong, Emma Sun, A. Capstick et al. · 0 citations
#artificial intelligence Open access Sep 2026

Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts

Layer-wise Distribution Alignment (LDA), a lightweight inference-time correction that uses layer-wise calibration statistics to align reduced-routing representations with the default configuration, is proposed, recovering much of the performance lost induced by the distributional shift under reduced routing while prese...

Dohyeon Kim, Bedionita Soro, Sung Ju Hwang · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.