This survey identifies critical research gaps, including interpretable routing, expert communication, modality integration, modality integration, and lifelong multimodal learning, and positions this survey as a foundation for future research toward interpretable and sustainable multimodal Mixture-of-Experts system.
L. Zheng, Wenhao Liang, W. Zhang et al.· 0 citations
This work audits reading for routing entropy in Attention-Residual variants of Swin-Tiny and DeiT-Small, trained from scratch on CIFAR-10/100 with a soft-binned calibration auxiliary loss, establishing control-dependent gains and incomplete estimator recovery, not the absence of conditional routing information.
Wenhao Liang, Lin Yue, W. Zhang et al.· 0 citations
When a classifier is recalibrated from only a few thousand held-out examples, the capacity of the calibration map becomes a statistical design choice rather than a purely architectural one: a scalar map can underfit structured residual miscalibration, while a highly adaptive map can be hard to estimate reliably from so...
Wenhao Liang, L. Zheng, Lin Yue et al.· arXiv.org· 0 citations
Routing signals of modern vision transformers -- expert gates, attention-residual weights and halting scores -- often improve probes that predict whether the model is correct, and the improvement is commonly read as evidence that routing carries information about errors beyond the model's outputs. We test this inferenc...
Wenhao Liang, Lin Yue, W. Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.