DA-MoE: descriptor-attention mixture-of-experts for multi-class gastrointestinal disease classification

Abstract

Computer-aided diagnosis (CADx) for gastrointestinal (GI) endoscopy increasingly depends on deep models trained end-to-end on raw images. However, raw images are often unavailable in legacy clinical systems or privacy-sensitive settings. This work presents a methodological study on multi-class GI disease classification from pre-extracted handcrafted descriptors on the public Kvasir benchmark, rather than a clinically validated deployment system. We propose a Descriptor-Attention Mixture-of-Experts (DA-MoE) model tailored to this descriptor-only scenario. DA-MoE first projects heterogeneous descriptors (JCD, Tamura, ColorLayout, EdgeHistogram, AutoColorCorrelogram, and PHOG) into a shared token space and applies transformer-style self-attention for descriptor-level fusion. A descriptor-aware mixture-of-experts classifier then performs sample-adaptive expert routing on the fused representation. Under a unified evaluation protocol using accuracy (ACC), macro F1, and Matthews correlation coefficient (MCC), with five-fold stratified cross-validation and repeated random splits for robustness assessment, DA-MoE achieves 77.4% accuracy and an MCC of 0.75 on the held-out test split, outperforming strong feature-based baselines including a residual multi-layer perceptron (Res-MLP). Ablation studies, hyperparameter analysis, MoE routing interpretability, full per-class metrics, and clustering analysis further show that DA-MoE produces more compact and better-separated representations in descriptor embedding space. These findings support DA-MoE as a practical component of feature-based GI CADx pipelines under data-sharing constraints, but prospective validation on raw endoscopic data remains necessary.