Skip to content

Contribution-Aware Bandwidth Allocation for Multimodal Split Learning

Sep 2026 · 0 citations · 33 references
Computer Science

TL;DR

The allocator, ModalShare, sets each modality's keep-ratio from a Shapley contribution score that the server computes over coalitions of activations it has already received, and shows that existing compressors underperform in multimodal settings, with ModalShare recovering what gains are left behind.

Abstract

Multimodal models are increasingly the default option for perception at the network edge, yet they are trained almost entirely in the datacenter, because a client holding several sensor streams cannot host an encoder per modality. Split Learning makes such training feasible by keeping only the first layers on the device, at the cost of an uplink that must carry smashed activations for every modality at every step. Existing compression schemes give each modality the same keep-ratio, so the shared budget is divided in proportion to smashed-activation dimension, a quantity unrelated to how much each modality contributes to the fused prediction. We make that division an explicit decision and call it inter-modality allocation: under a fixed uplink budget, every policy transmits the same expected payload and differs only in how that payload is split across modalities. Our allocator, ModalShare, sets each modality's keep-ratio from a Shapley contribution score that the server computes over coalitions of activations it has already received. Measuring this score adds no uplink traffic and no client-side computation, and needs no prior knowledge of which stream is which. ModalShare improves accuracy over equal keep-ratios by 15.4 and 12.4 percentage points on CREMA-D and MVSA at matched payload in 5x compression, with strong performance across three compressors, three datasets, and four budgets. We show that existing compressors underperform in multimodal settings, with ModalShare recovering what gains are left behind.

View source

Similar papers

#machine learning Open access Sep 2026

Importance-Aware Feature Sparsification for Wireless Split Learning

This work proposes importance-aware class-balanced sparsification (ICS), a lightweight approach in which the server ranks feature channels using Grad-CAM-based scores obtained from the true-class logit during backpropagation, and extends ICS to parallel split learning and to transformer-based models.

Bumjun Kim, Yoon Huh, Wan Choi · 0 citations
#machine learning Preprint Sep 2026

Latency-Aware Client Assignment for Parallel Split Learning With Global Sampling

Latency Budgeted Parallel Split Learning with Global Sampling introduces Latency Budgeted Parallel Split Learning with Global Sampling, which separates each pooled batch's integer class target from the choice of clients that supply its examples.

Mohammad Kohankhaki, Valentin Rentschler, A. Schmeink · 0 citations
Preprint Aug 2026

Meta-Learning Where to Allocate Experts: Task-Conditioned Layer-Wise Compression for MoEs

MetaNet is proposed, a support-set controller that predicts, for each layer, an expert-retention threshold and a bounded routing bias and provides a tunable accuracy-expert-activation trade-off on DeepSeek-MoE-16B-Chat.

Rong-Feng Wang, Shitao Weng, Zhiquan Wang et al. · 0 citations
#edge computing Preprint Sep 2026

Iapetus: Content-Aware Hierarchical Scheduling for Collaborative ViT Inference in LEO Satellite Networks

sys is presented, a content-aware hierarchical scheduler that screens constellation-wide options to retain a bounded candidate set, then refines each candidate into a complete token compression and layer offloading trajectory using quality prediction and joint planning and balances per-task latency, energy, and quality...

Yan Chen, Yun-Xiang Zhang, Guan-Jun Jiang et al. · 1 citation
#machine learning Preprint Aug 2026

SplitLite: Low-Rank Residual Compression for Split Learning

SplitLite is proposed, a communication-efficient split federated LoRA fine-tuning method that exploits the low effective rank structure of consecutive-epoch activation and gradient residuals, thereby significantly reducing both activation uplink and gradient downlink traffic.

Tao Li, Yu-Lin Tang, Qi Guo et al. · 0 citations
#machine learning Preprint Sep 2026

UniCache: Task- and Type-Aware KV Cache Compression for Unified Multimodal Models

Unified multimodal models combine understanding, generation, and editing within a single network, offering a promising foundation for versatile multimodal applications. However, growing multimodal contexts make KV cache storage and access increasingly costly. Existing KV cache compression methods are typically tailored...

Wan-Qi Yang, Yue-Xiao Ma, Mei Xie et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.