Skip to content
#machine learning Open access

Importance-Aware Feature Sparsification for Wireless Split Learning

Sep 2026 · IEEE Journal on Selected Areas in Communications · Vol 44, pp. 5888-5904 · 0 citations · 40 references
Computer Science Engineering

TL;DR

This work proposes importance-aware class-balanced sparsification (ICS), a lightweight approach in which the server ranks feature channels using Grad-CAM-based scores obtained from the true-class logit during backpropagation, and extends ICS to parallel split learning and to transformer-based models.

Abstract

Wireless split learning (SL) reduces on-device computation by offloading upper layers to a server, yet transmitting high-dimensional intermediate features at each iteration remains a major communication bottleneck. Existing methods select features at the client side using task-agnostic criteria such as magnitude, statistics, or clustering, which increases client-side processing and often degrades accuracy under non-independent and identically distributed (non-i.i.d.) client data. We propose importance-aware class-balanced sparsification (ICS), a lightweight approach in which the server ranks feature channels using Grad-CAM-based scores obtained from the true-class logit during backpropagation. The per-class scores are aggregated into a class-balanced, label-agnostic importance vector that mitigates head-class bias under label skew, and each client reuses this vector in the next round to retain the top- $N$ feature channels, incurring no additional client-side forward or backward passes. We further derive a non-asymptotic convergence bound that isolates the sparsification-induced error and characterizes how the sparsification ratio and mini-batch size jointly affect convergence under a fixed communication budget, and we analyze the communication and computational overhead of ICS against representative baselines. Beyond sequential CNN-based SL, we extend ICS to parallel split learning and to transformer-based models. Experiments show that ICS consistently outperforms the baselines, with larger gains under severe non-i.i.d. partitions.

Read PDF

Similar papers

Preprint Oct 2026

HintKD: Hint Knowledge Distillation for Bandwidth-Constrained Cloud-Edge Inference

Collaborative inference between the edge cloud and user equipment (UE) is a promising paradigm for deploying large deep neural networks (DNNs) in 6G networks. However, existing cloud-edge inference and distillation schemes often require the real-time transmission of high-dimensional intermediate features or soft probab...

Wan-Ling Luo, Jia-Yi Zhang, Bo-Kai Xu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Scaling of Wireless Foundation Models via Representation Diversity and Multi-Branch Architectures

Wireless foundation models learn representations from unlabeled radio signals for reuse across downstream tasks. Scaling model capacity is a common strategy for learning richer representations and improving downstream performance. However, its gains are less consistent in wireless self-supervised learning when pretrain...

Ahmed P. Mohamed, Ahmed Aboulfotouh, Hatem Abou-Zeid · 0 citations
Open access 2026

CoSCF: Gradient-Aware Surrogate Optimization for Task-Oriented Collaborative Sensing

Multi-view WiFi sensing enhances robustness over single-link sensing, but transmitting all intermediate features incurs prohibitive communication overhead. Existing compression methods lack mechanisms to quantify per-view task relevance, making uniform compression inefficient for task-oriented sensing. Therefore, we fo...

Yi-Yun Zhang, Gong-Pu Wang, Fei-Fei Gao et al. · 0 citations
#machine learning Preprint Oct 2026

RIPPLE in Still Water: Zero-Shot Clustering in Federated Learning with Wavelet Scattering Transform

Clustered Federated Learning (FL) partitions a client population into groups of similar local distributions and trains one specialized model per cluster, mitigating client drift that degrades single-model methods under non-IID data. Prior methods discover cluster structure inside the training loop through gradient simi...

Alessandro Licciardi · 0 citations
Open access Sep 2026

Radar Signal Sorting via a Dual-Layer Multi-Head Attention Broad Learning System

Radar signal sorting is a fundamental task in passive electromagnetic sensing that separates pulses emitted by multiple radars from densely interleaved streams. Existing supervised deep learning methods often require large labeled datasets and iterative optimization, limiting their applicability in non-cooperative, lat...

Long-Kun Kuang, Xuan Chen, Ming-Yang Du et al. · 0 citations
#machine learning Preprint Sep 2026

Latency-Aware Client Assignment for Parallel Split Learning With Global Sampling

Latency Budgeted Parallel Split Learning with Global Sampling introduces Latency Budgeted Parallel Split Learning with Global Sampling, which separates each pooled batch's integer class target from the choice of clients that supply its examples.

Mohammad Kohankhaki, Valentin Rentschler, A. Schmeink · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.