Skip to content
Preprint

Efficient Exact Unlearning for Federated Zeroth-Order Optimization

Oct 2026 · 0 citations · 52 references
Computer Science

TL;DR

Two stable zeroth-order learning algorithms are designed and proved exact unlearning for both algorithms: the model and saved state have the same joint distribution as retraining with the corresponding learner on retained data.

Abstract

Federated zeroth-order optimization (FedZOO) reduces the memory and communication costs of collaborative training for edge intelligence by avoiding backpropagation while communicating only random seeds and scalar coefficients. However, existing methods lack efficient mechanisms for exact unlearning to fully remove the influence of training data when users request deletion. A key insight is that total variation stability of the learning algorithm creates overlap between model distributions before and after data deletion, allowing exact unlearning to reuse computation through coupling. Based on this observation, we jointly design two stable zeroth-order learning algorithms and their corresponding exact-unlearning procedures: \textsc{FedZOO-S} obtains stability through client and mini-batch subsampling, enabling \textsc{FedZOUL-S} to perform efficient unlearning by preserving earlier updates and restarting from the first affected step. \textsc{FedZOO-NS} further combines subsampling with Gaussian noise to increase overlap between update distributions; \textsc{FedZOUL-NS} exploits this overlap through rejection--reflection coupling to reuse affected updates when the coupling is accepted. We prove exact unlearning for both algorithms: the model and saved state have the same joint distribution as retraining with the corresponding learner on retained data. We further establish stability and convergence guarantees for the learning algorithms and derive upper bounds on expected unlearning cost. Experiments demonstrate effective unlearning with competitive model utility and up to $56.1\%$ less cumulative unlearning time than re-optimization. Compared with first-order baselines, our methods reduce communication overhead by up to approximately four orders of magnitude.

View source

Similar papers

Oct 2026

MMT: Achieving Exact Federated Unlearning with Improved Post-unlearning Performance

Two exact federated unlearning algorithms are proposed, Bi-Models Training (bmt) and Multi-Models Training (mmt), that ensure immediate exact unlearning by design while achieving better performance post-unlearning.

Ze-Yu Zhang, N. Bui, A. Verma et al. · 1 citation
#federated learning Open access Oct 2026

FeDLRT: Federated Dynamical Low-Rank Training

Abstract. We propose a federated dynamical low-rank training (FeDLRT) scheme to reduce client compute and communication costs—two significant performance bottlenecks in horizontal federated learning. Our method builds upon dynamical low-rank time integration schemes for manifold-constrained optimization to create a gl...

Steffen Schotthöfer, M. P. Laiu · 0 citations
Preprint Oct 2026

Split Federated Unlearning

This work provides the first systematic formulation of Split Federated Unlearning (SFU), encompassing client-wise and class-wise removal, counterfactual retraining, and partition-specific update constraints, and theoretically characterize historical influence propagation and mode-dependent unlearning feasibility.

Jia Chen, Ye-Cheng Lv, Xin-Rong Hu et al. · 0 citations
Open access 2026

Fed-ZOE: Communication-Efficient Over-the-Air Federated Learning via Zeroth-Order Estimation

As 6G and beyond networks grow increasingly complex and interconnected, federated learning (FL) emerges as an indispensable paradigm for securely and efficiently leveraging decentralized edge data for AI. By virtue of the superposition property of communication signals, over-the-air FL (OtA-FL) achieves constant commun...

Jonggyu Jang, Hyeonsu Lyu, D. Love et al. · 4 citations
Review Open access Nov 2026

FedHST: Federated Hybrid Spatial–Temporal Optimization for robust non-IID learning

Server-side adaptive optimizers in Federated Learning (FL), such as FedAdam and FedAvgM, scale global updates using historical temporal statistics but do not explicitly model instantaneous cross-client variance within a single communication round. Under severe statistical heterogeneity (non-IID data), this can leave th...

Duy-Dong Le, Thai-Thinh Dang · 0 citations
Preprint Oct 2026

FedFit: Federated Dynamic Sparse Training via Fisher Information scoring

Cross-device Federated Learning (FL) is frequently bottlenecked by the prohibitive memory and communication costs of training deep neural networks on resource-constrained edge hardware. While federated dynamic sparse training aims to alleviate these costs by adjusting sparse structures during training, existing methods...

Meng Bi, Hong Huang, Jin-Long Song et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.