A Dual-Level Selective Unlearning Framework for Multi-Agent Decentralized Federated Learning

Abstract

In agentic AI-enabled edge computing, decentralized federated learning (DFL) leverages peer-to-peer model aggregation to improve the performance of on-device large language models (LLMs) without introducing a single point of failure, thereby enhancing local agents’ capabilities for decision-making. To protect the right to be forgotten for each agent, as required by data regulations such as the General Data Protection Regulation (GDPR), federated unlearning aims to remove the influence of a target agent’s data from the trained LLM while preserving model utility. However, existing federated unlearning methods predominantly assume centralized architectures and face two critical challenges when extended to decentralized federated learning systems: 1) requiring all remaining agents to participate in the unlearning process incurs prohibitive overhead; and 2) updating all Low-Rank Adaptation (LoRA) modules indiscriminately leads to excessive resource consumption while potentially degrading model performance. To tackle these challenges, we propose a dual-level selective unlearning framework (DSU) for decentralized federated learning. At the agent level, DSU selects retained agents by matching historical LoRA update sketches, label sketches, and exposure to the withdrawn agent. Meanwhile, we derive a sensitivity score from the DFL training trajectory that measures each LoRA module’s accumulated influence from the target agent via the energy of its effective weight changes, and then update only the most sensitive modules while freezing the rest. Theoretical analysis relates selective unlearning to full-participation unlearning, and experiments show that DSU reduces the audited target-agent influence while preserving retained utility and reducing both participating agents and updated LoRA modules in the evaluated settings. The public reproducibility package is available at https://github.com/DGL-codes/LLM-DFL