Skip to content

Privacy Preserving Gossip Learning

Sep 2026 · 0 citations · 23 references
Computer Science Mathematics

TL;DR

This work builds on Tuning without Forgetting (TwF) method to preserve previously learned mappings and shows that TwF provides an indistinguishability guarantee for the learner whenever the set of protected agents contains another sample with the same label.

Abstract

We propose a decentralized privacy-preserving learning algorithm in which each agent holds a single private sample and a shared model. Samples are learned sequentially, and each update must preserve the endpoint mappings at previously learned samples while protecting private data. This gives each agent three roles: (i) a learner that updates the model parameters, (ii) a teacher whose sample is learned at the current iteration, and (iii) a protected agent whose sample has already been learned. We build on Tuning without Forgetting (TwF) method to preserve previously learned mappings and show that TwF provides an indistinguishability guarantee for the learner whenever the set of protected agents contains another sample with the same label. For the teacher, we formulate a minimax optimal control problem that models the differential privacy noise as a worst-case disturbance to prevent performance loss while maintaining the same level of privacy for the gradient. For the protected agents, we compute the projections locally and aggregate them using a private push-sum gossip protocol. We prove geometric convergence of the decentralized gossip algorithm and of the distributed projection for TwF.

View source

Similar papers

#artificial intelligence Preprint Aug 2026

Performative Privacy: When Differential Privacy Maximizes Utility

It is shown, through a theoretical study of the dynamics and numerical experiments, that a finite privacy budget can outperform non-private estimation in the long term when the feedback loop between leakage and participation is sufficiently strong.

Uddalak Mukherjee, Edwige Cyffers, Y. Chevaleyre · 0 citations
Conference Open access Sep 2026

Privacy-Preserving Split Learning via Noise Adaptive Representation Training

Noise-Adaptive Training (NAT), a client-side protocol that prevents the server from accessing clean smashed data, is proposed and shows that NAT consistently reduces property inference risk while maintaining high task accuracy and introducing negligible client-side overhead.

Nexhi Sula, Nafiza Tabassoum, Jie Hou et al. · 0 citations
Preprint Oct 2026

Soft Voting for Policy-Aware Private Data Synthesis

Blowfish privacy relaxes differential privacy (DP) by protecting only the attribute-value substitutions a data owner specifies as edges of a policy graph. A sparser policy can reduce the noise required by a mechanism, but only when the released statistic changes less across protected substitutions than across arbitrary...

Ying-Ge Hu, G. R. Babu, Mostafa Milani · 0 citations
Open access Sep 2026

Distortion-Constrained Minimax Data Sanitization with Multiple Adversaries

We study a privacy-preserving data-sharing setting where a privatizer transforms private data into a sanitized release observed by an authorized reconstructor and two unauthorized adversaries, each with access to side information correlated with the private data. An adversary is considered stronger when it estimates th...

Amirarsalan Moatazedian, Yauhen Yakimenka, Re-Mi A. Chou et al. · 0 citations
Preprint Oct 2026

NOMAI: Scalable Federated Learning with Private Weights

The twin goals of privacy-preserving federated learning (FL) -- harnessing distributed data while protecting the parties who contribute it -- demands three properties: model privacy, data privacy, and integrity. Achieving all three in a realistic deployment -- with a single coordinator and millions of users -- remains...

Yi-Yi Sun, Mushtari Sadia, Amrita Roy Chowdhury · 0 citations
Preprint Aug 2026

Privacy Without Regret: Differentially Private Inference-Time Alignment

Private Inference-Time Pessimism (PrivITP) is introduced, which combines $\chi^2$-regularized rejection sampling with a two-phase Gaussian mechanism, and achieves ex-post $(\epsilon,\delta)$-DP with a privacy cost independent of the number of responses, cleanly decouples the regularization parameter from the privacy pa...

I. Jain, Nandini Bhattad, Sayak Ray Chowdhury · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.