Skip to content

Revisiting the Provable-Auditable Privacy Gap of DP-SGD

Aug 2026 · 0 citations
Computer Science

TL;DR

A lightweight defense framework that generically augments optimization methods in the ML pipeline to have significantly-improved empirical privacy on standard benchmarks is given, and it is shown that the framework comes at no theoretical privacy cost when augmenting DP-SGD, unlike previously-proposed defenses against membership inference attacks.

Abstract

Differential privacy (DP) has traditionally been used to provide theoretical upper bounds on an algorithm's stability to changing its training data. In modern private machine learning applications, achieving strong tradeoffs between utility and theoretical privacy is challenging, and thus one may optimistically hope that existing theoretical privacy analyses are loose. Recent work on privacy auditing has adopted a dual viewpoint, instead lower bounding the true privacy of an algorithm by constructing empirical distinguishing events. The auditing literature has thus far yielded a pessimistic outlook on the looseness of theoretical privacy bounds for DP-SGD, the de facto private training method in modern ML, as nearly-matching empirical lower bounds have been achieved under various threat models [NHSBTJCT23, AC24, CBP25]. In this work, we propose the empirical privacy lower bound of an algorithm as a concrete metric to optimize for, complementary to the theoretical upper bound. We give a lightweight defense framework that generically augments optimization methods in the ML pipeline to have significantly-improved empirical privacy on standard benchmarks. Moreover, we show that our framework comes at no theoretical privacy cost when augmenting DP-SGD, unlike previously-proposed defenses against membership inference attacks. We evaluate our defense against a broad range of audit constructions, models, and datasets to demonstrate its flexibility.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Canaries in the Bank: Auditing User-Level Privacy in Private Evolution

A protocol-aware empirical audit is introduced in which the server commits to a single shared candidate bank and replaces roughly 1% of its entries with probes derived from a known, non-private canary, to quantify the gap between formal worst-case privacy and leakage achievable through protocol-valid candidate-bank man...

Sai Aparna Aketi, Enayat Ullah, Shripad Gade · 0 citations

Certification-Based Differentially Private Learning

This work generalizes the abstract gradient training (AGT) framework and introduces Abstract Gradient Sampling (AGS), an algorithm that enables reachability-based analysis to provide guarantees for private learning and finds that under matched conditions, the private learning algorithm can outperform standard private l...

Mihnea Ghitu, Matthew Wicker · 1 citation
2026

PI-SAFE: Practical Privacy-Preserving LLM Inference With Adversarial Fine-Tuning for Optimized Utility

Cloud-based Large Language Model (LLM) inference services typically require users to submit plain-text inputs, thereby posing severe privacy risks. Existing privacy-preserving paradigms are mostly task-specific and often necessitate pervasive modifications to the entire server-side model. This reliance introduces subst...

Wentao Zhong, Yu-Ting Li, Di-Cong Yu et al. · 0 citations
Review Oct 2026

Unifying Privacy Accounting: Information Equivalence and Information Loss

This paper considers four mainstream curve-based privacy notions within a unified information-theoretic framework, establishing information equivalence among the two directional privacy profiles of $(\varepsilon,\delta)$-DP, the pair of hypothesis-testing trade-off functions, and the extended privacy-loss distribution.

Buxin Su, Qiao-Shi Yang, Yi-Ding Su et al. · 0 citations
Preprint Aug 2026

Revisiting Continuous Noise Sampling for Multi-Party Differential Privacy

This paper revisits the continuous noise sampling protocols and makes several improvements in both security and efficiency and turns to discrete sampling at the granularity of individual biased bits to address the security and efficiency issues together.

Yu-Cheng Fu, Tianhao Wang · 1 citation
Preprint Aug 2026

Spectrum-Aware Bounds on Invertibility for Privacy-Enhancing Instance Encoding

A family of new bounds that are tighter, applicable even to fully deterministic encoders, and applicable even to fully deterministic encoders can extend beyond MSE to other norm-based similarity metrics, by properly accounting for the encoder's spectral structure.

Seokjin Hwang, Yu-Ting Li, Kiwan Maeng · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.