Skip to content
Open access

CPSGD: differentially private stochastic gradient descent with chaotic post-processing and momentum optimization

Sep 2026 · Engineering Research Express · Vol 8, pp. 205203 · 0 citations · 28 references
Physics

TL;DR

Chaos post-processing and momentum-optimized differentially private stochastic gradient descent (CPSGD), which formally proves that each CPSGD step preserves the (ϵ, δ)-differential privacy (DP) guarantee of the underlying Gaussian mechanism.

Abstract

Differentially private stochastic gradient descent (DP-SGD) protects training data by adding calibrated Gaussian noise to clipped gradient updates; however, the resulting perturbations inevitably interfere with optimization and reduce classification accuracy. To address this limitation, this paper proposes chaotic post-processing and momentum-optimized differentially private stochastic gradient descent (CPSGD). CPSGD first applies the sampled Gaussian mechanism to the averaged clipped gradient and then adds a normalized Logistic-map sequence generated independently of the private dataset. Momentum is then applied to the post-processed private gradient. Because both the chaotic perturbation and the momentum update operate only on the output of the Gaussian mechanism, they constitute post-processing operations and introduce no additional privacy loss. We formally prove that each CPSGD step preserves the (ϵ, δ)-differential privacy (DP) guarantee of the underlying Gaussian mechanism, and we account for iterative privacy loss using Rényi DP. Experiments on the CIFAR-10, CIFAR-100, and Fashion-MNIST datasets show that the proposed method improves classification performance across different privacy budgets. Compared with DP-SGD, CPSGD improves classification accuracy by 5.14 percentage points on CIFAR-10 at ϵ = 4, 9.52 percentage points on CIFAR-100 at ϵ = 8, and 8.79 percentage points on Fashion-MNIST at ϵ = 1, indicating an improved empirical trade-off between privacy preservation and model utility.

Read PDF

Similar papers

Open access Oct 2026

DP-RVFL: A Differentially Private Randomized-Feature Neural Model for Privacy–Utility Efficient Learning

Differentially private stochastic gradient descent (DP-SGD) enables neural network training with formal privacy guarantees, but its utility depends on model architecture, data modality, and privacy budget. DP-RVFL freezes random hidden features and privately trains an output layer with a direct input connection. We com...

Emad T. Elkabbash, Alshaimaa Abo-alian, Tamer Abdelkader et al. · 0 citations
#machine learning Preprint Oct 2026

Towards Hyperparameter Transfer for Differentially Private Optimization

Scaling laws and hyperparameter tuning remain central challenges in training large-scale models, particularly under differential privacy (DP), where optimization is further complicated by noise injection and per-sample gradient clipping. While Maximal Update Parametrization ($\mu$P) allows for zero-shot hyperparameter...

Tian-Ze Wang, Zhi-Qi Bu, Lin-Jun Zhang · 0 citations
#machine learning Preprint Nov 2026

Statistical and Privacy Guarantees for Deep M-Regression via Stochastic Gradient Langevin Dynamics

In this article, we study deep nonparametric regression trained via stochastic gradient Langevin dynamics (SGLD), a variant of stochastic gradient descent that injects carefully calibrated Gaussian noise during optimization. Using a PAC-Bayesian analysis tailored to deep ReLU networks, we establish an excess-risk bound...

Ye-Heng Ge, Jian Huang, Shu-Yuan Wu · 0 citations
#machine learning Preprint Oct 2026

Sharpness-aware Orthogonalized Momentum Algorithms

Adversarial sharpness penalty per iteration of a base optimizer, known as sharpness-aware minimization (SAM), is an accepted technique to improve generalization in deep learning. Recently, Muon has emerged as a dominant optimization algorithm for training large language models (LLMs). It achieves lower sample complexit...

Geetika, Shivani Singh Chauhan, Gauranshi Gupta et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints

Privacy-preserving machine learning presents significant deployment challenges on the cloud for intelligent systems with confidential data. Fully Homomorphic Encryption (FHE) offers a compelling solution for secure computation, preserving data confidentiality of cloud computations. However, applying FHE to reinforcemen...

Abid Mohamed Nadhir, A. Al Hanbali, Beggas Mounir · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.