Sep 2026· Engineering Research Express· Vol 8, pp. 205203· 0 citations· 28 references
Physics
TL;DR
Chaos post-processing and momentum-optimized differentially private stochastic gradient descent (CPSGD), which formally proves that each CPSGD step preserves the (ϵ, δ)-differential privacy (DP) guarantee of the underlying Gaussian mechanism.
Abstract
Differentially private stochastic gradient descent (DP-SGD) protects training data by adding calibrated Gaussian noise to clipped gradient updates; however, the resulting perturbations inevitably interfere with optimization and reduce classification accuracy. To address this limitation, this paper proposes chaotic post-processing and momentum-optimized differentially private stochastic gradient descent (CPSGD). CPSGD first applies the sampled Gaussian mechanism to the averaged clipped gradient and then adds a normalized Logistic-map sequence generated independently of the private dataset. Momentum is then applied to the post-processed private gradient. Because both the chaotic perturbation and the momentum update operate only on the output of the Gaussian mechanism, they constitute post-processing operations and introduce no additional privacy loss. We formally prove that each CPSGD step preserves the (ϵ, δ)-differential privacy (DP) guarantee of the underlying Gaussian mechanism, and we account for iterative privacy loss using Rényi DP. Experiments on the CIFAR-10, CIFAR-100, and Fashion-MNIST datasets show that the proposed method improves classification performance across different privacy budgets. Compared with DP-SGD, CPSGD improves classification accuracy by 5.14 percentage points on CIFAR-10 at ϵ = 4, 9.52 percentage points on CIFAR-100 at ϵ = 8, and 8.79 percentage points on Fashion-MNIST at ϵ = 1, indicating an improved empirical trade-off between privacy preservation and model utility.
Differentially private stochastic gradient descent (DP-SGD) enables neural network training with formal privacy guarantees, but its utility depends on model architecture, data modality, and privacy budget. DP-RVFL freezes random hidden features and privately trains an output layer with a direct input connection. We com...
Emad T. Elkabbash, Alshaimaa Abo-alian, Tamer Abdelkader et al.· Future Internet· 0 citations
Scaling laws and hyperparameter tuning remain central challenges in training large-scale models, particularly under differential privacy (DP), where optimization is further complicated by noise injection and per-sample gradient clipping. While Maximal Update Parametrization ($\mu$P) allows for zero-shot hyperparameter...
Tian-Ze Wang, Zhi-Qi Bu, Lin-Jun Zhang· 0 citations
In this article, we study deep nonparametric regression trained via stochastic gradient Langevin dynamics (SGLD), a variant of stochastic gradient descent that injects carefully calibrated Gaussian noise during optimization. Using a PAC-Bayesian analysis tailored to deep ReLU networks, we establish an excess-risk bound...
Adversarial sharpness penalty per iteration of a base optimizer, known as sharpness-aware minimization (SAM), is an accepted technique to improve generalization in deep learning. Recently, Muon has emerged as a dominant optimization algorithm for training large language models (LLMs). It achieves lower sample complexit...
Geetika, Shivani Singh Chauhan, Gauranshi Gupta et al.· 0 citations
Privacy-preserving machine learning presents significant deployment challenges on the cloud for intelligent systems with confidential data. Fully Homomorphic Encryption (FHE) offers a compelling solution for secure computation, preserving data confidentiality of cloud computations. However, applying FHE to reinforcemen...
Abid Mohamed Nadhir, A. Al Hanbali, Beggas Mounir· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.