CPSGD: Differentially Private Stochastic Gradient Descent with Chaotic Post-Processing and Momentum Optimization

Abstract

Differentially private stochastic gradient descent (DP-SGD) protects training data by adding calibrated Gaussian noise to clipped gradient updates; however, the resulting perturbations inevitably interfere with optimization and reduce classification accuracy. To address this limitation, this paper proposes chaotic post-processing and momentum-optimized differentially private stochastic gradient descent (CPSGD). CPSGD first applies the sampled Gaussian mechanism to the averaged clipped gradient and then adds a normalized Logistic-map sequence generated independently of the private dataset. Momentum is then applied to the post-processed private gradient. Because both the chaotic perturbation and the momentum update operate only on the output of the Gaussian mechanism, they constitute post-processing operations and introduce no additional privacy loss. We formally prove that each CPSGD step preserves the (ε, δ)-differential privacy guarantee of the underlying Gaussian mechanism, and we account for iterative privacy loss using Rényi differential privacy. Experiments on the CIFAR-10, CIFAR-100, and Fashion-MNIST datasets show that the proposed method improves classification performance across different privacy budgets. Compared with DP-SGD, CPSGD improves classification accuracy by 5.14 percentage points on CIFAR-10 at ε = 4, 9.52 percentage points on CIFAR-100 at ε = 8, and 8.79 percentage points on Fashion-MNIST at ε = 1, indicating an improved empirical trade-off between privacy preservation and model utility.