Faster Learning under Relaxed Local Differential Privacy

Abstract

We consider density estimation under the relaxed local differential privacy condition that the privatized distributions are $\alpha$-close in total variation distance. We show that adding independent noise with a convenient symmetrized Gamma distribution to each sensitive observation attains the $\alpha$-TV-LDP. We prove that the deconvolution estimator of $r$-Sobolev smooth functions attains the pointwise rate $(n\alpha)^{-\frac{2r-1}{2r}}$ up to log factors which is faster than $(n\alpha^2)^{-\frac{2r-1}{2r+1}}$ under the classical $\alpha$-LDP and closer to the nonprivate minimax rate $n^{-\frac{2r-1}{2r}}$. Next, we use a Goldenshluger-Lepski procedure to build a free of the smoothness adaptive procedure and show optimality of our rates in the convolution model of our privatisation scheme. We illustrate the benefits of this simple privacy mechanism by implementing a neural network estimator which does not need to add more noise in the optimization steps. Numerical results show significant improvement of the estimation rate over the Laplace and the private-SGD mechanisms.