Jacobian-Guided Anisotropic Noise Reshaping for Enhancing Representation Utility under Local Differential Privacy
TL;DR
This work proposes a novel approach that mitigates noise in task-relevant subspaces of the data representation by identifying task-critical subspaces via the Jacobian of a public downstream model, selectively attenuates noise along these directions, and reshapes the isotropic noise of standard LDP mechanisms into an anisotropic distribution.
Abstract
While Local Differential Privacy (LDP) serves as a foundational primitive for distributed data collection, its stringent randomization requirements often lead to severe degradation in data representation utility. This degradation stems from the task-agnostic nature of conventional LDP mechanisms, which perturb all dimensions without accounting for their relative importance to the downstream objective. To address this issue, we propose a novel approach that mitigates noise in task-relevant subspaces of the data representation. Our method identifies task-critical subspaces via the Jacobian of a public downstream model, selectively attenuates noise along these directions, and reshapes the isotropic noise of standard LDP mechanisms into an anisotropic distribution. The resulting mechanism preserves the privacy guarantee of the underlying LDP randomizer while heterogeneously modulating the impact of noise across task directions, thereby substantially enhancing data utility. The approach is applicable to both linear and nonlinear models and can be seamlessly integrated with existing LDP mechanisms. Extensive experiments on CIFAR-10-C under brightness corruption at the highest severity level demonstrate that integrating our approach improves classification accuracy by approximately 8 percentage points for Laplace and 20 percentage points for PrivUnit variants at $\epsilon=7.5$. The source code is available at https://github.com/ymha/jacobian-anr-ldp.