PATE-TDPM: Differentially Private Tabular Data Synthesis on Long-Tailed Distribution

Abstract

While the tension between sharing valuable data and safeguarding individual privacy has grown increasingly acute, differentially private data synthesis has emerged as a principled solution. Although a series of research has been conducted in this field, the effectiveness on datasets with long-tail distribution attributes—a prevalent characteristic of real-world data where rare categories hold significant analytical value—remains largely unexplored. Empirical experiments show that existing data synthesis methods struggle to capture the long-tailed characteristics. To address this gap, we initiate a novel framework (PATE-TDPM) for differentially private data synthesis on long-tailed tabular datasets. The framework employs diffusion models to capture diverse data patterns, thereby mitigating the bias of dominant class during training Generative Adversarial Networks (GANs). To more effectively address the heterogeneity of tabular data, the framework integrates two different types of diffusion models. Meanwhile, PATE-TDPM incorporates the Private Aggregation of Teacher Ensembles (PATE) mechanism to protect privacy, thereby alleviating the negative impact of gradient clipping and noise injection in Differentially Private Stochastic Gradient Descent (DPSGD) on tail-class representations. The privacy property of PATE-TDPM is proved by theoretical analysis. Extensive evaluations across linear and non-linear tasks on real datasets demonstrate superior utility of PATE-TDPM in preserving long-tailed structures under rigorous DP guarantees.