This work compares three synthesis paradigms, statistical, adversarial, adversarial, and diffusion-based, on two benchmarks and provides task-driven guidance for selecting a synthesizer in learning analytics.
Abstract
Privacy regulations and institutional policies limit the sharing of educational data, constraining reproducibility in learning analytics. Prior evaluations of synthetic data on benchmarks such as OULAD have examined statistical or adversarial synthesizers in isolation, rarely jointly assessing fidelity, utility, privacy, and explainability. We compare three synthesis paradigms, statistical (Gaussian Copula), adversarial (CTGAN), and diffusion-based (TabDDPM), on two benchmarks (OULAD: 32,593 records; ASSISTments: 8,519) across five evaluation axes: distributional fidelity (SDMetrics), downstream utility (Train on Synthetic, Test on Real), discriminative realism (classifier two-sample test), membership-inference privacy, and feature-importance preservation (SHAP). The pipeline is repeated over five random seeds with bootstrap confidence intervals and Bonferroni-corrected permutation tests (<inline-formula> <tex-math notation="LaTeX">$\alpha \prime \approx ~0.0028$ </tex-math></inline-formula>). Four findings emerge: First, TabDDPM delivers the strongest classification utility: on OULAD, a Random Forest achieves TSTR AUC <inline-formula> <tex-math notation="LaTeX">$= 0.962~\pm ~0.001$ </tex-math></inline-formula>, within 0.5 percentage points of the real-data baseline. Second, all synthesizers exhibit near-chance membership-inference risk under the evaluated kNN-based threat model (worst-case effective AUC <inline-formula> <tex-math notation="LaTeX">$\le 0.527$ </tex-math></inline-formula>). Third, distributional fidelity does not predict task utility; CTGAN scores highest on SDMetrics yet does not yield the smallest utility gap. Fourth, TabDDPM best preserves real-data feature-importance rankings on OULAD (Spearman <inline-formula> <tex-math notation="LaTeX">$\rho =0.846$ </tex-math></inline-formula>, p < 0.001). ASSISTments SHAP correlations (<inline-formula> <tex-math notation="LaTeX">$\rho ~ \ge 0.950$ </tex-math></inline-formula>) reflect a low-dimensionality ceiling effect rather than meaningful synthesis quality differences. These results provide task-driven guidance for selecting a synthesizer in learning analytics. Scope is limited to static tabular benchmarks; temporal, sequential, multimodal, and fairness-aware synthesis remain outside the present scope. Runtime results are based on CPU execution, so neural synthesizers may run faster under GPU acceleration.
This survey highlights how optimization- and certification-oriented reasoning can provide a useful framework for reasoning about differences in models, supporting tasks ranging from model training and selection to auditing and certification.
This paper studies embedding-space privacy as a representation-level learning problem. Rather than altering raw records directly, the proposed framework applies embeddingspace transformation to full-record representations through Gaussian perturbation and adversarial representation sanitization. The method is evaluated through ablation across utility metrics, linkage attacks, attribute-inference attacks, and membership-inference tests. The primary empirical evaluation uses a synthetic fusion recommendation benchmark built from MovieLens [1], [2] 32M behavior and Adult-derived demographics [3], while a secondary synthetic medical benchmark is used to examine cross-domain transferability under more constrained conditions. The strongest results appear in the recommendation experiments. Under grouped demographic privacy evaluation, the combined condition preserves recommendation utility with $N D C G {@} K=0.6312$ while reducing exact and entity linkage from 0.7090/0.7204 to 0.0001/0.0000. Sensitive-target attacker performance remains near the majority baseline, supporting the claim of empirical privacy improvement without visible ranking degradation in that benchmark. The healthcare experiments also demonstrate meaningful embedding transformation and linkage reduction, though the current benchmark remains datalimited and therefore less conclusive for utility-focused evaluation. Overall, the findings support the conclusion that embeddingspace transformation can preserve downstream utility while substantially reducing linkage risk and sensitive-information recoverability under explicit attacker evaluation. The findings support embedding-space transformation as a practical privacypreserving strategy for embedding-driven AI systems under explicit attacker evaluation.
D. Panagoulias, Evangelia-Aikaterini Tsichrintzi, E. Sakkopoulos· International Conference on...· 0 citations
High-quality data are essential for machine learning and data-driven research, yet data scarcity and privacy concerns remain major obstacles in many domains. Generative models have recently emerged as a promising approach to synthesize data that follow the same statistical distribution as real datasets. However, generative models are vulnerable to membership inference attacks, which threaten data confidentiality by exploiting model outputs to infer whether specific samples were used in training. Existing defense strategies struggle to simultaneously preserve data utility and provide robust privacy protection. To address this challenge, we propose our PPGM-GAN, a Privacy-Preserving GAN for synthetic data against membership inference attack to balance both data utility and data privacy. PPGM-GAN balances privacy and utility through a privacy-utility tradeoff function that quantifies and optimizes both aspects under different adversarial knowledge. To enhance data utility, we incorporate conditional generation and key-attribute screening to ensure sufficient representation of infrequent attribute values. Additionally, differential privacy is employed during training to prevent overfitting and reduce privacy leakage. Experimental results demonstrate that PPGM-GAN outperforms state-of-the-art privacy-preserving generative models, producing high-utility synthetic data under the same privacy constraints.
Guizhang Cui, Guowei Wu, Lin Yao et al.· ACM Transactions on Privacy...· 0 citations
Machine-learning-as-a-service (MLaaS) platforms let organizations deploy predictive models without revealing their internal architecture, but this opacity does not guarantee the privacy of the data used to train them. This paper presents a systematic framework for membership inference attacks, in which an adversary with only black-box query access to a deployed classifier determines whether a specific individual's record was part of its training set. We introduce a shadow-model methodology that trains auxiliary classifiers on synthetically generated data produced via model-based synthesis, known population statistics, or noisy real data to learn the behavioral differences a target model exhibits on seen versus unseen inputs. Evaluated against commercial platforms including Google Prediction API and Amazon ML across seven datasets spanning healthcare, retail, location, and image domains, our attacks achieve median precision of 0.657 and 0.678 on the two platforms, respectively, with healthcare records showing vulnerability of approximately 65.7% precision. We identify overfitting, output granularity, and class imbalance as primary drivers of leakage, and evaluate mitigations such as output truncation, temperature scaling, and regularization, finding that each offers only partial protection at the cost of utility. These findings expose significant gaps in current regulatory frameworks such as GDPR and HIPAA regarding inference-based privacy violations.
Pramod Prakash· International Journal of Int...· 0 citations
Generative Artificial Intelligence (GenAI) has transformed data generation through advanced models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Diffusion Models, and Large Language Models (LLMs), enabling the creation of synthetic datasets that closely resemble real-world data while addressing challenges related to privacy, accessibility, and regulatory compliance. As synthetic data becomes increasingly adopted across healthcare, education, finance, public administration, and social science research, ensuring its quality, reliability, fairness, and trustworthiness has emerged as a critical research priority. This literature review examines recent developments in synthetic data quality evaluation between 2020 and 2026, focusing on key dimensions including utility, fidelity, privacy preservation, fairness, diversity, robustness, interpretability, and governance. The review traces the evolution of evaluation methodologies from traditional statistical similarity measures toward multidimensional assessment frameworks such as SynEval, SynthEval, Benchmarking Synthetic Tabular Data Framework, SynAE, and ESDAE. A structured comparison of these frameworks is presented using criteria including analytical accuracy, scalability, privacy protection, fairness assessment, and interpretability. The review further explores emerging approaches for explainable synthetic data assessment, fairness-aware synthetic data generation, and Privacy-Enhancing Technologies (PETs), including differential privacy, federated learning, secure multi-party computation, and privacy-preserving generative models. In addition, real-world case studies from healthcare, education, public policy, and social science research are examined to demonstrate how synthetic data quality evaluation directly influences decision-making, research validity, and policy outcomes. The findings indicate that while modern generative models can produce highly realistic and analytically useful datasets, persistent challenges remain, including the lack of standardized benchmarking protocols, utility–privacy trade-offs, privacy leakage risks, bias amplification, limited explainability, and governance concerns. The review concludes that future research should prioritize internationally accepted evaluation standards, explainable and fairness-aware assessment frameworks, stronger privacy-preserving mechanisms, and comprehensive governance models to support the responsible, transparent, and trustworthy deployment of synthetic data in the Generative AI era.
N. Emran, Ruhaila Maskat, Abdulrazzak Ali· International journal of res...· 0 citations
: As small-scale, open-source Large Language Models (LLMs) proliferate for on-device and privacy-centric applications, understanding the trade-offs between their utility and behavioural reliability becomes critical. This study evaluates a suite of instruction-tuned LLMs, Gemma, Llama, and Qwen ( ≤4 B parameters), treating the model family, and scale as the primary units of analysis. Retrieval-Augmented Generation (RAG) is employed as a controlled experimental condition to assess utility gains on the Natural Questions (NQ) benchmark, while utilizing native, non-augmented configurations to establish a fairness baseline via the Bias Benchmark for QA (BBQ). The findings reveal that while RAG significantly enhances utility, often doubling Exact Match (EM) scores, these gains are non-uniform and architecture-dependent, with certain families exhibiting greater "retrieval-readiness" than others. Paradoxically, the fairness analysis shows that providing explicit context in disambiguated settings can increase stereotype engagement rather than suppressing it. These results suggest a fundamental disconnect between a model's capacity for factual accuracy and its ability to maintain social fairness, highlighting the need for multi-dimensional evaluation frameworks for small-scale systems.
M.J.F. Valdez, Arghir-Nicolae Moldovan· Proceedings of the 15th Inte...· 0 citations