PFN-Syn achieves the highest average downstream utility among evaluated synthetic generators, with higher generation throughput than training-based baselines, while maintaining competitive distributional similarity and distance-to-closest-record privacy behavior.
Synthetic tabular data is increasingly used to support data sharing and machine learning development, where access to real data is restricted or sensitive. However, existing tabular data generators often require dataset-specific training or fine-tuning, data preprocessing, and extensive hyperparameter tuning to achieve plausible results. In addition, existing models may still struggle with mixed data types, modeling complex feature relationships, and pose a privacy risk for information leakage. With the rise of Large Language Models (LLMs) and their expanding capabilities across many domains, we investigate whether few-shot LLMs can be used in a training-free setting as tabular data generators. We propose FewShotTabLLM, a few-shot, training-free generation framework for tabular data that leverages schema-enriched context, representative sampling, and dataset profiling. We evaluate our approach across five widely used tabular benchmark datasets and compare it against strong training-based baselines from different model families (CTGAN, TVAE, TabDDPM, BeGReaT). A three-axis evaluation protocol has been used that covers representativeness, realism, and privacy. Across all evaluations, FewShotTabLLM outperforms baselines and surpasses them on many datasets, achieving realism, boundary adherence, and privacy preservation across all experiments. Overall, the results suggest that training-free fewshot LLM synthesis can serve as an alternative to existing models and a prominent research direction for improving dependency preservation and privacy-aware generation. Our code is available at: https://github.com/BMW-InnovationLab/FewShotTabLLM
Hadi Koubeissy, Michel El Khoury, Marc Kamradt et al.· Annual International Compute...· 0 citations
Synthetic data, i.e., data simulated from some statistical model, are an important tool for both privacy protection and artificial intelligence pipelines. In the privacy context, synthetic data enable agencies to disseminate record-level information while reducing disclosure risks. In the artificial intelligence context, synthetic data allow analysts to augment training sets, increase coverage of rare cases, and support experimentation when genuine data are scarce. For each usage, we discuss key considerations and methods for generating synthetic data, including sequential modeling, differentially private synthesis, deep generative models, and large language models. Throughout, we highlight key trade-offs between data usefulness, privacy protection, and model reliability. We conclude by outlining some open research challenges and future directions for synthetic data development.
Yinyihong Liu, Jerome P. Reiter· Annual Review of Statistics...· 0 citations
Differential privacy (DP) imposes fundamental trade-offs between privacy and statistical fidelity in synthetic data generation. While access to public data has been shown to improve these trade-offs empirically, existing approaches use public data only indirectly, through pre-processing (e.g., using pre-trained generative models) or post-processing steps (e.g., matching target statistics estimated from public datasets), while relying on domain-agnostic DP mechanisms. In this work, we lay the theoretical framework to study the principled incorporation of public data into DP mechanisms themselves. We consider normalized histograms as distribution estimators and characterize the asymptotically optimal domain-aware privacy mechanism within a specific class of DP mechanisms. We introduce PubMix, a public-data-aware DP mechanism that can be used in histogram-based data synthesis pipelines. Our experiments demonstrate that PubMix significantly improves synthetic data generation quality compared to domain-agnostic privacy mechanisms.
Sajani Vithana, Sangwon Jung, Haoyang Hu et al.· 0 citations
Machine learning pipelines are increasingly deployed in high-stakes domains where robustness against data inconsistencies, bias, and distributional shifts is crucial. However, real-world datasets often suffer from limitations such as data scarcity, class imbalance, and privacy constraints. Synthetic data generation has emerged as a promising strategy to overcome these challenges and enhance pipeline robustness. This paper presents a comprehensive survey and analysis of synthetic data generation techniques, classifying them by data modality, generation method, and application purpose. We explore how synthetic data contributes to the resilience of ML pipelines against failure modes such as concept drift, noise, and adversarial attacks. Through empirical case studies across domains, we demonstrate the practical benefits and limitations of integrating synthetic data into training and evaluation pipelines. Finally, we discuss ethical considerations and outline future research directions toward building more robust, fair, and privacy-preserving ML systems using synthetic data.
Dennis M. Ritchie, Allen Newell· International Journal of Dat...· 0 citations
Learning to rank (LTR) traditionally requires large-scale training data to generalize effectively. In low-data domains where expert annotation is scarce, the performance of LTR methods degrades sharply. Foundation models have alleviated similar data dependencies in other domains via in-context learning, but a foundation model for ranking with tabular features has not been explored yet. We propose prior-data fitted networks (PFNs) as a strong method for ranking in low-data settings. First, we demonstrate that PFNs, which are originally trained for classification, successfully outperform classification baselines on ranking data. Next, we evaluate PFNs as rankers, showing that they surpass state-of-the-art tuned baselines in low-data regimes. We introduce a novel sampling and inference scheme to obtain pairwise predictions from PFNs' native pointwise architecture, analogous to pairwise LTR. To address the limited context window of the transformers underlying PFNs, we propose a dynamic support set selection strategy for queries that scales PFNs beyond random subsampling. Our experimental results show that PFNs are an effective foundation model for ranking that provides significant gains when data is limited.
David Vos, Samarth Bhargav, M. D. Rijke et al.· International Conference on...· 0 citations
Federated learning (FL) offers a promising approach to privacy-preserving clinical risk prediction, but its deployment remains limited by restricted data sharing, client heterogeneity, class imbalance, and the lack of realistic tabular electronic health record (EHR) benchmarks. Synthetic data generation may alleviate data scarcity, yet its integration with federated optimisation has received limited systematic study. We propose SynPre-FL, a unified framework combining high-fidelity synthetic EHR generation with synthetic-pretrained FL for robust prediction under non-IID conditions. A latent autoencoder-diffusion model generates privacy-preserving synthetic cohorts, which are used to warm-start federated training. This pretraining is followed by heterogeneity-aware optimisation using class-balanced local objectives, proximal regularisation, and adaptive server aggregation. Post-hoc calibration and federated-safe explainability support reliable and interpretable risk estimates. Experiments show that the synthetic generator preserves univariate, bivariate, and multivariate structure while protecting against membership-inference and reconstruction attacks. The generated data achieve strong downstream utility under TSTR, TRTS, and model-based evaluations. Across federated settings with 5, 10, and 15 heterogeneous clients, SynPre-FL consistently improves robustness and scalability over baseline methods, especially under severe non-IID fragmentation. Calibration improves probability reliability, while SHAP analysis produces stable and clinically coherent feature attributions across federation sizes. SynPre-FL therefore provides a practical and reproducible framework for combining synthetic data with FL to enable privacy-aware, interpretable, and robust clinical prediction from distributed tabular EHR data.
Akarsh K. Nair, Muhammad Arifur Rahman, N. Shopland et al.· 0 citations