Few-Shot LLMs as Synthetic Tabular Data Generators

Abstract

Synthetic tabular data is increasingly used to support data sharing and machine learning development, where access to real data is restricted or sensitive. However, existing tabular data generators often require dataset-specific training or fine-tuning, data preprocessing, and extensive hyperparameter tuning to achieve plausible results. In addition, existing models may still struggle with mixed data types, modeling complex feature relationships, and pose a privacy risk for information leakage. With the rise of Large Language Models (LLMs) and their expanding capabilities across many domains, we investigate whether few-shot LLMs can be used in a training-free setting as tabular data generators. We propose FewShotTabLLM, a few-shot, training-free generation framework for tabular data that leverages schema-enriched context, representative sampling, and dataset profiling. We evaluate our approach across five widely used tabular benchmark datasets and compare it against strong training-based baselines from different model families (CTGAN, TVAE, TabDDPM, BeGReaT). A three-axis evaluation protocol has been used that covers representativeness, realism, and privacy. Across all evaluations, FewShotTabLLM outperforms baselines and surpasses them on many datasets, achieving realism, boundary adherence, and privacy preservation across all experiments. Overall, the results suggest that training-free fewshot LLM synthesis can serve as an alternative to existing models and a prominent research direction for improving dependency preservation and privacy-aware generation. Our code is available at: https://github.com/BMW-InnovationLab/FewShotTabLLM