Investigating Local Robustness of TabPFN on Small Numerical Binary Classification Tasks
It is discovered that TabPFN is more robust than all baselines on synthetic data under single-step FGSM attacks for moderate-to-large perturbation budgets, but that this advantage largely disappears under iterative PGD, suggesting that TabPFN’s gradient landscape obstructs single-step attacks.