Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
This work introduces Bayesian in-context experimenters: transformer policies trained to imitate a Bayesian posterior Neyman teacher, and proves this amortized policy can be learned via empirical risk minimization using supervised pretraining.