Benchmarking Attention for Tabular Foundation Models
Tabular in-context learners such as TabPFN, Mitra, or ConTextTab rely on alternating row and column attention over 2D sequences of latent embeddings. These attention patterns differ markedly from the one-dimensional case in language models: row attention involves longer sequences while column attention operates on much...
Maximilian Schambach, Clemens Biehl, Sam Thelin
· 0 citations