Unified Table Understanding via Context-Aware Cell Embedding Learning
TL;DR
UTUEL is introduced, a self-supervised, task-agnostic framework that casts Table Representation Learning as a Joint-Embedding Predictive Architecture (TRL-JEPA) and substantially improves Table Retrieval over generic embedders while remaining competitive on Column Type Annotation and Table Cell Retrieval.
Abstract
The rapid progress of Large Language Models (LLMs) has diverted attention from how table cells should be represented, the tabular analogue of word embeddings in NLP. Processing tables as flat token sequences, LLMs overlook cell-level semantics and lack a single, reusable representation that transfers across Table Understanding (TU) tasks. We introduce Unified Table Understanding via Embedding Learning (UTUEL), a self-supervised, task-agnostic framework that casts Table Representation Learning as a Joint-Embedding Predictive Architecture (TRL-JEPA). Each cell is linearized with its row and column context through a Semantic Meta-Path (SMP); an encoder refines the initial LLM embeddings into context-aware cell-level embeddings, and a predictor recovers a cell’s context embedding from its latent representation, distinguishing identical values in different contexts. Learned once per table, these embeddings are reused across queries and pooled to column- or table-level for different TU tasks. On a new benchmark and a large-scale dataset, UTUEL supports online Table Question Answering and, reusing the same embeddings without retraining, substantially improves Table Retrieval over generic embedders while remaining competitive on Column Type Annotation and Table Cell Retrieval. It achieves this with a compact 2.9–11.8M-parameter encoder, orders of magnitude smaller than the evaluated baselines in both parameters and per-query token cost.
