CASE (Context-Aware Semantic Embeddings), a novel framework that bridges the gap between the semantic understanding of Large Language Models and the statistical capabilities of tabular learners, substantially improves the performance of tabular learners on semantically rich datasets, particularly in low-data regimes.
Abstract
While modern tabular learners excel at capturing statistical patterns, they frequently operate in a semantic vacuum, treating textual features as discrete symbols, ignoring the rich semantics inherent in feature names or cell entries. We propose CASE (Context-Aware Semantic Embeddings), a novel framework that bridges the gap between the semantic understanding of Large Language Models (LLMs) and the statistical capabilities of tabular learners. Unlike existing methods that embed rows in isolation, CASE utilizes a contextualization strategy: we pre-fill the KV cache of a custom-trained Gemma 3-based Tabular Language Model with a representative sample of rows to establish a persistent anchor of the dataset's semantics. This ensures that generated row embeddings are dynamically contextualized, resolving semantic ambiguities and anchoring representations in domain-specific context. Our experiments across several benchmarks (CARTE, TextTab, and TabArena) demonstrate that CASE substantially improves the performance of tabular learners on semantically rich datasets, particularly in low-data regimes.
UTUEL is introduced, a self-supervised, task-agnostic framework that casts Table Representation Learning as a Joint-Embedding Predictive Architecture (TRL-JEPA) and substantially improves Table Retrieval over generic embedders while remaining competitive on Column Type Annotation and Table Cell Retrieval.
Willy Carlos Tchuitcheu, Tan Lu, Arthur Van Beersel et al.· IEEE Transactions on Knowled...· 0 citations
An iterative framework that automates the extraction of interpretable, schema-bound categorical features from unstructured text for tabular prediction models that guarantees instance-level interpretability and provides a fully transparent, semantic audit trail for every prediction.
Mixture of Memory Embeddings (MoME), a context-aware memory mechanism that replaces each token's single memory row with a mixture of M slots and uses a learned gate over the hidden state to choose which slots to read at each position, shows more promising memory-size scaling trend at sub-billion scale and remains effic...
Mu-Chen Li, Leonid Sigal, Ren-Jie Liao· 0 citations
This work introduces DocHop, a benchmark for integrated chart--context reasoning in document-style images and constructs DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, to enable systematic evaluation.
Zhuo-Ran Yu, Le Thien Phuc Nguyen, Jaden Park et al.· 1 citation
SCSP is a training-free framework that leverages semantic compression for informative token selection in long-context embedding and can be integrated into both zero-shot and fine-tuned models in a plug-and-play manner, consistently improving their performance.
Zi-Feng Cheng, Jie Zheng, Zhiwei Jiang et al.· 0 citations
DARTopic is a domain-agnostic framework that constructs token-level semantic graphs from frozen PLM embeddings and jointly trains a GNN encoder with topic inference, demonstrating robustness to PLM choice and favorable runtime efficiency over fine-tuning based alternatives.
Seung-Won Seo, W. Cho, Yongmin Yoo· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026