Skip to content

Enhancing Tabular Learners with Context-Aware Semantic Embeddings

Aug 2026 · 0 citations · 30 references
Computer Science

TL;DR

CASE (Context-Aware Semantic Embeddings), a novel framework that bridges the gap between the semantic understanding of Large Language Models and the statistical capabilities of tabular learners, substantially improves the performance of tabular learners on semantically rich datasets, particularly in low-data regimes.

Abstract

While modern tabular learners excel at capturing statistical patterns, they frequently operate in a semantic vacuum, treating textual features as discrete symbols, ignoring the rich semantics inherent in feature names or cell entries. We propose CASE (Context-Aware Semantic Embeddings), a novel framework that bridges the gap between the semantic understanding of Large Language Models (LLMs) and the statistical capabilities of tabular learners. Unlike existing methods that embed rows in isolation, CASE utilizes a contextualization strategy: we pre-fill the KV cache of a custom-trained Gemma 3-based Tabular Language Model with a representative sample of rows to establish a persistent anchor of the dataset's semantics. This ensures that generated row embeddings are dynamically contextualized, resolving semantic ambiguities and anchoring representations in domain-specific context. Our experiments across several benchmarks (CARTE, TextTab, and TabArena) demonstrate that CASE substantially improves the performance of tabular learners on semantically rich datasets, particularly in low-data regimes.

View source

Similar papers

#large language models Open access Nov 2026

Unified Table Understanding via Context-Aware Cell Embedding Learning

UTUEL is introduced, a self-supervised, task-agnostic framework that casts Table Representation Learning as a Joint-Embedding Predictive Architecture (TRL-JEPA) and substantially improves Table Retrieval over generic embedders while remaining competitive on Column Type Annotation and Table Cell Retrieval.

Willy Carlos Tchuitcheu, Tan Lu, Arthur Van Beersel et al. · 0 citations

LLMs as Feature Engineers for Text-and-Tabular Prediction

An iterative framework that automates the extraction of interpretable, schema-bound categorical features from unstructured text for tabular prediction models that guarantees instance-level interpretability and provides a fully transparent, semantic audit trail for every prediction.

Merwan Barlier, Blaž Škrlj · 1 citation
#artificial intelligence Preprint Sep 2026

MoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup

Mixture of Memory Embeddings (MoME), a context-aware memory mechanism that replaces each token's single memory row with a mixture of M slots and uses a learned gate over the hidden state to choose which slots to read at each position, shows more promising memory-size scaling trend at sub-billion scale and remains effic...

Mu-Chen Li, Leonid Sigal, Ren-Jie Liao · 0 citations
#artificial intelligence Preprint Sep 2026

DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents

This work introduces DocHop, a benchmark for integrated chart--context reasoning in document-style images and constructs DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, to enable systematic evaluation.

Zhuo-Ran Yu, Le Thien Phuc Nguyen, Jaden Park et al. · 1 citation
#natural language process... Preprint Sep 2026

Selecting What Matters: Semantic Compression-Guided Selective Pooling for Long-Context Embeddings

SCSP is a training-free framework that leverages semantic compression for informative token selection in long-context embedding and can be integrated into both zero-shot and fine-tuned models in a plug-and-play manner, consistently improving their performance.

Zi-Feng Cheng, Jie Zheng, Zhiwei Jiang et al. · 0 citations
Preprint Aug 2026

Domain-Agnostic Neural Topic Modeling with Contextual Token-Level Semantic Graph Representation

DARTopic is a domain-agnostic framework that constructs token-level semantic graphs from frozen PLM embeddings and jointly trains a GNN encoder with topic inference, demonstrating robustness to PLM choice and favorable runtime efficiency over fine-tuning based alternatives.

Seung-Won Seo, W. Cho, Yongmin Yoo · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.