Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 13304-13307· 0 citations· 14 references
TL;DR
This tutorial presents a systematic overview of this emerging paradigm of tabular foundation models, which treats tables as a common representation that can capture information from tabular data, time series, and graphs within a shared learning framework.
Abstract
Structured data such as tabular data, time series and graphs powers many core data mining applications including recommendation, forecasting and user behavior analysis. Conventional approaches such as statistical models, classical machine learning methods and deep neural networks have achieved strong results. Yet most methods are designed for a single task or dataset and lack the ability to generalize across diverse structured data problems. Recent advances in foundation models point to a new direction for structured data modeling. Inspired by progress in natural language processing and computer vision, emerging research explores large-scale pretraining, synthetic data generation and in-context learning (ICL) to build more general-purpose models. In particular, tabular foundation models provide a promising path toward unifying heterogeneous structured modalities. This perspective treats tables as a common representation that can capture information from tabular data, time series, and graphs within a shared learning framework. Early studies show encouraging capabilities including cross-task generalization, few-shot adaptation and knowledge transfer across datasets. This tutorial presents a systematic overview of this emerging paradigm. We review classical foundations, introduce recent tabular foundation models, and discuss key challenges in pretraining data generation, model design, and multi-task learning.
Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.
Morad Laglil, Bertrand Pracca, Emilie Devijver et al.· 1 citation
This survey reviews the evolution of language models from early statistical approaches to modern Transformer-based architectures and summarizes key developments, including attention mechanisms, scaling laws, alignment techniques, and efficient inference methods.
P. Peykani, V. Charles, Ali Emrouznejad et al.· Archives of Computational Me...· 0 citations
Graph machine learning has witnessed rapid progress across both academia and industry. However, most existing methods are developed under the in-distribution (I.D.) hypothesis, which assumes that training and testing graph data are drawn from the same distribution. In real-world applications—ranging from dynamic knowledge graphs to evolving biomedical networks—this assumption is frequently violated, resulting in severe performance degradation under distribution shifts. Addressing this challenge has become a key focus in recent years, leading to the development of novel paradigms that move beyond the I.D. setting. This tutorial presents a comprehensive overview of three emerging and synergistic directions for tackling distribution shifts in graph learning. First, we highlight Graph LLMs, which combine the representational power of large language models with graph structures to enable flexible, in-context, and few-shot learning on graphs. Second, we introduce adaptation techniques for both GNNs and Graph LLMs, including graph neural architecture search and continual learning strategies for evolving data. Third, we cover generalization methods that incorporate causality and invariance principles to build robust graph models under unseen distributions. We will advocate novel, high-quality research findings, as well as innovative solutions to the challenging problems in graph machine learning under distribution shifts and the applications on graphs. This topic is at the core of the scope of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, and is attractive to machine learning as well as data mining audience from both academia and industry.
Xin Wang, Haoyang Li, Haibo Chen et al.· Proceedings of the 32nd ACM...· 0 citations
Learning to rank (LTR) traditionally requires large-scale training data to generalize effectively. In low-data domains where expert annotation is scarce, the performance of LTR methods degrades sharply. Foundation models have alleviated similar data dependencies in other domains via in-context learning, but a foundation model for ranking with tabular features has not been explored yet. We propose prior-data fitted networks (PFNs) as a strong method for ranking in low-data settings. First, we demonstrate that PFNs, which are originally trained for classification, successfully outperform classification baselines on ranking data. Next, we evaluate PFNs as rankers, showing that they surpass state-of-the-art tuned baselines in low-data regimes. We introduce a novel sampling and inference scheme to obtain pairwise predictions from PFNs' native pointwise architecture, analogous to pairwise LTR. To address the limited context window of the transformers underlying PFNs, we propose a dynamic support set selection strategy for queries that scales PFNs beyond random subsampling. Our experimental results show that PFNs are an effective foundation model for ranking that provides significant gains when data is limited.
David Vos, Samarth Bhargav, M. D. Rijke et al.· International Conference on...· 0 citations
This survey deeply explains the basic principles of representation learning, and introduces its practical application cases in various fields, and points out the main limitations of current models and prospects the future research directions.
Zhiyong Wang, Qiang He, Jun Mou et al.· Expert systems· 0 citations
Foundation models are increasingly deployed as agentic data-and-memory systems built on pretrained parameters, retrieval corpora, external knowledge stores, and persistent interaction histories. For the Knowledge Discovery and Data Mining (KDD) community, this matters because recommendation, search, temporal modeling, enterprise knowledge systems, and AI for science are tasked with organizing long-tail, hierarchical, and relational data while supporting retrieval, adaptation, and memory at scale. Yet Euclidean latent spaces can be a limited fit for tree-like or ontology-rich structure. Hyperbolic geometry offers a useful modeling tool: its exponential volume growth supports compact representations of hierarchy, association, and asymmetric neighborhoods. This lecture-style tutorial covers hyperbolic methods for data organization, retrieval, and memory layers in foundation-model systems: manifold operations, scalable neural primitives, retrieval-aware pipelines, recommendation and knowledge systems, agent memory, multimodal and scientific data modeling, and lifecycle operations including fine-tuning, editing, and unlearning. We emphasize when curved geometry can improve KDD systems and how to evaluate and deploy those gains responsibly. Homepage: https://hyperboliclearning.github.io/events/kdd2026tutorial.
Jiahong Liu, Menglin Yang, Irwin King· Proceedings of the 32nd ACM...· 0 citations