Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 1381-1392· 0 citations· 19 references
Abstract
Pre-trained tabular prediction models based on Prior-Data Fitted Networks (PFNs), such as TabPFN and LimiX, have achieved remarkable progress in supervised learning, demonstrating immense potential across real-world scenarios and diverse downstream tasks. However, a critical question remains systematically unexplored: Does pre-training on data generated via causal mechanisms truly endow models with the ability to comprehend underlying causal structures? Furthermore, can these models leverage such causal information to achieve stable prediction across environments? To address these fundamental questions, we propose StablePFN, a novel tabular foundation model that integrates explicit causal awareness with stable predictive modeling. Leveraging a key yet largely overlooked advantage of the PFN paradigm, the availability of ground-truth causal structure during synthetic data generation, we train StablePFN to jointly identify the Markov Boundary (MB) of the target variable and perform the primary prediction task. We introduce an end-to-end ''Decouple-Discover-Predict'' architecture that utilizes sample decoupling weights to guide MB discovery, and employs a hard attention masking mechanism during inference to incorporate causal structural knowledge. Extensive experiments on both synthetic and real-world benchmarks demonstrate that StablePFN significantly outperforms state-of-the-art baselines in cross-environment prediction settings, particularly in challenging high-bias scenarios.
We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling. Rather than centering the network on the $p(y \mid x, D_{\mathrm{context}})$ objective of conventional tabular PFNs, it is designed around learning $p(x, y \mid D_{\mathrm{context}})$, a context-dependent representation of the joint structure underlying data generation. Pretraining uses synthetic datasets generated by structural causal models (SCMs) spanning diverse graph structures, functional mechanisms, and observation processes. Evaluations on TabArena, TALENT, and BCCO show that LimiX-2 outperforms current dataset-specific models and tabular foundation models. Beyond predictive performance, the CMN paradigm also promotes causal awareness in LimiX-2: its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.
Xingxuan Zhang, Gang Ren, Hao Yuan et al.· 0 citations
Predictive process monitoring (PPM) leverages event logs to forecast the future of running process instances, for instance, predicting the next activity, the remaining time until case completion, or the time to the next event. While PPM research in recent years has been dominated by deep sequence models trained from scratch, such as Long Short-Term Memory (LSTM) models, foundation-model approaches---particularly large language models (LLMs)---are increasingly explored for PPM. At the same time, tabular foundation models with in-context learning capabilities offer a promising alternative but have not yet been systematically benchmarked for PPM. Thus, it remains unclear whether classical sequence-based models remain competitive in this evolving landscape. This paper compares the three modeling paradigms both conceptually and empirically through a controlled benchmark across multiple datasets and prediction tasks. The results show that sequence models consistently perform best for next activity prediction, whereas tabular foundation models are competitive on temporal tasks, with LLMs usually lagging behind despite higher cost.
L. Fertig, Lukas Kirchdorfer, Tobias Sesterhenn· arXiv.org· 0 citations
World modeling enables intelligence to anticipate consequences, guide interventions, and learn from interaction. Yet predictive models remain domain-specific: can a common learning principle support world modeling across radically different systems? We introduce JEPA-Anything, a domain-agnostic framework based on orthogonal predictive factorization (OPF). Extending joint-embedding predictive architectures, OPF decomposes latent targets into complementary factors, learns them through dedicated pathways, and recombines them within a shared predictive design. We evaluate JEPA-Anything across seven domains: vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather. Experiments span representation learning, intervention prediction, out-of-distribution generalization, and long-horizon dynamics, including 10 matched dynamics tasks, forecasting of over 1,000 clinical events, and 100-step molecular rollouts across four systems. Against matched JEPA baselines, JEPA-Anything improves reported metrics on all 10 dynamics tasks and reduces single-intervention prediction error on Interventional Pong by 34.8%. It achieves the lowest one-step and 100-step molecular errors among compared methods in all four systems. Beyond prediction, a factor-nominated biological intervention receives experimental support in cell co-cultures, patient-derived organoids, tumor fragments, and mice; latent orbital modes recover the Keplerian scaling exponent with a fitted slope of -1.4991. These results support a common factorized predictive principle across heterogeneous worlds, connecting world modeling with intervention and experimentally grounded scientific discovery. Code: https://github.com/Gen-Verse/JEPA-Anything
Tao-Yong Cui, Zhong-Yao Wang, Xin-Yue Xu et al.· 0 citations
Causal graph learning seeks to identify informative causal subgraphs and exclude non-causal elements to explain GNN predictions. However, existing methods struggle with spurious correlations and robust generalization, particularly when faced with biased data. To address these challenges, we propose the Explainable Causal Learning with disentAngled uncertainty and Interventional Reasoning(ECLAIR) framework, which identifies precise causal structures for improved and generalizable prediction interpretation. ECLAIR reduces irrelevant information transmission by using dual-perspective attention scores to separate causal from non-causal features and employs uncertainty learning to quantify subgraph certainty. Its lightweight tiered parametric-efficient optimization balances certainty-driven refinement with uncertainty-aware generalization, optimizing both accuracy and computational efficiency for resource-constrained settings. Extensive experiments on two synthetic and seven real-world datasets demonstrate ECLAIR's state-of-the-art performance in bridging disentanglement gaps, particularly under high data bias and out-of-distribution scenarios. Qualitative visualizations further reveal clearer causal structures isolated from spurious backgrounds as bias intensifies. Our code and datasets will be made available upon acceptance.
Traditional machine learning models achieve strong predictive performance but often are unable to reliably uncover causal relationships required for reliable decision-making, particularly in observational data where controlled experiments are not feasible. This limitation creates a critical gap between prediction and actionable insight, as correlation-based models are vulnerable to confounding bias and poor generalization under distributional shifts. To address this challenge, this study proposes a unified causal machine learning framework that integrates structural causal modeling with data-driven estimation techniques to enable robust causal discovery and effect estimation. The methodology combines hybrid causal structure learning (constraint-based and score-based approaches) with advanced causal effect estimation methods, including propensity score techniques and doubly robust estimators. The framework is evaluated on both synthetic datasets with known causal structures and real-world datasets to assess its accuracy, robustness, and interpretability. Experiments are conducted using multiple runs with controlled settings to ensure reproducibility and statistical validity. The results demonstrate that the proposed framework significantly outperforms traditional predictive models and standalone causal methods. It achieves higher causal discovery accuracy with improved precision and recall of causal edges, reduces estimation error in Average Treatment Effect (ATE), and maintains stable predictive performance under distributional shifts. Statistical analysis confirms significant improvements (p < 0.01) with large effect sizes, indicating strong reliability and robustness. This research aims to bridge the gap between prediction and explanation by enabling machine learning systems to generate actionable, interpretable, and causally valid insights. The findings highlight the importance of integrating causal reasoning into data science workflows to support informed decision-making, intervention planning, and trustworthy AI development.
Maria Ulfa, M. Alshar'e, Dharmesh Dhabliya et al.· Journal of Data Science· 0 citations
