From Imperfect Geotechnical Data to Accurate Prediction: Rockburst Classification Using the Pretrained Transformer TabPFN Model

Abstract

Predicting underground mining rockbursts remains challenging because geotechnical data sets are often small, sparse, incomplete, and noisy, resulting from inconsistent testing standards, high acquisition costs, measurement errors, and geological variability. Consequently, traditional machine learning (ML) models frequently exhibit poor predictive accuracy and reliability. To address these issues, this study introduces the Tabular Prior-Data Fitted Network (TabPFN), a novel pretrained transformer that leverages approximate Bayesian inference to generalize effectively from small, noisy data sets. Moreover, this model can natively handle missing values, mitigate overfitting, reduce computational time, and simplify practical deployment, all without requiring fine-tuning. The model was evaluated on 150 rockburst cases using five key parameters to predict four rockburst intensity classes. To quantify its advantages over traditional ML methods, TabPFN was systematically benchmarked against support vector machine, multilayer perceptron, decision tree, random forest, and extreme gradient boosting. The results demonstrated TabPFN’s superior predictive accuracy on noisy and imbalanced data sets, and this advantage persisted in limited-data scenarios. It also performed robustly on incomplete data sets, with further gains when paired with imputation methods like k -nearest neighbors. Furthermore, the model provided strong interpretability, identifying the elastic energy index as the most critical predictive factor, followed by tangential stress and buried depth. TabPFN’s practical utility was further confirmed through successful application to real-world engineering projects (93% accuracy). Ultimately, this study establishes TabPFN as a powerful and reliable tool for rockburst prediction and highlights the broader potential of this pretrained model to effectively address the persistent challenge of scarce, poor-quality data in engineering geology.