Predictive modeling of user retention based on online behavior in media service platforms

Abstract

Due to rapid competition among digital media platforms, the focus has now shifted toward user retention and continued engagement rather than user acquisition. User retention is a vital metric for prediction, enabling media service providers to implement tailored interventions and curb churn-related losses. Nevertheless, much of the current literature relies on proprietary datasets, thereby reducing reproducibility, transparency, and comparability across studies. To fill this gap, this paper presents a clear and replicable predictive modeling framework for user retention by using publicly available data. We built a rich set of behavioral, temporal, content-based, and demographic features using the MovieLens 1M dataset, which includes over 1 million ratings from 6040 users, and categorized users based on a 30-day inactivity threshold. Several machine learning algorithms, including Logistic Regression, Decision Tree, Support Vector Machine, Random Forest, Multilayer Perceptron, Gradient Boosting, XGBoost, LightGBM, CatBoost, and TabPFN 2.5, were trained and tested using cross-validation and common performance indicators, including accuracy, F1-score, and AUC. TabPFN 2.5 achieved the highest overall predictive performance, with an accuracy of 81.4%, an F1-score of 0.764, and an AUC of 0.866, while Gradient Boosting remained the strongest interpretable conventional ensemble model, achieving an accuracy of 80.2%, an F1-score of 0.752, and an AUC of 0.852. The findings also indicate that behavioral and time-related variables, especially rating frequency, inter-rating intervals, and genre diversity, are the strongest predictors of user retention. Altogether, this work provides a reproducible baseline for predicting retention using open data and offers clear insight into variables related to continuous user interaction. Because the MovieLens 1M dataset reflects historical user interactions, the findings should be interpreted primarily as a reproducible methodological baseline for retention modeling rather than as a direct representation of current media-platform behavior.