Simulated versus non-simulated imputed data: A comparison of handling missing data techniques for numerical covariates

Abstract

Missing data poses a great challenge in much research, and if not appropriately handled, can negatively impact the analysis and bias the results and study conclusions. This article assesses the performance of numerous methods used to impute missing data by using Root Mean Squared Error (RMSE) under various missing data proportions and two common missingness mechanisms, namely Missing at Random (MAR) and Missing Completely at Random (MCAR). RMSE values were used to judge the accuracy of imputation techniques using real data and across different simulation settings. The results showed magnitude of RMSE increased with increased proportions of missing data, regardless of the mechanisms used to generate missing data. Under MCAR conditions, real and simulated datasets exhibited similar patterns of performance for multiple imputation-based techniques, including Expectation-Maximization via Bootstrapping (EMB), Multiple Imputation by Chained Equations (MICE), and Predictive Mean Matching (PMM). Contrary, single imputation-based techniques like Series Mean demonstrated notably different values of RMSE. Nonetheless, PMM displayed the best performance among all applied techniques, yielding minimum values of RMSE of 5.8 for the simulated and 7.5 for the real dataset under 0.15 (15%) proportion of missing data under MAR mechanism. The study recommends the application of multiple imputation-based techniques when handling missing data, as they result in the lowest bias and improve performance. Also, researchers are advised to compare results from both unimputed and imputed-based results to ascertain the potential risk of missing data values prior to making study inferences.