Enhancing Medical Data Imputation Using a Denoising Autoencoder with Missing-Neighborhood Perturbation.
TL;DR
Experimental results demonstrate that MI_NPDAE consistently outperforms baseline methods across various missing mechanisms and ratios, maintaining lower imputation errors, and the imputed data significantly improves performance in downstream predictive tasks, highlighting the practical value of the approach in clinical data analysis.
Abstract
In real-world clinical settings, the diverse types and unknown causes of missing data in tabular medical datasets pose significant challenges for accurate imputation. In particular, non-random missingness-where missing values are related to unobserved variables-limits the effectiveness of many existing imputation models. To address these challenges, we propose a robust and generalized imputation method: Multiple Imputation based on Neighborhood Perturbation Denoising Autoencoder (MI_NPDAE). MI_NPDAE identifies optimal donor records by leveraging neighborhood information, which is used as input to the autoencoder. The model reconstructs perturbed inputs to learn robust feature representations around missing regions, while the introduction of additive noise exposes the model to a variety of missingness patterns, enhancing its adaptability. We evaluate MI_NPDAE on two datasets: a publicly available Breast dataset and a lung cancer nutrition dataset from the Chinese Anti-Cancer Society. Experimental results demonstrate that MI_NPDAE consistently outperforms baseline methods across various missing mechanisms and ratios, maintaining lower imputation errors. Moreover, the imputed data significantly improves performance in downstream predictive tasks, highlighting the practical value of our approach in clinical data analysis.