New multi-task PCA procedures are proposed that exploit similarity structure across tasks to improve eigenspace estimation while remaining robust to outlier tasks, addressing a key challenge in robust multi-task learning.
Abstract
Principal component analysis (PCA) is a fundamental tool for learning low-dimensional structure from high-dimensional data. When data are collected from multiple sources, the underlying task distributions may exhibit unknown degrees of similarity, with some tasks potentially arising from arbitrary distributions. We propose new multi-task PCA procedures that exploit similarity structure across tasks to improve eigenspace estimation while remaining robust to outlier tasks. We establish non-asymptotic convergence rates and show that the proposed procedures attain minimax optimal rates in a range of regimes. One of the procedures builds on the matrix-depth notion of Chen, Gao, and Ren (2018) and can achieve the optimal dependence of the estimation error on the proportion of outlier tasks, addressing a key challenge in robust multi-task learning. Extensive simulations and real-data analyses demonstrate the effectiveness of the proposed methods.
Spectral methods are central to high-dimensional statistics and machine learning, underlying procedures for covariance estimation, matrix denoising, representation learning, clustering, and latent variable modeling. In this work, we focus primarily on spectral estimators for high-dimensional factor models, where leadin...
We develop a transfer learning framework for high-dimensional sparse canonical correlation analysis (CCA) with multiple heterogeneous source datasets. Our goal is to improve estimation of the target canonical coefficient matrices by borrowing information from related sources. We characterize source-target similarity th...
Disentangling shared (joint) structures from source-specific (individual) variations is a fundamental task in multi-source data integration. Existing joint-individual models often rely on computationally intensive optimization or loose spectral bounds, leading to suboptimal separation accuracy and poor scalability. In...
Conditional dimension reduction (cDR) learns low-dimensional latent coordinates while accounting for observed covariates that represent known sources of variation in the data. Conditional Multidimensional Scaling (cMDS) is a cDR technique that works directly with dissimilarity data. Its standard squared-stress formulat...
Principal component analysis and factor analysis are foundational to the study of high-dimensional data, yet their efficacy depends entirely on correctly identifying the latent dimension $r$. While parallel analysis (PA) is widely regarded as the gold standard for this task, its performance suffers in high-dimensional...
Simulations and an empirical application show that SMPCA is more accurate and stable than competitors under heavy-tailed distributions and outlier contamination, while retaining competitive performance under light-tailed settings.
Dong-Xu Yang, Wanfeng Liang, Le Zhou et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
