Accelerating the Canonical Polyadic Alternating Least Squares Optimization via a Randomized Interpolative Decomposition

Abstract

We present a novel leverage score-based sampling strategy for the randomized alternating least squares optimization (ALS) of the canonical polyadic decomposition (CPD-ALS). Unlike previous strategies, we determine row-wise samples for the CPD-ALS problem from the leverage scores of the target tensor which is being decomposed. We demonstrate that, when rows are sampled according to the leverage score distribution of the matricized target tensor, each least squares subproblem of the CPD-ALS problem achieves $(1+\epsilon)-$relative accuracy in the residual norm with probability at least $1-\delta$ using a sampling $s=\frac{R\gamma}{\beta} \max\left(\frac{4}{\delta \epsilon}, \frac{144\ln(2R/\delta)}{\epsilon_{0}^{2}}\right)$, where $\epsilon_{0}$ is a constant, $\beta$ is leverage score's approximation constant, $R$ is the target rank and $\gamma$ captures the coherence between the Khatri Rao product (KRP) of the CPD factor matrices and the exact KRP; $\gamma$ decreases as the ALS iterates converge. To efficiently approximate the leverage score distribution for each matricization of the target tensor without explicitly computing leverage scores we use a randomized strong rank-revealing QR (sRRQR) factorizations, SE-QRCS. By construction, this QR-based leverage score sampling method outperforms previously published schemes as it does not, in principle, require the resampling of the target tensor or recomputing the leverage scores of the KRP, minimizing the computational and storage overhead of the CPD-ALS procedure.