Riemannian Regression

Abstract

Classical linear regression assumes that the relevant geometry of the predictor space is Euclidean and that all centered observations contribute to the least-squares fit in the same geometric scale. This paper proposes \emph{Riemannian Regression}, a regression framework in which the usual vector differences are replaced by locally weighted differences induced by a data-dependent similarity structure. We introduce a generalized framework, termed {\em Riemannian Regression}, extending classic regression to any data endowed with a local distance structure. By equipping data tables with local metrics, we adapt regression model to incorporate manifold geometry. Given a similarity matrix $S=(S_{ij})$, obtained from UMAP, ISOMAP, or DBSCAN \cite{mcinnes,isomap,dbscan}, we define the dissimilarity coefficient $\rho_{ij}=1-S_{ij}$ and the induced subtraction $ x_i\ominus x_j=\rho_{ij}(x_i-x_j). $ A Riemannian center $g=x_\lambda$ is selected as a discrete Fr\'{e}chet mean, and regression is performed on the Riemannian-centered variables $X_R=W X_{c,\lambda}$ and $y_R=W y_{c,\lambda}$, where $W=\operatorname{diag}(\rho_{1\lambda},\ldots,\rho_{n\lambda})$. The resulting estimator has the weighted least-squares form $ \widehat\beta_R=(X_{c,\lambda}^{t}W^2X_{c,\lambda})^{-1}X_{c,\lambda}^{t}W^2y_{c,\lambda}. $ The proposed approach preserves the linear form of the regression model while changing the geometry of the fit. The paper develops three ways to construct the local metric: UMAP-based fuzzy similarities, ISOMAP-based normalized geodesic distances, and DBSCAN-based density similarities. Simulated examples and the Abalone data set illustrate how Riemannian Regression can reduce the influence of locally anomalous observations and adapt to regions with different local densities.