This paper derives the essential Riemannian tools and proposes efficient algorithms for their implementation of orthogonal two-factor matrices -- a flexible family of structured matrices formed by block-diagonal factors interleaved with fixed permutations.
Abstract
In this paper, we are concerned with matrices formed by block-diagonal factors interleaved with fixed permutations -- a flexible family of structured matrices. This class has recently drawn interest in deep learning architectures for its balanced expressivity-efficiency trade-off, yet efficient computational strategies for working with it remain to be found. We approach this problem through Riemannian geometry and examine under what conditions this class admits a smooth manifold structure. For the practically important case of orthogonal two-factor matrices, we derive the essential Riemannian tools and propose efficient algorithms for their implementation. The algorithms leverage automatic differentiation, support parameter sharing within each factor, and avoid explicit dense matrix construction. We test them within the Riemannian optimization framework on the best matrix approximation problem and for parameter-efficient fine-tuning of large language models. Beyond the two-factor setting, we study the geometric and matrix-theoretic properties of factorizations with a larger number of block-diagonal factors.
We propose an approach based on Riemannian optimization to compute a nearest normal matrix to a given one. The problem can be formulated as the minimization of a smooth function either on the manifold $U(n)$ of unitary matrices of size n or on the flag manifold $U (n)/U (1)^n$. The flag manifold is particularly suitabl...
This work is concerned with the numerical computation of Gross-Pitaevskii ground states using the tensor train (TT) format. We regard the problem as the minimization of the Gross-Pitaevskii energy functional under a unit mass constraint, and propose a first-order Riemannian optimization scheme that preserves both the m...
Ivan Bioli, Carlo Marcati, M. Rakhuba· 0 citations
Projection-free methods replace projections by linear minimization oracles, making their convergence fundamentally sensitive to the geometry of the feasible region. We develop a geometric framework for this dependence in Riemannian Frank--Wolfe optimization. A power-type double-geodesic modulus quantifies the intrinsic...
This paper proposes a novel Tensor Train (TT)-based tensor-on-tensor regression optimization framework for variable selection based on mode-1 hyperslice sparsity, and designs an alternating iterative algorithm equipped with a preconditioned metric to efficiently solve the proposed model.
It is proved that Riemannian Gradient Descent with diminishing step sizes converges linearly when RGA holds with order $r=1, and at rate $O(k-1/(2r-2)})$ for $r>1$.
Pu-Qian Wang, Nikos Zarifis, Jelena Diakonikolas· 0 citations
The concept of tensor eigenpairs has attracted increasing research attention in the past decades. Recent works have focused on a special class termed regular simplex tensors, which are constructed from an equiangular tight frame of n vectors in (n-1) dimensional space for n>= 3 and order m>= 3. Existing works focus on...
Lei Wang· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 13, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
MIT News · Artificial Intelligence· news.mit.eduSep 29, 2026
Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.