Jul 2026· Numerical Linear Algebra with Applications· Vol 33· 0 citations· 18 references
Abstract
In this work, we present the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation, such as for Tucker and tensor‐train formats, to the more general tree tensor network format, enabling a unified treatment of various existing methods. Our method retains the key features of the generalized Nyström approximation for matrices, that is, randomized, single‐pass, streamable, and cost‐effective. Additionally, the structure of the sketches allows for parallel implementation. We provide a deterministic error bound for the algorithm and, in the specific case of Gaussian dimension reduction maps, also a probabilistic one. We also introduce a sequential variant of the algorithm, referred to as sequential tree tensor network Nyström (STTNN), which offers better performance for dense tensors. Furthermore, both algorithms are well‐suited for the recompression or rounding of tensors in the tree tensor network format. Numerical experiments highlight the efficiency and effectiveness of the proposed methods.
This work presents the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation to the more general tree tensor network format, enabling a unified treatment of various existing methods.
Alberto Bucci, Gianfranco Verzella· Numerical Linear Algebra wit...· 3 citations
A novel reformulation of the butterfly structure as a compact tensor network is proposed, leading to speedups of orders of magnitude and significant accuracy improvements for large matrices compared to state‐of‐the‐art low‐rank matrix and QTT tensor completion algorithms.
Navjot Singh, Edgar Solomonik, X. Li et al.· Numerical Linear Algebra wit...· 0 citations
A new SVD-based tensor decomposition method for tensor networks with arbitrary graph topologies is introduced, and it is found that the graph-format representation attains comparable or better accuracy than the classical tensor train and hierarchical Tucker tensor formats, while using substantially fewer degrees of freedom at lower computational cost.
Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the difficulty of performing nonlinear operations. Here, we introduce iterative tensor network transformations (ITNTs), a general algorithmic framework for the element-wise evaluation of elementary and nonlinear filtering functions on data encoded as tensor trains (TTs), a class of tensor networks. Our approach operates entirely in the compressed domain, enabling efficient computation on exponentially large datasets while maintaining a controlled computational cost. We demonstrate its power in two key areas: (I) evaluating highly nonlinear elementary and filtering functions on a 3D reactive flow field, enabling high-fidelity reaction rate computation and region filtering, and (II) finding extrema in complex optimization problems, such as solving Max-SAT instances on spaces up to $2^{70}$ configurations. These results establish ITNT as a foundational tool that provides tensor network methods with the capability for general-purpose data science and large-scale optimization.
Xiao Wang, Tomohiro Hashizume, Pia Siegl et al.· 2 citations
Abstract.
This work proposes an efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format. The two settings we consider are variational inference and density estimation, whereby one has access to either the unnormalized analytic formula of the distribution or the samples generated from the distribution. In particular, the compression is done through a two-stage approach. In the first stage, we use existing subroutines to encode the distribution function in a tensor train format. In the second stage, we use an NTT ansatz to fit the obtained tensor train. For the NTT fitting procedure, we use a log barrier term to ensure the positivity of each tensor component and then utilize a second-order alternating minimization scheme to accelerate convergence. In practice, we observe that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed. Through challenging numerical experiments, we show that our approach can accurately compress target distribution functions.
Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and Data Available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/Xun-Tang123/Non_Negative_Tensor_Train and in the supplementary materials ( Non_Negative_Tensor_Train-main.zip [51.4MB]), linked from the main article webpage. [Formula: see text]
Xun Tang, R. Dwaraknath, Lexing Ying· SIAM Journal on Scientific C...· 0 citations
Traditional model reduction techniques for neural network compression usually leverage existing matrix/tensor decomposition methods to explore the redundancy among model parameters, which may overlook the local structures of the models. In this work, we propose a novel low-rank approximation algorithm, called Dimension-Augmented Tensor-Train (DATT) decomposition, which integrates dimension slicing with the Tensor-Train decomposition. DATT allows different partitions to be represented using distinct, non-shared spanning sets. This enables the decomposition to adapt more flexibly to localized structure, potentially achieving better compression. With the localized low-rank structures, DATT provides a more compact representation of neural network weights. We evaluated the performance of DATT with the other state-of-the-art low-rank approximation-based model compression algorithms, and demonstrated that DATT outperforms them in terms of parameter reduction and model accuracy. In particular, for compressing ResNet-20 on CIFAR-10, DATT achieves a 77.00% reduction in parameters with only a 0.37% drop in top-1 accuracy.
Yu-Chuan Tai, Cheng-Yu Sie, Che-Rung Lee· Annual International Compute...· 0 citations