Jun 2026· arXiv.org· Vol abs/2606.26900· 0 citations· 25 references
Computer ScienceMathematics
TL;DR
A deterministic incremental TT compression algorithm that operates directly on streaming TT-formatted data and corresponds to a compressed analogue of standard proper orthogonal decomposition for full-order snapshot data, enabling reduced-order models to be constructed directly from streaming low-rank solution data through operations on TT cores, without first reconstructing full snapshots.
Abstract
High-dimensional tensor data streams arise naturally in scientific and engineering applications, such as simulations of kinetic equations and quantum systems, where samples become available sequentially and are often already represented in compressed low-rank tensor formats. Existing streaming tensor-train (TT) algorithms typically construct or update representations from dense tensor data or randomized sketches. However, when high-dimensional data are generated directly in TT or related low-rank formats, reconstructing dense tensors solely for the purpose of compression is unnecessary and computationally prohibitive. We develop a deterministic incremental TT compression algorithm that operates directly on streaming TT-formatted data. Given a new TT tensor, the proposed method updates an accumulated TT representation through core-wise projection, residual orthogonalization, and adaptive enrichment, retaining only the complementary information that cannot be represented within a prescribed tolerance. By operating entirely at the level of TT cores, the algorithm avoids reconstructing either the incoming tensor or the accumulated full tensor. We establish approximation error bounds for the proposed incremental approach. Moreover, we show that the accumulated TT representation corresponds to a compressed analogue of standard proper orthogonal decomposition for full-order snapshot data, enabling reduced-order models to be constructed directly from streaming low-rank solution data through operations on TT cores, without first reconstructing full snapshots. Numerical experiments on parametric radiative transfer equations demonstrate that the proposed method achieves comparable reconstruction accuracy with substantially reduced wall time and yields efficient and accurate ROMs directly from compressed low-rank data.
Traditional model reduction techniques for neural network compression usually leverage existing matrix/tensor decomposition methods to explore the redundancy among model parameters, which may overlook the local structures of the models. In this work, we propose a novel low-rank approximation algorithm, called Dimension-Augmented Tensor-Train (DATT) decomposition, which integrates dimension slicing with the Tensor-Train decomposition. DATT allows different partitions to be represented using distinct, non-shared spanning sets. This enables the decomposition to adapt more flexibly to localized structure, potentially achieving better compression. With the localized low-rank structures, DATT provides a more compact representation of neural network weights. We evaluated the performance of DATT with the other state-of-the-art low-rank approximation-based model compression algorithms, and demonstrated that DATT outperforms them in terms of parameter reduction and model accuracy. In particular, for compressing ResNet-20 on CIFAR-10, DATT achieves a 77.00% reduction in parameters with only a 0.37% drop in top-1 accuracy.
Yu-Chuan Tai, Cheng-Yu Sie, Che-Rung Lee· Annual International Compute...· 0 citations
This work presents the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation to the more general tree tensor network format, enabling a unified treatment of various existing methods.
Alberto Bucci, Gianfranco Verzella· Numerical Linear Algebra wit...· 3 citations
Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the difficulty of performing nonlinear operations. Here, we introduce iterative tensor network transformations (ITNTs), a general algorithmic framework for the element-wise evaluation of elementary and nonlinear filtering functions on data encoded as tensor trains (TTs), a class of tensor networks. Our approach operates entirely in the compressed domain, enabling efficient computation on exponentially large datasets while maintaining a controlled computational cost. We demonstrate its power in two key areas: (I) evaluating highly nonlinear elementary and filtering functions on a 3D reactive flow field, enabling high-fidelity reaction rate computation and region filtering, and (II) finding extrema in complex optimization problems, such as solving Max-SAT instances on spaces up to $2^{70}$ configurations. These results establish ITNT as a foundational tool that provides tensor network methods with the capability for general-purpose data science and large-scale optimization.
Xiao Wang, Tomohiro Hashizume, Pia Siegl et al.· 2 citations
Tensors represent multidimensional data and cover various areas of scientific computing. The Canonical Polyadic Decomposition (CPD) emerges to extract latent patterns from large but highly sparse tensors. In real-world scenarios, tensor slices often arrive dynamically over time in streaming form, making traditional CPD algorithms inefficient in processing the entire tensor at each time step. Streaming CPD processes tensor slices incrementally, exploiting a forgetting factor to adjust the weight of historical information to capture dynamics. Current optimizations mainly focus on CPU platforms, failing to meet the real-time processing requirements of modern applications. Efficiently deploying streaming CPD on GPU remains challenging due to frequent data transfers and memory operations throughout the complex workflow, as well as the intricate computational patterns of bottleneck operators. To address these challenges, we propose ASTRA, a framework that optimizes the holistic streaming CPD process on GPU. At the system level, ASTRA integrates the memory pool and asynchronous strategy with compact storage format to reduce idle cycles and execution latency. At the operator level, ASTRA adopts dedicated techniques to customize the kernel implementations based on operator characteristics as well as an analytical model to adjust the parameters. Finally, ASTRA devises a similarity-aware updating mechanism to reduce the overall workload by skipping the calculations of highly similar adjacent time slices. The experimental results show that ASTRA achieves an average speedup of 1.71 × over the state-of-the-art methods with sufficient accuracy.
Wenqing Lin, Jianuo Sheng, Shuqin Feng et al.· International Conference on...· 0 citations
We develop a tensor reduced-order modeling (TROM) framework for optimization-based inverse problems governed by parameter-dependent dynamical systems. The approach approximates the parameter-to-observation map directly in tensor-train format, using either TT-SVD or TT-Cross compression, and integrates the resulting representation into a regularized nonlinear least-squares formulation. Beyond accelerating forward evaluations, the low-rank tensor structure is used to reformulate the inverse problem in reduced coordinates, assemble the Gauss--Newton quantities without forming the full observation-space Jacobian, and perform TROM-based objective minimization over the discrete parameter grid. This tensor optimization step can be used either as a stand-alone approximate minimization procedure or as a data-informed initialization for a subsequent Gauss--Newton solve. The method is studied for two inverse problems: an inverse heat-transfer problem in a heterogeneous medium, where the unknown parameters describe the locations of multiple low-conductivity inclusions, and a FitzHugh--Nagumo parameter-estimation problem with a highly nonconvex optimization landscape. Numerical experiments assess the effects of ROM approximation error, measurement noise, regularization, initialization, spatial discretization, and increasing parameter dimension. The results show that TROM can reproduce the behavior of full-order inversion at a substantially reduced online cost. The experiments also demonstrate that reduced-coordinate inversion, tensor-based optimization, and appropriate regularization improve robustness in higher-dimensional, noisy, and strongly nonconvex regimes.
S. Islam, Andreas Mang, Maxim A. Olshanskii· 1 citation
A novel reformulation of the butterfly structure as a compact tensor network is proposed, leading to speedups of orders of magnitude and significant accuracy improvements for large matrices compared to state‐of‐the‐art low‐rank matrix and QTT tensor completion algorithms.
Navjot Singh, Edgar Solomonik, X. Li et al.· Numerical Linear Algebra wit...· 0 citations