Jul 2026· International Conference on Supercomputing· pp. 752-765· 0 citations· 43 references
Computer Science
TL;DR
A new method for automatically generating parametric mappings agnostic of the input problem sizes for the class of DAGs of tensor computations is introduced, based on the insight that in parametric DAGs not all parameters are equally important.
Abstract
Tensor computations are an important class of operations widely used in domains such as computational chemistry, machine learning, and various types of physical simulations that demand distributed-memory clusters. Recent work has shown that generating efficient mappings for multi-operator Directed Acyclic Graphs of distributed-memory tensor computations is possible by leveraging non-linear formulations underpinned by Satisfiability Modulo Theories (SMT) solvers. However, this approach is sensitive to the problem size, grid shape, and count of Processing Elements (PEs) given. We introduce a new method for automatically generating parametric mappings agnostic of the input problem sizes for the class of DAGs of tensor computations. The approach relies on three building blocks: the insight that in parametric DAGs not all parameters are equally important, an offline algorithm to generate a small bounded number of mapping classes – forming a family of mappings – and a sampling space that enables the generation of sufficiently large and diverse family of mapping classes. We show the effectiveness of our parametric mappings on a cluster of multi-GPUs, comparing against NVIDIA cuTensorNet, the Cyclops Tensor Framework (CTF) and PyTorch. Our mappings achieve speedups up to 22.1 × and 2.35 × over NVIDIA cuTensorNet, on matrix-based and tensor-based DAGs, respectively. When benchmarked against PyTorch, we observe speedups of 3.41 ×.
Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the difficulty of performing nonlinear operations. Here, we introduce iterative tensor network transformations (ITNTs), a general algorithmic framework for the element-wise evaluation of elementary and nonlinear filtering functions on data encoded as tensor trains (TTs), a class of tensor networks. Our approach operates entirely in the compressed domain, enabling efficient computation on exponentially large datasets while maintaining a controlled computational cost. We demonstrate its power in two key areas: (I) evaluating highly nonlinear elementary and filtering functions on a 3D reactive flow field, enabling high-fidelity reaction rate computation and region filtering, and (II) finding extrema in complex optimization problems, such as solving Max-SAT instances on spaces up to $2^{70}$ configurations. These results establish ITNT as a foundational tool that provides tensor network methods with the capability for general-purpose data science and large-scale optimization.
Xiao Wang, Tomohiro Hashizume, Pia Siegl et al.· 2 citations
A novel reformulation of the butterfly structure as a compact tensor network is proposed, leading to speedups of orders of magnitude and significant accuracy improvements for large matrices compared to state‐of‐the‐art low‐rank matrix and QTT tensor completion algorithms.
Navjot Singh, Edgar Solomonik, X. Li et al.· Numerical Linear Algebra wit...· 0 citations
This work presents the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation to the more general tree tensor network format, enabling a unified treatment of various existing methods.
Alberto Bucci, Gianfranco Verzella· Numerical Linear Algebra wit...· 3 citations
Tensors represent multidimensional data and cover various areas of scientific computing. The Canonical Polyadic Decomposition (CPD) emerges to extract latent patterns from large but highly sparse tensors. In real-world scenarios, tensor slices often arrive dynamically over time in streaming form, making traditional CPD algorithms inefficient in processing the entire tensor at each time step. Streaming CPD processes tensor slices incrementally, exploiting a forgetting factor to adjust the weight of historical information to capture dynamics. Current optimizations mainly focus on CPU platforms, failing to meet the real-time processing requirements of modern applications. Efficiently deploying streaming CPD on GPU remains challenging due to frequent data transfers and memory operations throughout the complex workflow, as well as the intricate computational patterns of bottleneck operators. To address these challenges, we propose ASTRA, a framework that optimizes the holistic streaming CPD process on GPU. At the system level, ASTRA integrates the memory pool and asynchronous strategy with compact storage format to reduce idle cycles and execution latency. At the operator level, ASTRA adopts dedicated techniques to customize the kernel implementations based on operator characteristics as well as an analytical model to adjust the parameters. Finally, ASTRA devises a similarity-aware updating mechanism to reduce the overall workload by skipping the calculations of highly similar adjacent time slices. The experimental results show that ASTRA achieves an average speedup of 1.71 × over the state-of-the-art methods with sufficient accuracy.
Wenqing Lin, Jianuo Sheng, Shuqin Feng et al.· International Conference on...· 0 citations
Modern machine learning (ML) methods are highly effective for prediction tasks, but many commonly used representations reduce complex data to fixed dimensional embeddings that may suppress multiscale structural organization. The Mapper algorithm from topological data analysis (TDA) provides a different perspective by decomposing data into overlapping local regions connected through a nerve construction, producing a structured representation that captures geometric organization, local statistical behavior, and relational connectivity simultaneously. In this work, we develop a framework for learning over Mapper induced structured representations. Rather than treating Mapper as a preprocessing step that produces a graph for downstream learning, we treat the full Mapper construction as part of the representation itself. We study mathematical properties of these representations, including invariance under relabeling, a distance functional on the space of Mapper representations, structural complexity of multiscale decompositions, and learning oriented stability under representation perturbations. Experiments on time series and graph classification datasets validate the proposed framework through controlled studies of representation ablation, Mapper parameter sensitivity, and the geometry of the induced representation space. Together, these results demonstrate how the proposed mathematical framework enables systematic comparison, interpretation, and analysis of Mapper representations, providing practical tools for studying representation geometry, structural complexity, and learning stability in learning tasks.
This paper introduces Tensor Field Models (TFMs), realization-level Mathematical Structures in which a learned Operator maps a product of admissible component-section families to a prescribed family of time-dependent tangent sections on a Generative State Manifold. Analytic and dynamical restrictions are encoded through the choice of admissible families rather than imposed by the root definition. Constructed, component-separable, and Tensor Bundle TFMs provide structured refinements of this common object. In the conditional realizations considered here, a structured condition $c=(c_1,\ldots,c_n)$ is mapped componentwise to a reusable collection $\mathbf H_c=(H_{c_1}^{(1)},\ldots,H_{c_n}^{(n)})$. In the architectures evaluated here, the component representations remain distinct and are combined only by the Field Operator to produce the generated Vector Field. All learned models are trained using Flow Matching. Experiments show that TFMs can improve performance and that amortized sampling enabled by reusable condition representations can accelerate generation.