ExaModels.jl is presented, a Julia-based algebraic modeling system that exploits this structure to evaluate the objective, the constraints, and their derivatives in parallel and yields O(1) evaluation time regardless of the number of data points.
Abstract
Large-scale nonlinear programs almost always exhibit partially separable and repetitive structure, yet most existing algebraic modeling systems do not take advantage of it. A nonlinear optimization solver queries the objective, the constraints, and their derivatives at every iteration, so the speed of these evaluations bears directly on the overall solution time. We present ExaModels.jl, a Julia-based algebraic modeling system that exploits this structure to evaluate the objective, the constraints, and their derivatives in parallel. At its core is a single-instruction, multiple-data abstraction that represents a nonlinear program as a small number of algebraic patterns, each repeated over many data points. Because the patterns are visible at compile time, a specialized model and derivative evaluation kernel is compiled for each pattern. Applying that kernel independently across the data points maps naturally onto GPU parallelism and, with sufficiently many threads, yields O(1) evaluation time regardless of the number of data points. On the largest instances of the Luksan-Vlcek library, GPU execution speeds up sparse Hessian evaluation by 76x over single-threaded CPU evaluation, and by 30x on COPS and 7.3x on PGLIB-OPF.
Simple design principles that avoid severe load imbalance and expose parallelism in reductions that appear inherently sequential are described and implemented in cuPSLP, a GPU-accelerated version of PSLP, a CPU-based presolver that is itself several times faster than a state-of-the-art commercial presolver.
This paper introduces rlaopt, a PyTorch-based package for large-scale optimization and scientific computing using randomized numerical linear algebra (RandNLA). Despite substantial progress in RandNLA-based algorithms, few implementations combine GPU acceleration with a simple interface for specifying optimization prob...
Pratik Rathore, Zachary Frangella, P. Nobel et al.· 0 citations
Many applications require solving large numbers of independent linear systems on GPUs. While this need is well addressed for small to large systems, tiny ones, understood here as systems of dimension below 32, remain challenging. This is especially relevant in constitutive law evaluation, where millions of integration...
Tristan Chenaille, F. Cuteri, Raphaël Prat et al.· 0 citations
This work presents parallel algorithms for computing dependent partitions, and shows that these algorithms perform well on accelerators like GPUs and enable dynamic and on-line partitioning during steady-state execution.
Rohan Chanani, Alex Aiken, Michael Bauer et al.· 0 citations
A GPU-resident, batched Levenberg--Marquardt solver that optimizes constants across a structurally heterogeneous population of expression trees using a fixed number of population-wide CUDA launches per iteration.
NumPy is a widely used Python library for numerical scientific computing, known for its declarative APIs and its optimized implementations. However, writing efficient NumPy programs, which often entails using vectorized array operations instead of explicit Python loops, may not be straightforward. This can be difficult...