SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximation

Abstract

Deep neural networks (DNNs) and Kolmogorov-Arnold networks (KANs) are flexible function approximators, but their accuracy can require substantial network depth, width, and trainable parameter counts. Beyond making the resulting networks less transparent, large parameter spaces can increase training cost and produce solutions that are sensitive to optimization and parameter initialization, leading to variability in generalization error. For these reasons, we propose Shallow Universal Polynomial Networks (SUPNs), which retain a single hidden layer while increasing the expressivity of each unit through a learnable multivariate polynomial inside each activation, which we refer to as a polynomial pre-activation. This construction trades architectural depth and width for greater expressivity within each hidden unit, reducing parameter counts and training cost while preserving a shallow computational graph. We establish two complementary universal approximation results: SUPNs are universal by increasing network width at fixed polynomial complexity, as in classical shallow-network approximation, and by increasing polynomial complexity at fixed finite width. We further prove that finite SUPNs can approach the best approximation attainable in a prescribed polynomial space to arbitrary relative tolerance, derive explicit quasi-optimal parameters, and establish corresponding convergence rates. We empirically evaluate SUPNs against DNNs, KANs, and polynomial projection on smooth, nonsmooth, and discontinuous function-approximation problems in one, two, and ten dimensions using more than 16,000 trained models. Across the target functions studied, SUPNs frequently achieve lower approximation error and variability than DNNs and KANs at comparable parameter counts while retaining lower training cost, supporting their use as efficient function approximators when the target structure is not known a priori.