Stochastic neural network broadens deep learning from prediction‐oriented function approximation to a richer framework for statistical inference, and two related architectures are discussed: kernel‐expanded StoNet and sublinearly structured DNNs, which can be viewed as structural or theoretical extensions of the StoNet framework.
Abstract
Deep neural networks (DNNs) have achieved remarkable success in prediction, but their deterministic formulation makes many statistical inference tasks difficult. StoNet, short for stochastic neural network, addresses this limitation by reformulating a DNN as a probabilistic latent‐variable model, in which the outputs of selected hidden layers or units are treated as latent variables. The article describes the formulation of StoNet and its asymptotic equivalence to a conventional DNN, showing how StoNet can serve both as a deep learning model and as an analytical device for studying DNNs. It reviews training algorithms for StoNet and discusses its applications to statistical inference problems, including nonlinear sufficient dimension reduction, causal inference with observed and unobserved confounders, Granger‐causality learning for nonlinear time series, nonlinear variable selection, and prediction uncertainty quantification for large‐scale DNNs. The article also discusses two related architectures: kernel‐expanded StoNet and sublinearly structured DNNs, which can be viewed as structural or theoretical extensions of the StoNet framework. Under suitable architectural conditions, the former is immune to local traps in training and tends to outperform conventional DNNs in prediction; whereas the latter achieves feature‐learning consistency in over‐parameterized regimes when the training sample size is sufficiently large. Together, these developments show that StoNet broadens deep learning from prediction‐oriented function approximation to a richer framework for statistical inference.
This work proposes a methodology based on the standard ReLU Deep Neural Networks (DNN) to make predictions and quantify their uncertainty, and builds the so-called pertinent prediction interval (PPI) with the DNN model estimator.
Frequency Selective Neural Network is introduced, pioneering a foundation architecture guaranteeing physical interpretability without sacrificing expressive power of deep learning, and converges directly on physically meaningful frequency bands, providing a highly scalable, interpretable paradigm for robust pattern rec...
SLiSDE, a family of Neural SDE models built from structured linear stochastic layers, is introduced and it is proved well-posedness, a discretisation error bound, validity of the change of measure, and a universality result: the terminal laws of the gated stack are dense in the space of square-integrable laws.
Francesco Piatti, Andrea Iannucci, Thomas Cass· arXiv.org· 0 citations
A boundary-first inference schedule that partitions a model into chunks, first coordinates hidden states at chunk boundaries, and then refines representations within each chunk is proposed, which instantiate in predictive coding networks (PCNs), a local-learning framework in which hidden activities and prediction error...
This work develops a unified taxonomy of memory mechanisms and reviews three classes of external memory, including explicit modules, retrieval augmentation, and agentic stores, under a common framework for what is retained, how it is written and accessed, and how it persists.
M. Nguyen, H. Nguyen, Manh Nguyen et al.· 0 citations