Skip to content
Review Open access

An Introduction to Stochastic Deep Learning

Sep 2026 · WIREs Computational Statistics · Vol 18 · 0 citations · 71 references

TL;DR

Stochastic neural network broadens deep learning from prediction‐oriented function approximation to a richer framework for statistical inference, and two related architectures are discussed: kernel‐expanded StoNet and sublinearly structured DNNs, which can be viewed as structural or theoretical extensions of the StoNet framework.

Abstract

Deep neural networks (DNNs) have achieved remarkable success in prediction, but their deterministic formulation makes many statistical inference tasks difficult. StoNet, short for stochastic neural network, addresses this limitation by reformulating a DNN as a probabilistic latent‐variable model, in which the outputs of selected hidden layers or units are treated as latent variables. The article describes the formulation of StoNet and its asymptotic equivalence to a conventional DNN, showing how StoNet can serve both as a deep learning model and as an analytical device for studying DNNs. It reviews training algorithms for StoNet and discusses its applications to statistical inference problems, including nonlinear sufficient dimension reduction, causal inference with observed and unobserved confounders, Granger‐causality learning for nonlinear time series, nonlinear variable selection, and prediction uncertainty quantification for large‐scale DNNs. The article also discusses two related architectures: kernel‐expanded StoNet and sublinearly structured DNNs, which can be viewed as structural or theoretical extensions of the StoNet framework. Under suitable architectural conditions, the former is immune to local traps in training and tends to outperform conventional DNNs in prediction; whereas the latter achieves feature‐learning consistency in over‐parameterized regimes when the training sample size is sufficiently large. Together, these developments show that StoNet broadens deep learning from prediction‐oriented function approximation to a richer framework for statistical inference.

Read PDF

Similar papers

Preprint Aug 2026

Prediction Inference of Time Series with Standard ReLU Deep Neural Networks

This work proposes a methodology based on the standard ReLU Deep Neural Networks (DNN) to make predictions and quantify their uncertainty, and builds the so-called pertinent prediction interval (PPI) with the DNN model estimator.

Kejin Wu · 0 citations
#artificial intelligence Preprint Aug 2026

Frequency Selective Neural Networks as a Foundation Architecture for Time Series Learning

Frequency Selective Neural Network is introduced, pioneering a foundation architecture guaranteeing physical interpretability without sacrificing expressive power of deep learning, and converges directly on physically meaningful frequency bands, providing a highly scalable, interpretable paradigm for robust pattern rec...

Hui Huang, Ye Sun, Shixuan Hu · 0 citations

Structured Neural SDEs for Functional Calibration

SLiSDE, a family of Neural SDE models built from structured linear stochastic layers, is introduced and it is proved well-posedness, a discretisation error bound, validity of the change of measure, and a universality result: the terminal laws of the gated stack are dense in the space of square-integrable laws.

Francesco Piatti, Andrea Iannucci, Thomas Cass · 0 citations
#artificial intelligence Preprint Sep 2026

A Study of Hidden-State Optimization Order in Predictive Coding Networks

A boundary-first inference schedule that partitions a model into chunks, first coordinates hidden states at chunk boundaries, and then refines representations within each chunk is proposed, which instantiate in predictive coding networks (PCNs), a local-learning framework in which hidden activities and prediction error...

Xue-Yuan Li, Danilo Vasconcellos Vargas · 0 citations
#artificial intelligence Review Sep 2026

Memory in Deep Time-Series Models

This work develops a unified taxonomy of memory mechanisms and reviews three classes of external memory, including explicit modules, retrieval augmentation, and agentic stores, under a common framework for what is retained, how it is written and accessed, and how it persists.

M. Nguyen, H. Nguyen, Manh Nguyen et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.