This paper studies the estimation of a marginal regression function from independent units with repeated binary, count, or continuous responses using ReLU deep neural networks, and derives an oracle inequality for unequal cluster sizes and a rate for compositionally smooth functions.
Abstract
In this paper, we study the estimation of a marginal regression function from independent units with repeated binary, count, or continuous responses using ReLU deep neural networks. In the model, we assume that the dependence is generated by an unobserved random mean function within each unit. We then fit a neural network with a convex generalized regression loss. We show an oracle inequality by separating conditional measurement variation from between-unit variation. In addition, we prove that with $n$ units and $m$ measurements per unit, ReLU networks can attain an integrated mean squared error of order $n^{-1}+(nm)^{-2\beta/(2\beta+d)}$, up to logarithmic factors, over $\beta$-H\"older classes. We also derive a weighted oracle inequality for unequal cluster sizes and a rate for compositionally smooth functions. For pointwise ensemble inference, we give a projection central limit theorem and prove infinitesimal jackknife consistency under an explicit asymptotic linearity condition. Simulations and real data examples are provided to support our theoretical findings and practical implications.
Under mild design conditions, satisfied by a broad class of correlated random designs, it is shown that EBMoM consistently estimates a growing number of moments and hence the prior itself, provided that $n\geq p^{1-o(1)}$.
Zhou Fan, Yandi Shen, Hao-Yu Wang et al.· 0 citations
A (computationally inefficient) adaptive estimator that, so long as $p$ is a mixture of $k$ symmetric log-concave densities, achieves error comparable with the optimal estimator that knows $p$ and has $\tilde\Theta(n/k)$ samples.
A finite-sample theory for continuous density ratio estimation and importance-weighted learning under target shift is established and the resulting regression estimator attains the minimax-optimal rates of standard kernel regression.
It is shown that the expected $L_2$ error of the estimate achieves a rate of convergence which depends on $d^*$ and not on the dimension $d$ of the design.
The no-data criterion is a safe default, matching or improving on truncation for HSGP, VFF and VISH, with substantial gains for VISH and improvements over a recently developed selection heuristic for that basis family.
Marnix Van Soom, I. De Boi· arXiv.org· 0 citations
Estimation of a regression function from independent and identically distributed data is considered. The $L_2$ error with integration with respect to the design variable is used as the error criterion. An initially randomly pruned fully connected deep neural network with logistic squasher as activation function is fitt...
M. Kohler, V. M. Römer, Adam Krzyżak· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 6, 2026
A machine-learning approach called CheMeleon provides a practical way to develop new drugs or other molecules with much less detailed, expensive experimentation.
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.