Sumio Watanabe

A Statistical Difference between Single-Layer Learning and Hierarchical Learning in Wide Neural Networks

It is shown that training the input-to-hidden weights yields a smaller generalization error than keeping them fixed, indicating that singularities play an essential role even in wide neural networks.