Critical initialization destabilizes higher input derivatives in wide scalar-input networks
The edge-of-chaos condition preserves first-order input perturbations in wide randomly initialized networks, but physics-informed losses, score matching and derivative regularization depend on higher input derivatives. For smooth scalar-input fully connected networks, using a joint Gaussianity of the finite derivative...
Prashant Singh, Pranav Singh
· 0 citations