W. Gerstner

Teacher Geometry Shapes Learnability in Teacher-Student Networks

This work formalizes learnability as the success rate of converging to the global minimum, as a function of overparameterization, learning algorithm, student initialization distribution, and teacher geometry, and shows that differentially increasing the learning rate of the readout layer and decreasing the learning rat...