Dan Goodman

Emergent Generalization by Representation Learning in Artificial Neural Networks

It is shown that an explicit information bottleneck forcing a recurrent neural network to learn a low-dimensional representation is necessary for rotational and out-of-distribution generalisation in a time-series prediction task, and the ability of neural networks to learn compact, distributed and emergent representati...