Hong-Yi Wang

Width-Independent Compressibility of Deep Neural Networks

A uniform compressibility theorem for deep multilayer perceptrons with analytic activations is proved, which involves a novel derivative-matching technique which is aware of the low-dimensional input, and a layer-wise reweighting that preserves the input-output mapping.