A. Saydjari

Interleaved Noise Injection Improves Clean, Corrupted, and OOD Performance

It is shown that the effect of interleaved noise injection stems from penalizing the failure modes encouraged by the inductive bias of the models: impulse noise works against the locality bias of convolutional (ResNet) architectures, and Gaussian noise reduces the tendency of attention-based models to pick up large-sca...