Zhiqiang Tao

Random Sparse Networks Training with Sharpness-Aware Regularization.

Over-parameterization is critical for optimizing neural networks, whereas training sparse networks directly often fails to achieve satisfactory performance. However, the Lottery Ticket Hypothesis (LTH) demonstrates that a randomly initialized dense model has a sparse subnetwork that can be identified through iterative...