Toward Solving the Learning Rate Problem in Convolutional Neural Networks Using an Enhanced Cyclical Learning Rate Strategy
The Cyclical Local Adaptive Learning Rate Strategy (CLARA) is proposed, an enhanced CLR that introduces scheduler-level adaptive learning rate adjustment and dynamically adjusts the lower learning rate bound in response to training behaviour to improve convergence consistency during training.