Normalized Low-Rank Adaptation
Normalized Low-Rank Adaptation (NoRA) is introduced, a simple yet effective method that normalizes the down-projection matrices during training, improving standard LoRA without requiring repeated normalization throughout training.