NoRA improves LoRA training with a one-line change, a paper summary says

A post summarizing a paper on Normalized Low-Rank Adaptation (NoRA) says the method normalizes LoRA’s down-projection matrices during training. It reports faster convergence, better final performance, more stable training and less catastrophic forgetting—the loss of previously learned capabilities—across pretraining, supervised fine-tuning and reinforcement learning. The post also says applying the normalization just once, at initialization, improves standard LoRA and is cheaper than repeating it throughout training.