Manifold-Constrained Hyper-Connections for Parameter-Efficient Finetuning
Manifold-Constrained Hyper-Connections, a generalisation of residual connections, is investigated as a novel PEFT approach, wrapping frozen OLMo-2 backbones with learned residual routing modules, identifying residual routing as a distinct and promising novel PEFT axis.