Daniel Soudry

Optimal Explicit and Implicit Regularization Strengths in High-Dimensional Continual Linear Regression

We study continual linear regression in a high-dimensional regime, where a learner observes a sequence of tasks and updates its parameters sequentially. In this setting, we derive an explicit closed-form expression for the generalization loss of a model trained with gradient descent for a fixed number of steps per task...