Mohammad Reza Rezaei

Online Label-Free Gradient Matching for Continual Language Model Training

Language models in customer-facing products must be continually updated, since the data they serve shifts over time. As retraining on the full accumulated data is impractical, each update should instead run on a small set of examples carrying the training signal of all data seen so far. The strongest current methods bu...