Xing-Jian Bai

Reactive Optimizers

A reactive optimizer that uses a pretrained language-model agent to inspect training dynamics and rewrite executable optimizer code during a single training run is studied, evaluating this approach on language modeling and image generation when a recipe has not been tuned for the target setting.