Training ML Models to Adapt at Test Time

Abstract

Modern machine learning systems concentrate most of their computation beforedeployment and then use the resulting model as a fixed predictor. This design is brittle when deployment differs from training and poorly matched to difficult problems whose solution benefits from additional reasoning, feedback, or interaction after the input is known. This thesis studies a complementary paradigm: training models to use computation at test time as a means of adapting their behavior to the instance at hand. The central claim is that test-time computation should be understood as a learned adaptive procedure rather than simply a longer prediction. A useful procedure must decide which operations to perform, which information to seek, how to interpret intermediate feedback, and when further effort is worthwhile. For foundation models, these decisions form a sequential process that may include generation, verification, search, revision, and interaction. Learning such a process requires more than examples of successful final answers: it requires signals that distinguish operations that make genuine progress from those that merely resemble successful behavior. This perspective connects robustness under distribution shift with scalable reasoning. In both settings, a static predictor is limited by information unavailable during training, while adaptation can exploit evidence revealed at deployment. The thesis develops principles for constructing that evidence, assigning credit to intermediate decisions, and optimizing policies that use it. Structured assumptions and counterfactual data expose failure modes that ordinary training can overlook. Verification and outcome feedback provide scalable supervision, while reinforcement learning converts that supervision into adaptive behavior. The resulting procedures can improve with additional test-time compute, remain effective beyond the budgets seen during training, and allocate effort according to problem difficulty rather than spending a fixed amount on every instance. Across the theoretical analyses and empirical studies, three conclusions emerge. First, additional computation is valuable only when training rewards progress rather than length or superficial imitation. Second, the model's initialization acts as a prior over the adaptive procedures that learning can discover, so data, representation, and optimization must be designed together. Third, effective test-time adaptation depends on balancing exploration, exploitation, and information acquisition under a finite compute budget. These principles apply across robustness, mathematical reasoning, and interactive decision making, and suggest a path from fixed predictors toward systems that continue to learn how to solve a problem after it is presented."