Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure true algorithmic performance. We present results of the updated AlgoPerf V1 benchmark, in which we evaluate a new set of training algorithms on wall-clock time-to-targe...
Priya Kasimbeg, Ahmed Khaled, Aahlad Chandrabhatta et al.· 0 citations
A framework for efficient scaling law construction that formulates data collection as a Bayesian optimization problem is proposed, and metrics for comparing scaling law fitting methods under constrained compute budgets are introduced.
This study establishes a first baseline and scaling procedure for the development of future OpenEuroLLM models, and characterize the dependence of loss on model capacity and dataset size, evaluating recently proposed scaling forms that explicitly model their interaction.
Niccolò Ajroldi, Diana-Alexandra Onutu, Haider Al-Tahan et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.