This work proposes and demonstrates an alternate objective for a new interpretability objective: understanding model internals to predict behavior and assess reliability under distribution shift, and successfully uses attention patterns to predict which rule each model follows on OOD data.
Victoria R. Li, Jenny Kaufmann, Tian Qin et al.· arXiv.org· 13 citations· ⚡1
Wasserstein curriculum paths is presented, a simple transport-based framework that decouples factors by representing curricula as trajectories of training distributions over discrete difficulty levels and improves hard-level performance relative to exposure-matched static sampling.
Compute-Data (CD) scaling laws are proposed, a unified framework that bridges compute-optimal scaling, where data scales freely with compute, and data-optimal scaling, where the corpus is fixed while compute can grow without bound.
Tian Qin, K. Hamidieh, David Alvarez-Melis· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.