This work introduces PotARCin, a benchmark that extends ARC by assessing understanding of a task's underlying abstract rule across five dimensions: Definition, Classification, Constrained Generation, Editing, and Inversion, and introduces P-ARC, a held-out hand-crafted test set on which models achieve 1-8% accuracy acr...
Claas Beger, Ryan Yi, Melanie Mitchell· 0 citations
A novel two-step TTT protocol is introduced: first finetune only the task embedding (Embed-TTT), then freeze it and finetune the backbone, which shows that Embed-TTT consistently yields improved task embeddings, ones that align better with underlying task rules, improve embedding-based retrieval, and enable accurate li...
A. Deliège, Claas Beger, Marc Van Droogenbroeck et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.