This work introduces PotARCin, a benchmark that extends ARC by assessing understanding of a task's underlying abstract rule across five dimensions: Definition, Classification, Constrained Generation, Editing, and Inversion, and introduces P-ARC, a held-out hand-crafted test set on which models achieve 1-8% accuracy acr...
Claas Beger, Ryan Yi, Melanie Mitchell· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.