SIGMA is proposed, a data generation and training pipeline enabling alignment self-improvement that generalizes to out-of-distribution settings and shows that a Model Spec balancing harmlessness and helpfulness, test-time reasoning for safety deliberation, and high-quality rubrics from SIGMA's task designer agent are c...
Jing-Yu Zhang, Shruti Palaskar, Daniel Khashabi et al.· 0 citations
CLUE (Calibrated Latent Uncertainty Estimation), which combines self-consistent teacher calibration and distribution-based confidence probing to produce calibrated predictions, is introduced, which achieves equivalent or stronger performance than zero-shot baselines up to 32B parameters across all modalities.
Dengjia Zhang, Alexander Martin, William Jurayj et al.· arXiv.org· 4 citations
This work presents AutoMat, a benchmark for evaluating LLM-based agents'ability to reproduce claims from computational materials science, and curates a set of claims from real materials science papers to test whether coding agents can recover and execute the end-to-end workflow needed to support (or undermine) such cla...
Ziyang Huang, Yiyun Cao, A. Shargh et al.· arXiv.org· 9 citations
The Information Abundance Paradox is supported and it is suggested that scaling toward near-infinite context is not simply a matter of supplying more data, even when high-quality long-context data is abundant.
Arda Uzunouglu, Benjamin Van Durme, Daniel Khashabi· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.