ClaimNov is introduced, a fine-grained, interpretable claim-level novelty module that catches 97% of disguised papers at a 1.5% false-alarm rate, and cuts the LLM judge's false alarms from 11.8% to 1.5%.
Miri Liu, Xi-Yu Zhao, En-Cheng Xie et al.· 0 citations
Competence-based evaluation (CBE) is argued for, which tests whether models reliably apply the same competence across a family of problems and provides diagnostic profiles and an evaluation protocol that can grow with future models.
Instruction-following language models can comply with requests that conflict with intended behaviors and safety alignment. We introduce a training-free weight-editing approach for reducing compliance with targeted undesirable instructions. Our method needs only forward passes on a small set of prompt pairs per target b...
Ali Ebrahimpour-Boroojeny, Xiao-Cong Yang, Samuel Schapiro et al.· 0 citations
Large language models (LLMs) hold broad knowledge in one set of weights, but lack interpretability and trustworthy reasoning, especially in high-stakes use cases. A neuro-symbolic model, with its interpretable architecture and System 2 reasoning capacity, is a promising alternative if built at the scale of LLMs; howeve...
Xiao-Cong Yang, Xuan-Yu Meng, Yu-Ji Zhang et al.· 0 citations
This work investigates language models as scalable evaluators by comparing direct prompt-based, prompt-derived feature, and representation-based methods, and shows that representation-based approaches consistently outperform prompt-based judging in predicting user engagement, suggesting they provide a reliable foundati...
Varun Joshi, Eva C. Song, Cheng-Xiang Zhai· 0 citations
Large language models (LLMs) are increasingly integrated into educational settings, yet educators lack robust, standards-aligned tools to evaluate their effectiveness in K-12 science contexts. Existing benchmarks predominantly assess general language or advanced scientific reasoning, leaving a critical gap in understan...
Noah L. Schroeder, Yessy Ambarwati, Yu-Ji Zhang et al.· arXiv.org· 0 citations
It is proposed to formalize GIANT generally as an agent interacting with its users to minimize their effort on finishing a task as well as the agent's resource overhead and propose a probabilistic modeling framework for optimizing GIANT's interactions with its users.
Cheng-Xiang Zhai· Annual International ACM SIG...· 0 citations
The results reveal a pattern distinct from prior findings on long-tail vulnerability during acquisition and retention: among facts that models already answer correctly, those associated with highly connected entities are more likely to be corrupted by neighboring updates, and updates to such facts propagate errors more...
Yu-Ji Zhang, Wei-Bing Wang, Cheng Qian et al.· 0 citations
This work presents StudentSim, a training framework that turns sparse per-student data into individualized simulators through pooled training followed by per-student specialization, and introduces StudentSimEval, a standardized protocol covering 60 students across chess, second-language English writing, and mathematics...
Ke Yang, Chenglong Wang, Michel Galley et al.· 1 citation
VGI-bench is introduced, containing 27 tasks and 810 instances, organized by a two-level taxonomy of task domains and skill tags for fine-grained evaluation of visual reasoning capabilities of video generation models, and it is hoped VGI-bench will help stimulate the development of next-generation video generation mode...
Xuan He, Cong Wei, Yuhao Cheng et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.