The framework and EvalAwareBench provide the tools to measure, attribute, and mitigate evaluation awareness, building the foundation for future solutions to ground evaluation awareness in social psychology.
Changling Li, Terry Jingchen Zhang, Jie M. Zhang et al.· arXiv.org· 1 citation
This work introduces RAPTOR - a Role-Aware Private Training framework, which alternates shared and expert optimization and targets each failure directly, using expert-specific clipping and noise together with a public expected-owner denominator and a count-independent update schedule that avoids conditioning on private...
Duc Dm, Khai Le-Duc, D. Nguyen et al.· 0 citations
Evaluations of social bias in LLMs largely focus on whether models generate or imply biased content. However, as LLMs are increasingly used as judges of bias, they may exhibit social biases in subtler ways in how they evaluate biased content, which current methods do not systematically capture. We call this second-orde...
Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed et al.· arXiv.org· 0 citations
Cue-induced bias is best understood not as a single flaw in LLMs but as a family of causally effective linear directions that are largely shaped by alignment tuning.
Prakhar Gupta, Terry Jingchen Zhang, Florent Draye et al.· arXiv.org· 1 citation
Evaluating three frontier models across six games in homogeneous and heterogeneous groups over 10 rounds, it is found that different models interpret announcements incompatibly, some as binding commitments and others as cheap talk, producing payoff gaps that emerge in Round~0 and persist across all 10 rounds.
Jerick Shi, Terry Jingchen Zhang, Bernhard Scholkopf et al.· 0 citations
Concept-Targeted Attribution (CTA) provides a framework for moving from behavioral probe accuracy to mechanistic explanations of probe performance, enabling more detailed audits of internal concept representations, including safety-critical ones.
V. Palit, Florent Draye, Terry Jingchen Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.