Knowledge that a language model appears to forget during finetuning often remains stored and can be recovered, a phenomenon called spurious forgetting. Finetuning on new facts can even produce forgetting that undoes itself: recall of the old facts collapses, recovers as training continues on new facts alone, and only t...
V. Palit, Florent Draye, Nicolas Zucchet et al.· 0 citations
This work introduces RAPTOR - a Role-Aware Private Training framework, which alternates shared and expert optimization and targets each failure directly, using expert-specific clipping and noise together with a public expected-owner denominator and a count-independent update schedule that avoids conditioning on private...
Duc Dm, Khai Le-Duc, D. Nguyen et al.· 0 citations
Evaluations of social bias in LLMs largely focus on whether models generate or imply biased content. However, as LLMs are increasingly used as judges of bias, they may exhibit social biases in subtler ways in how they evaluate biased content, which current methods do not systematically capture. We call this second-orde...
Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed et al.· arXiv.org· 0 citations
Cue-induced bias is best understood not as a single flaw in LLMs but as a family of causally effective linear directions that are largely shaped by alignment tuning.
Prakhar Gupta, Terry Jingchen Zhang, Florent Draye et al.· arXiv.org· 1 citation
Concept-Targeted Attribution (CTA) provides a framework for moving from behavioral probe accuracy to mechanistic explanations of probe performance, enabling more detailed audits of internal concept representations, including safety-critical ones.
V. Palit, Florent Draye, Terry Jingchen Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.