Large vision-language models (LVLMs) and language models (LLMs) often exhibit attention sinks, whereby particular positions, such as the beginning-of-sequence (BOS) token, receive a disproportionate share of attention. Under standard causal masking, however, the BOS query attends only to itself, so the representation t...
Hui Zhu, Saleh Zare Zade, Rafi Ibn Sultan et al.· 0 citations
This work introduces DSAR (Deceptive Safety Alignment Rate), a metric that jointly assesses reasoning traces and final answers to quantify their safety inconsistency, and proposes SARA (Safety-Aware Reasoning Alignment), an RL-based method that rewards both safety-aware reasoning and safe final answers.
Xiang-Yu Zhou, Saleh Zare Zade, Rafi Ibn Sultan et al.· arXiv.org· 0 citations
This work shows that LRMs'reasoning processes can be systematically steered by prepending counter-aligned few-shot conversations containing explicit CoT traces, leading to unsafe generations on harmful queries and unwarranted refusals on benign ones, and formalizes this attack as SRCF (Steering Reasoning via Counter-Al...
Xiang-Yu Zhou, S. Z. Zade, Dong-Xiao Zhu· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.