Experiments show that with only 1K synthesized samples, AdvSafe-aligned LRMs achieve significantly stronger jailbreak robustness than existing baselines, with almost no utility degradation, demonstrating that learning unsafety knowledge enables a superior robustness-utility trade-off and generalizes beyond seen attack patterns.
Abstract
Large reasoning models (LRMs) achieve remarkable success on complex tasks but remain vulnerable to harmful prompts that induce unsafe outputs. Recent methods align LRMs using direct refusals or safety rationales, yet often focus on prompt patterns rather than intrinsic attack mechanisms. As a result, these pattern-centric alignments struggle to generalize across diverse jailbreaks, compromising adversarial robustness and reasoning utility. We propose AdvSafe, a dual-adversarial framework that enables LRMs to internalize unsafety knowledge by explicitly deconstructing adversarial mechanisms. This moves beyond pattern-dependent traces, fostering robust cognitive defense without compromising reasoning utility. Our pipeline operates via a two-phase adversarial game. First, in adversarial synthesis, an autonomous agent dynamically crafts deceptive jailbreak prompts, adapting its strategies to breach a strong teacher model. Second, in adversarial extraction, the breached teacher executes a cognitive counter-attack. For every successful jailbreak, the teacher unmasks the camouflage, explaining why the attack succeeds and how such prompts can be identified and mitigated. This dual-adversarial process yields a compact reasoning dataset capturing rich, generalizable unsafety knowledge. Student models trained on this dataset implicitly acquire safety alignment through intrinsic threat comprehension. Experiments show that with only 1K synthesized samples, AdvSafe-aligned LRMs achieve significantly stronger jailbreak robustness than existing baselines, with almost no utility degradation. Furthermore, AdvSafe improves robustness against out-of-distribution prompts, demonstrating that learning unsafety knowledge enables a superior robustness-utility trade-off and generalizes beyond seen attack patterns.
Multi-turn jailbreak attacks have emerged as a critical safety threat to LLMs, as harmful objectives are decomposed across a sequence of apparently benign turns to bypass guardrails. Existing defenses lack the reasoning capacity to identify evolving manipulation patterns, often trading helpfulness for safety by over-refusing benign requests related to sensitive topics. We introduce Trace, a multi-turn defense with trajectory-aware structured reasoning. Before generating each response, the model identifies manipulation cues from the trajectory, evaluates both the benign and adversarial interpretations of user intent, assigns a jailbreak score, and commits to an action: Allow, Caution, or Decline. We curate 4k multi-turn adversarial conversations from five attack frameworks, pair them with 2.4k benign dialogs, and 600 sensitive-but-benign conversations. We train Llama-3.1-8B-Instruct with SFT and GRPO under a multi-component reward that jointly optimizes helpfulness on benign prompts and robustness against jailbreak attempts. Across seven multi-turn attack benchmarks, Trace attains an average attack success rate (ASR) of 14.5% against 31.4% for the strongest baseline and 74.9% for the undefended target, while significantly raising the attacker effort required per successful jailbreak. Trace also balances usability and safety, achieving a 93.3% average compliance on over-refusal benchmarks.
This paper proposes a novel LVLM attack method, called BadPhase with further backdoor designs, to implant adversarial phase as triggers into any image inputs via data poisoning so as to control the LVLMs’ predictions and finds that LVLMs are sensitive to the phase-aware image structure.
Daizong Liu, Junhao Dong, Xiang Fang et al.· 0 citations
Multimodal guard models have emerged as critical safety components for screening content in vision-language systems. While adversarial research has extensively studied jailbreaking attacks that produce false negatives, the inverse threat of inducing false positives on benign inputs remains unexplored. We introduce Unsafe Induction Attacks, where adversaries distribute imperceptibly perturbed safe images that trigger guard models to reject legitimate user requests, causing a ''Boy Who Cried Wolf'' effect that degrades service availability and erodes trust. This reveals an availability failure mode in deployed safety filters. To realize this threat under diverse user prompts, we propose Unsafe Semantic Distillation (USD), which aligns adversarial perturbations with distributional representations of unsafe content rather than prompt-specific instances. Evaluated on four state-of-the-art guard models across realistic user simulation scenarios, USD achieves 84% attack success rates, outperforming existing methods and exposing fundamental vulnerabilities in current multimodal safety architectures. WARNING: This paper contains harmful content.
Shuo Shi, Ruiping Yin, Naen Xu et al.· Proceedings of the 32nd ACM...· 1 citation
Large reasoning models (LRMs) expose a new safety failure mode: adversarial prompts can manipulate reasoning context, task decomposition, or capability interpretation so that harmful objectives are processed as legitimate reasoning steps. Existing safeguards, including safety reminders, external classifiers, and self-checking wrappers, are often brittle because they either inspect the adversarial prompt directly or ask the target model to perform additional safety reasoning on the same surface that attacks exploit. We introduce Capability-Routed Guard (CRG), a model-agnostic inference-time guardrail for closed-source LRMs, where defenders cannot inspect hidden reasoning traces or modify model weights. CRG reframes prompt defense as a capability-routing problem: a side-channel controller first constructs a trusted representation of the user's authorized task, active context, safety evidence, and capability-transfer risk, separating executable intent from untrusted reasoning context. This representation supports route-specific execution, allowing CRG to block high-risk requests, constrain ambiguous ones, and forward low-risk requests through trusted active context. Finally, CRG applies TraceCheck to verify consistency with the authorized task and invokes a restricted fallback to preserve utility for low-risk benign prompts. Extensive experiments demonstrate that CRG effectively mitigates diverse reasoning-centric jailbreaks while preserving benign utility and avoiding common over-refusal issues. Further analysis shows that its components contribute complementary benefits, highlighting the importance of coordinated defense mechanisms for securing large reasoning models.
Multi-agent LLM pipelines orchestrate multiple specialized language model agents into structured workflows where intermediate outputs are passed across agents to solve complex tasks. This design introduces a security gap absent in single-agent settings: once an agent accepts adversarial content, it is propagated as trusted input throughout the pipeline. We argue that this vulnerability stems from the absence of boundary verification, a security primitive that enforces explicit validation of data as it crosses inter-agent boundaries, including content, identity, execution intent, and state integrity. Without such verification, modern pipelines embed implicit trust assumptions that are not adversarially robust, giving rise to structurally distinct attack surfaces (e.g., content injection, agent impersonation, plan deviation, and memory poisoning). Leveraging annotated production traces from the GAIA and SWE-Bench benchmark, we show that these vulnerabilities arise in benign deployments and largely evade existing evaluation frameworks. We further operationalize these failure modes within a controlled multi-agent setting and evaluate them across GPT-5-mini, Claude Sonnet 4.5, and Kimi K2.5 under identical pipeline configurations. The results reveal that attack success aligns with pipeline structure rather than model capability, indicating that adversarial vulnerability is fundamentally an architectural property and motivating a shift toward pipeline-level defenses.
Faisal Haque Bappy, Tahrim Hossain, T. S. Zaman et al.· 0 citations
The democratization of Cyber Threat Intelligence (CTI) through Large Language Models (LLMs) represents a paradigm shift in organizational security posture, enabling non-expert users to access and interpret complex threat data through intuitive natural language interfaces. However, this democratization introduces a critical paradox: the very mechanisms that lower the barrier to CTI consumption simultaneously expand the attack surface for adversarial manipulation. This paper presents a comprehensive investigation into the dual challenge of CTI democratization and adversarial resilience, proposing a novel Multi-Layer Adversarial Defense Framework (MLADF) and the CTI-SHIELD (Systematic Hardening and Intelligence-Enhanced LLM Defense) algorithm. Our threat model—formalized through the STRIDE methodology—identifies four primary adversarial attack classes: prompt injection, data poisoning, output manipulation, and model extraction, and systematically analyses their exploitation pathways in CTI deployment contexts. We introduce the Adversarial Resistance Score (ARS), a composite metric for evaluating the robustness of CTI LLM interfaces, and a Confidence-Aware Response Sanitization (CARS) technique combining Monte Carlo dropout uncertainty quantification with knowledgebase factgrounding. Experimental evaluation through structured red-team exercises demonstrates that MLADF reduces adversarial attack success rates by an average of 87.3% compared to unprotected baselines while maintaining 91.4% user comprehension accuracy and achieving calibration error (ECE) of 0.028—superior to all competing methods. A human-in-the-loop user study (n=48) confirms that our protected interface reduces adversarial susceptibility by 76.0% (p<0.001) without degrading usability. Ablation studies confirm all four MLADF layers are individually necessary, and cross-domain generalization experiments across five security verticals confirm AUC > 0.90 outside the CTI training domain.
Manas Kumar Yogi, K. C. Bheri· International Journal of Adv...· 0 citations