Jun 2026· 2026 8th International Conference on Electronic Engineering and Informatics (EEI)· pp. 231-234· 0 citations· 14 references
Abstract
Large language models (LLMs) may still produce unsafe responses under optimized adversarial prompts, even after safety alignment. Existing gradient-based jailbreak evaluation methods usually use raw gradient signals for discrete prompt search, but raw gradients may be unstable in both magnitude and direction. This paper proposes a gradient-calibrated adversarial prompt optimization framework for LLM safety evaluation. The framework adopts REINFORCE-style randomized discrete prompt optimization as the parent objective and inserts a unified calibration interface before gradient-derived token-search signals are used. We instantiate this interface with two complementary primitives: LSGM for layer-wise magnitude calibration and LiLA for representation-space directional calibration. Experiments on HarmBench show that joint calibration improves attack success rate on multiple open-source safety-aligned models. Ablation results further suggest that the two primitives are more stable when used jointly than when used alone. The framework provides a compact white-box diagnostic tool for analyzing gradient quality in discrete adversarial prompt optimization, while transfer and defense results reveal clear limitations in generalizability and practical robustness.
Recent jailbreak attacks demonstrate that large language models (LLMs) can be manipulated to generate harmful outputs through adversarial prompts even after robust alignment. However, prevailing methods typically focus on forcing a desired response at the output layer—a surface-level strategy that is brittle and often fails to bypass the more fundamental safety checks embedded within the model’s internal mechanisms. In contrast, we propose intermediate consistency greedy coordinate gradient (IC-GCG), an approach that extends the GCG framework to explicitly target the model’s internal alignment mechanism. Our method forces a harmful prompt’s hidden representations to mimic those of a semantically related benign counterpart within critical middle layers. By adapting a greedy coordinate optimization strategy to craft an adversarial suffix that minimizes this representational distance, IC-GCG creates “semantic camouflag” that bypasses the model’s internal safety judgments. Experiments on several aligned LLMs indicate that IC-GCG substantially outperforms state-of-the-art baselines, increasing the average attack success rate to 94.3% and proving particularly effective against strongly aligned models such as LLAMA2-7B-CHAT, where it boosts success by over 56 percentage points. Furthermore, the resulting suffixes display superior transferability to other models. Ablation studies confirm that this high efficacy critically depends on a strong semantic link between the harmful and benign prompt pair, empirically validating that the integrity of intermediate representations is a crucial vulnerability and highlighting the need for a new class of defenses that operate directly on the model’s internal states.
Zichu Ren, Donghai Zhu, Haibo Hong et al.· IEEE Internet of Things Jour...· 0 citations
Ensuring the safety of large language models (LLMs) requires rigorous evaluation against adversarial and jailbreaking attempts that can expose harmful behaviors. Malicious prompt generation, while inherently risky, is essential for systematically stress-testing model defenses and uncovering alignment vulnerabilities. Jailbreak attacks against large language models are often studied either through direct prompt rewriting or through search-based optimization over unconstrained prompt variants. We introduce TAPE-JB (Trait-Aligned Prompt Evolution for Jailbreaking LLMs), a genotype-guided evolutionary framework for adversarial prompt generation that searches over structured prompt traits while preserving the original harmful intention. Our method combines iterative prompt rewriting with an alignment-aware fitness function that jointly optimizes attack effectiveness, semantic faithfulness to the source intention, and realization of the intended prompt strategy. Across experiments, TAPE-JB consistently improves over simpler prompting baselines and outperforms CL-GSO on AdvBench, with a 14 percentage-point gain. Our results also show that the generated attacks retain cross-model transferability and that the proposed method remains effective in Polish (76% of attack success rate), demonstrating its applicability beyond the original English benchmark. These findings highlight the value of structured evolutionary search for systematic red-teaming of language models.
Karolina Seweryn, Anna Wróblewska, Szymon Łukasik· Proceedings of the Genetic a...· 0 citations
Experiments across three instruction-tuned models show that HiRoute achieves high safety rates across multiple safety benchmarks while preserving safe-response helpfulness, reducing over-refusal, and maintaining competitive performance on general-purpose tasks.
Fangzhou Chen, Shiji Zhao, Mengyan Wang et al.· 0 citations
Reliable evaluation of tool routing is critical as Large Language Models increasingly operate as autonomous agents. Current benchmarks face three structural limitations: data distributions that follow a power law leave rare scenarios underrepresented; the absence of adversarial hard negatives obscures performance differences across models; and annotation pipelines depend on LLM judgments that have not been validated through execution. In this paper, we introduce PluginEval, a benchmark constructed through a two-stage framework that systematically mitigates these limitations. First, we formulate tool routing as a sequence of three decisions and separate generation from verification. LLMs propose candidate calls, while deterministic validation and real API execution provide reliable quality signals. Second, we decompose each plugin by capability, intent, and boundary to identify trigger and exclusion scenarios. We then generate queries at different difficulty levels to fill coverage gaps, including adversarial negatives targeting three failure modes, and return them to the first stage for annotation. This process creates a closed loop that iterates until coverage converges. For evaluation, we move beyond aggregate accuracy. An LLM judge anchored to gold annotations classifies failures as missed calls, spurious calls, or parameter errors, producing a detailed error profile for each model. We evaluate five model families, including proprietary models and models with open weights, analyze their performance across difficulty levels and error categories, and validate the judge through agreement with human annotations.
While multimodal large language models (MLLMs) extend model capabilities beyond text, they also make safety alignment increasingly challenging. Multimodal safety alignment methods must address cross-modal jailbreaks, safety-awareness failures, and over-sensitive refusals. However, existing methods often rely on retraining or internal-state inspection, limiting their applicability to deployed closed-source MLLMs and motivating test-time safety alignment. We analyze this setting and identify two key obstacles, utility dominance and reasoning inertia, which cause models to overlook latent risks or follow malicious reasoning trajectories. Guided by these insights, we propose ReFrame, a training-free multimodal input reframing framework where two agents share a lightweight locally deployed MLLM: the evidence-generation agent constructs complementary risk and utility evidence, and the rewrite-and-routing agent converts it into a safe proxy prompt and image-routing decision before calling the downstream MLLM, without modifying it or accessing its internal information. Experiments across multiple MLLMs and benchmarks show that ReFrame improves jailbreak defense, safety awareness, and oversensitivity reduction while preserving multimodal utility.
Wenzheng Jiang, Xuankun Rong, Yuanzhao Zhai et al.· 0 citations
A new perspective of information theory is introduced to investigate LVLMs’ transferable characteristics by exploring the relative dependence between outputs of the LVLM model and input adversarial samples and formulate the complicated calculation of information gain as an estimation problem and incorporate such informative constraints into the adversarial learning process.
Xiaowen Cai, Daizong Liu, Xiaoye Qu et al.· Neural Information Processin...· 7 citations