May 2026· arXiv.org· Vol abs/2605.31349· 0 citations· 39 references
Computer Science
Abstract
Hateful meme detection remains a formidable challenge for vision-language models, as existing benchmarks are structurally observational - confounding rhetorical hate mechanisms with target community features and preventing causal evaluation of model vulnerabilities. To address this, we introduce FBHM, a systematically curated benchmark of Functionality Based Hateful Memes constructed along two orthogonal axes: 25 distinct rhetorical functionalities and 10 target communities (5,000 memes total). Benchmarking state-of-the-art VLMs reveals a severe generalization gap: models highly accurate on standard datasets catastrophically drop to near-random performance on FBHM, proving they exploit dataset-specific heuristics rather than robust multimodal reasoning. To efficiently close this gap, we propose LSV (learnable steering vectors), an ultra-low data regime strategy that applies a causal intervention objective on as few as 500 steering samples (50 unique base memes), boosting FBHM performance by ~30 Macro-F1 points while outperforming in-context learning and PEFT without degrading source-domain performance.
This work proposes an explanation-aware safety framework that augments binary harmfulness detection with structured, human-interpretable explanations capturing severity, strategies, trigger spans, ratio-nales, and derived safety factors, and introduces a human–LLM hybrid annotation and canonicaliza-tion pipeline.
Sunghee Dong, Sungwon Yi, K. Bae et al.· Proceedings of the Thirty-Fi...· 0 citations
MMJailBench is introduced, a factorized benchmark that systematically varies and combines these factors under controlled configurations, enabling fine-grained comparison and factor-level attribution and a modular multimodal jailbreak evaluation suite with full and lightweight configurations, multiple judge options, and...
Tian-Shi Wang, Jing-Song Wang, Ya-Fei Huang et al.· 0 citations
Large language models have achieved remarkable capabilities across diverse domains, yet their safety alignment remains vulnerable to jailbreak attacks. In this work, we identify a previously underexplored failure mode - safety generalization lag - where alignment trained predominantly on natural language fails to trans...
Current LLM detectors remain vulnerable to adversarial attacks because they focus on superficial statistics of the text. In this study, the robustness of state-of-the-art (SOTA) models was tested on a hierarchical attack dataset of 24,000 examples generated using T5, BART, and PEGASUS, based on the PAN 2025 dataset. An...
F. Özgür, Taner Tuncer, ´brahim Rıza Hallaç et al.· Automation, Control, and Inf...· 0 citations
A mechanistic analysis of the jailbreaking behavior in a large-scale, safety-aligned LLM, focusing on LLaMA-2-7B-chat-hf identifies computational circuits responsible for generating affirmative responses to jailbreak prompts and uncovers key attention heads and MLP pathways that mediate adversarial prompt exploitation.
Paria Mehrbod, Boris Knyazev, Guy Wolf et al.· 0 citations
Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model systems. Existing datasets use different labels, response formats, and metadata conventions, making detector results difficult to compare. We present RH-Detect, a benchm...
Jun-Wei Quan, E. Opryshko, Rohan Subramani et al.· 0 citations
