Skip to content

FENCE: A Financial and Multimodal Jailbreak Detection Dataset

Mirae Kim Seonghun Jeong Youngjun Kwak
Aug 2026
Artificial Intelligence Natural Language Processing

Abstract

Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vulnerable because they process both text and images, creating broader attack surfaces. However, available resources for jailbreak detection are scarce, particularly in finance. To address this gap, we present FENCE, a bilingual (Korean-English) multimodal dataset for training and evaluating jailbreak detectors in financial applications. FENCE emphasizes domain realism through finance-relevant queries paired with image-grounded threats. Experiments with commercial and open-source VLMs reveal consistent vulnerabilities, with GPT-4o showing measurable attack success rates and open-source models displaying greater exposure. A baseline detector trained on FENCE achieves 99 percent in-distribution accuracy and maintains strong performance on external benchmarks, underscoring the dataset's robustness for training reliable detection models. FENCE provides a focused resource for advancing multimodal jailbreak detection in finance and for supporting safer, more reliable AI systems in sensitive domains. Warning: This paper includes example data that may be offensive.

View source

Similar papers

#artificial intelligence Review Dec 2025

Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025

Investigating how experienced developers use agents in building software, including their motivations, strategies, task suitability, and sentiments finds that while experienced developers value agents as a productivity boost, they retain their agency in software design and implementation out of insistence on fundamental software quality attributes.

Ruanqianqian Huang, Avery Reyna, Sorin Lerner et al. · 19 citations · ⚡1
#artificial intelligence Open access Oct 2022

Adaptive surrogate modeling for high-dimensional spatio-temporal output

An adaptive surrogate modeling method for problems with very high-dimensional spatio-temporal outputs is developed that combines exploration and exploitation to improve the surrogate model accuracy with the fewest possible runs of the expensive physics-based model.

B. Kapusuzoglu, S. Mahadevan, Shunsaku Matsumoto et al. · 17 citations
#artificial intelligence Preprint Feb 2025

`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMs

An adaptive jailbreak attack framework for systematic evaluation of both cascaded pipelines and end-to-end large audio-language models under a unified experimental setting that achieves consistently higher attack success rates across diverse audio-based LLM systems.

Linghan Huang, Bo Li, Huaming Chen et al. · 12 citations · ⚡2
#artificial intelligence Review Open access Oct 2025

Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead

This review provides a systematic literature review of LLM-based Verilog code generation, analyzing 102 papers (70 published and 32 high-quality preprints) from SE, AI, and EDA venues and outlines a roadmap highlighting potential opportunities in LLM-assisted hardware design.

Guang Yang, Wei Zheng, Xiang Chen et al. · 11 citations · ⚡1

From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?

This work introduces Behavior-Outcome Freedom (F), a pre-synthesis diagnostic of signed behavior-outcome rank mismatch, and formalizes its candidate-conditional role through Signed Anchor-Rank Transfer, which preserves validated capability resources, removes runtime orchestration, and conditionally inherits pipeline guidance using a calibrated rule over F.

Binyan Xu, Dong Fang, Haitao Li et al. · 10 citations

Diffusion Models for Smarter UAVs: Decision-Making and Modeling

Simulation results confirm the effectiveness and benefits of DMs in generating neighbor velocity estimates in a four-UAV swarm coordination task using Deep Reinforcement Learning (DRL), and explore the integration of DMs with RL and DT.

Yousef Emami, Hao Zhou, Luís Almeida et al. · 9 citations

Related blog posts

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.