Skip to content

OverThink: Slowdown Attacks on Reasoning LLMs

Feb 2025 · arXiv.org · Vol abs/2502.02542 · 92 citations · ⚡ 9 influential · 111 references
Computer Science

TL;DR

This work evaluates Overthink on proprietary and open-source reasoning models across the FreshQA, SQuAD, and MuSR datasets, and shows that newer generations of RLMs, while showing a drastic increase in per-token cost, also exhibit up to a 2.3x increase in reasoning tokens, leaving them more vulnerable to Overthink attacks.

Abstract

A reasoning language model (RLM) generates costly reasoning tokens, often hidden from the users, that help it excel at many tasks. Our Overthink attack targets RLM-based applications (such as chatbots or coding agents) that rely on external context by forcing these models to generate substantially more reasoning tokens while still producing contextually correct answers. An adversary conducts the attack by injecting decoy reasoning problems into available content, optimized to elicit a large number of tokens. We craft decoy challenges (using Markov decision processes, language translation, or graphic comprehension) that appear benign individually yet have an adversarial impact when inserted in the context, allowing them to easily evade safety filters. We evaluate Overthink on proprietary and open-source reasoning models across the FreshQA, SQuAD, and MuSR datasets, where we observe 13x, 46x, and 12x increases, respectively. We also explore multimodal attacks using images, which cause up to a 2.7x increase in reasoning, as well as attacks on coding agents by injecting decoys into skills, README files, and code, resulting in up to a 17x increase. We explore several defenses and evaluate their efficacy against different attack strategies, highlighting that defending against Overthink is nontrivial. Finally, we show that newer generations of RLMs, while showing a drastic increase in per-token cost, also exhibit up to a 2.3x increase in reasoning tokens, leaving them more vulnerable to Overthink attacks.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Distillation Defenses Easily Break After Reinforcement Learning

It is argued that a more realistic threat model includes further training with reinforcement learning after distillation, and it is shown that simple attacks can steal reasoning capabilities from existing closed-source language models using data easily obtainable from current APIs, yielding reasoning improvements equiv...

S. Javaheri, Alexander Panfilov, O. Britton et al. · 0 citations
#natural language process... Preprint Oct 2026

Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models

Red-TTT is proposed, which updates the attacker's parameters during the attack on each behavior, and requires only sampling access to the victim and integrates into existing attack pipelines with no other changes, increasing attack success rate and cracking many behaviors previous method cannot.

Tong-Yan Hu, Hao Li, Xiaogeng Liu et al. · 0 citations

Decoding Guardrails: XAI-Guided Perturbation Analysis of Prompt Injection Detection

An exploratory case study that applies explainable artificial intelligence techniques to analyze how Prompt Guard 2 distinguishes malicious from benign prompts finds that Prompt Guard 2's decisions rely on the cumulative contribution of many tokens rather than a few dominant ones, yet saliency-guided synonym substituti...

Fernando Outeda, Gustavo Betarte, J. Campo et al. · 0 citations
#artificial intelligence Review Sep 2026

Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs

This is the first systematic, controlled study that isolates the scratchpad reasoning channel as an output-prefix attack vector, and the first to compare reasoning-only, output-prefix-only and reasoning-plus-output-prefix attacks across both exposed- and hidden-reasoning models.

Lukáš Brůna, Robert A. Bridges, Adam Ek · 0 citations
#artificial intelligence Preprint Sep 2026

SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration

A Systematization of Knowledge (SoK) that reframes jailbreak security around the full agentic execution pipeline, and develops unified taxonomies of attacks and defenses spanning user interaction, planning and reasoning, memory, tool use, and inter-agent communication, introduces a security--utility--efficiency evaluat...

Md. Jueal Mia, Yanzhao Wu, S. Uluagac et al. · 0 citations
#natural language process... Preprint Oct 2026

Reward Stealing Attack on Large Language Models

Experiments demonstrate that a single recovered reward generalizes across prompts and diverse models to reveal a fundamental alignment vulnerability, enabling ReSA to significantly outperform existing attacks in effectiveness and transferability.

Jia-Ming Qian, Peng-Yang Zhou, Jia-He Xu et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 3, 2026

Learn the rules: The hacker mindset

Research Manager Mike Walker reflects on a career spanning hacker culture, cybersecurity, and AI, sharing lessons about trust, teamwork, leadership, and why solving difficult problems requires curiosity, judgment, agility—and an understanding of the rules. The post Learn the rules: The hacker mindset appeared first on Microsoft Research.

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.