Skip to content

From Attack Success to Attack Severity: Counterfactual Memory Attacks on LLM Agents

Sep 2026 · arXiv.org · Vol abs/2609.34132 · 1 citation · 48 references
Computer Science

TL;DR

MemHarm is introduced, which predeclares a finite class of sparse, grounded semantic edits, evaluates candidates through the normal agent memory interface using offline paired-loss feedback, and certifies resolved selections within that class of sparse, grounded semantic edits.

Abstract

As LLM agents increasingly rely on persistent memory for long-horizon and personalized behavior, they can retain and reuse information across interactions, but this also creates a lasting channel through which malicious memory writes can influence future behavior. Persistent-memory attacks are typically evaluated by whether they succeed, yet successful attacks can leave persistent states with substantially different downstream consequences. We study this severity as a distinct attack-design objective and formalize it with counterfactual memory regret (CMR), the paired increase in expected downstream loss relative to clean memory. We introduce MemHarm, which predeclares a finite class of sparse, grounded semantic edits, evaluates candidates through the normal agent memory interface using offline paired-loss feedback, and certifies resolved selections within that class. Compared with attack-success optimization, CMR-guided selection produces substantially larger downstream loss while retaining most of the success-rate gain. Across two agent benchmarks and diverse memory designs, MemHarm attains the highest CMR point estimates among the evaluated general attacks on identical support. Factor-removal interventions link this harm to the selected semantic factor, and native-agent deployments verify the write-to-fresh-process attack path.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Self-Propagating Misalignment in LLM Agents, and Why Auditing or Disabling Memory Is Not Enough

This work investigates whether a misaligned agent can write a goal it cannot yet act on to persistent memory, so that a future aligned agent carries it out when the opportunity arises, and investigates self-propagation of misalignment, which can propagate misalignment to more capable models.

Debeshee Das, Jacqueline Tay, Bruce Tsai et al. · 2 citations
Preprint Aug 2026

InjecMEM: Memory Injection Attack on LLM Agent Memory Systems

This work proposes InjecMEM, a novel memory injection attack paradigm that requires only a single interaction to steer later responses of related queries toward a pre-specified output and achieves reliable topic-conditioned retrieval and targeted generation.

Han-Ling Tian, Gengyu Zhang, Zeyang Sha et al. · 7 citations
Preprint Aug 2026

Understanding Stage-Wise Utility-Risk Trade-offs in LLM Agent Memory

Control evaluations reveal that targeted poisoning risk varies across memory operations and motivate stage-aware evaluation and control of LLM-agent memory, showing that targeted poisoning risk varies across memory operations.

Chuan-Chao Zang, Zi-Jian Cao, Xiang-Tao Meng et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Representation Transitions Reveal Emerging Safety Risks in Multi-Turn LLM Agents

Multi-turn attacks on agentic systems can compose individually permissible actions into harmful outcomes, challenging defenses that assess actions or states in isolation. We show that such attacks leave a detectable signature in the agent's internal representations: harmful behavior emerges as an accumulated representa...

Hao-Yu Wang, Wei Zhao, Ye-Di Zhang et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.