Skip to content

Safe to Resume? Breaking Execution Continuity of Agent Execution via Rollback

Guanlong Wu Dahui Li Ke Jiang Jianyu Niu Cong Wang Yinqian Zhang
Sep 2026
Artificial Intelligence Cybersecurity

Abstract

AI agents are moving toward persistent, stateful execution across various applications, accumulating execution state and external effects that are costly to reconstruct after failures. Checkpoint and rollback (C/R) are becoming essential for recovery, yet their security implications remain largely unexplored. Correct rollback does not imply secure recovery: a faithfully restored checkpoint may resume an execution whose states, assumptions, and external effects never coexisted in any valid history. In this paper, we present the first systematic security study of checkpoint and rollback in existing agent systems. By examining representative agent C/R systems, we characterize the design space of existing C/R mechanisms and develop a general execution model that captures their recovery boundaries and state dependencies. From this model, we identify five fundamental failure modes spanning incomplete or inconsistent internal state, stale external dependencies, nondeterministic replay, and unrecorded external effects. We further demonstrate their security impact through three end-to-end attacks on Hermes, Cline, and LangGraph, enabling malware-verification bypass, unauthorized mail forwarding, and double payment. To systematically study these failures in practice, we develop a multi-agent analysis pipeline that reconstructs execution semantics, identifies violations of the five failure conditions, and validates them through actual rollback. Across five representative frameworks, our evaluation shows that these failures recur across heterogeneous C/R designs and stem from a common gap between the state restored by a checkpoint and the dependencies required for secure continuation.

View source

Similar papers

#artificial intelligence Open access May 2023

Evaluating the Performance of Large Language Models on GAOKAO Benchmark

GAOKAO-Bench is introduced, an intuitive benchmark that employs questions from the Chinese GAOKAO examination as test samples, including both subjective and objective questions that contribute a robust evaluation benchmark for future large language models and offers valuable insights into the advantages and limitations of such models.

Xiaotian Zhang, Chun-yan Li, Yi Zong et al. · 216 citations · ⚡17
#artificial intelligence Conference Open access Apr 2020

ECCOLA - a Method for Implementing Ethically Aligned AI Systems

Various recent Artificial Intelligence (AI) system failures, some of which have made the global headlines, have highlighted issues in these systems. These failures have resulted in calls for more ethical AI systems that better take into account their effects on various stakeholders. However, implementing AI ethics into practice is still an on-going challenge. High-level guidelines for doing so exist, devised by governments and private organizations alike, but lack practicality for developers. To address this issue, in this paper, we present a method for implementing AI ethics. The method, ECCOLA, has been iteratively developed using a cyclical action design research approach. The method aims at making the high-level AI ethics principles more practical, making it possible for developers to more easily implement them in practice.

Ville Vakkuri, Kai-Kristian Kemell, P. Abrahamsson · 64 citations · ⚡6

Ethically Aligned Design of Autonomous Systems: Industry viewpoint and an empirical study

Progress in the field of artificial intelligence has been accelerating rapidly in the past two decades. Various autonomous systems from purely digital ones to autonomous vehicles are being developed and deployed out on the field. As these systems exert a growing impact on society, ethics in relation to artificial intelligence and autonomous systems have recently seen growing attention among the academia. However, the current literature on the topic has focused almost exclusively on theory and more specifically on conceptualization in the area. To widen the body of knowledge in the area, we conduct an empirical study on the current state of practice in artificial intelligence ethics. We do so by means of a multiple case study of five case companies, the results of which indicate a gap between research and practice in the area. Based on our findings we propose ways to tackle the gap.

Ville Vakkuri, Kai-Kristian Kemell, Joni Kultanen et al. · 56 citations · ⚡6
#artificial intelligence Conference Open access Jun 2018

The Key Concepts of Ethics of Artificial Intelligence

The growing influence and decision-making capacities of Autonomous systems and Artificial Intelligence in our lives force us to consider the values embedded in these systems. But how ethics should be implemented into these systems? In this study, the solution is seen on philosophical conceptualization as a framework to form practical implementation model for ethics of AI. To take the first steps on conceptualization main concepts used on the field needs to be identified. A keyword based Systematic Mapping Study (SMS) on the keywords used in AI and ethics was conducted to help in identifying, defying and comparing main concepts used in current AI ethics discourse. Out of 1062 papers retrieved SMS discovered 37 re-occurring keywords in 83 academic papers. We suggest that the focus on finding keywords is the first step in guiding and providing direction for future research in the AI ethics field.

Ville Vakkuri, P. Abrahamsson · 39 citations · ⚡2

AI Ethics in Industry: A Research Framework

Artificial Intelligence (AI) systems exert a growing influence on our society. As they become more ubiquitous, their potential negative impacts also become evident through various real-world incidents. Following such early incidents, academic and public discussion on AI ethics has highlighted the need for implementing ethics in AI system development. However, little currently exists in the way of frameworks for understanding the practical implementation of AI ethics. In this paper, we discuss a research framework for implementing AI ethics in industrial settings. The framework presents a starting point for empirical studies into AI ethics but is still being developed further based on its practical utilization.

Ville Vakkuri, Kai-Kristian Kemell, P. Abrahamsson · 27 citations · ⚡3
#artificial intelligence Book Open access Apr 2020

A Multiple Case Study of Artificial Intelligent System Development in Industry

There is a rapidly increasing amount of Artificial Intelligence (AI) systems developed in recent years, with much expectation on its capacity of innovation and business value generation. However, the promised value of AI systems in specific business contexts might not be understood, and further integrated into the development processes. We wanted to understand how software engineering processes and practices can be applied to develop AI systems in a fast-faced, business-driven manner. As the first step, we explored contextual factors of AI development and the connections between AI developments to business opportunities. We conducted 12 semi-structured interviews in seven companies in Brazil, Norway and Southeast Asia. Our investigation revealed different types of AI systems and different AI development approaches. However, it is common that business opportunities involving with AI systems are not validated and there is lack of business-driven metrics that guide the development of AI systems. The findings have implications for future research on business-driven AI development and supporting tools and practices.

Anh Nguyen-Duc, Ingrid Sundbø, E. Nascimento et al. · 20 citations · ⚡1

Related blog posts