TellOnce: Correcting Coding Agents Once, Improving Their Future Actions

Abstract

Interactive LLM agents increasingly work with users across tasks, yet remembering a previous correction does not ensure that the agent changes its behavior in the future. In a diagnostic constructed from anonymized real-world agent interactions, agents using Mem0 still violate 57.5% of applicable preference checks. We introduce TellOnce, a drop-in personalization layer that turns conversational corrections into persistent, scoped, and revisable behavioral rules without updating the underlying agent model. TellOnce identifies which requirements should persist, determines when they apply, reconciles subsequent feedback with existing rules, and prepares reusable checks when reliable verification is possible. We evaluate whether learned corrections improve future behavior by freezing the learned state and testing on subsequent tasks without additional user corrections. Across three models on ClawArena and MemoryArena, TellOnce achieves the lowest preference violation rate in all six settings, reducing violations relative to PGMem by 19.3-42.3 percentage points on ClawArena and 3.0-22.9 points on MemoryArena, while maintaining broadly comparable task performance. On SWE-Together, TellOnce further improves task quality on new coding tasks by learning from feedback received during earlier tasks. Experiment code is available at https://github.com/YujunZhou/TellOnce_exp, and the deployable skill at https://github.com/YujunZhou/tellonce.