MedEvoEval: Evaluating Continual Evolution of Doctor Agents through Simulated Clinical Episodes

Abstract

Doctor agents are moving beyond single-turn answer generation toward evolving clinical decision systems that acquire evidence within episodes and reuse experience across episodes. Fixed-input medical QA and many interactive benchmarks provide limited support for evaluating how episode-level decisions interact with cross-episode experience. We introduce MedEvoEval, an executable longitudinal evaluation framework based on action-gated simulated outpatient episodes. Each case is converted into role-specific patient, examination, and manager views; evidence is revealed through valid actions, and structured traces link observations, actions, final outputs, scores, and optional experience write-back. Experiments show that these traces expose process costs hidden by final-answer scoring, characterize how MDT-style consultation reallocates resources, and support analyses of memory maturation, held-out transfer, update-stage response, and backward retention. On 200 held-out cases, the mature-memory condition with sequential write-back reduces interaction turns by 0.915 and improves composite utility by 0.0483 after Holm correction. MedEvoEval provides a concrete basis for evaluating experience-driven improvement, transfer, and retention. The accompanying artifact provides 700 processed episode records with provenance, role-specific schemas, and execution, scoring, and analysis code.