Autonomous programming agents are commonly evaluated by whether they solve software tasks, providing limited insight into the execution processes that lead to success or failure. We present a process-centric approach to evaluating agent behavior through trajectories. We introduce Graphectory, which captures the structu...
Shu-Yang Liu· Proceedings of the 41st IEEE...· 0 citations
We present Graphectory Viewer, a web-based tool for interactive, process-centric analysis of software-agent trajectories. Building on the Graphectory representation introduced in our previous work, Graphectory Viewer transforms heterogeneous raw trajectories into phase-aware graphs that connect low-level execution deta...
Charlie Jyu, Shu-Yang Liu, Reyhaneh Jabbarvand· Proceedings of the 41st IEEE...· 2 citations
LivePlan decouples judging from advising: a deterministic, rule-based monitor examines general signals over the trajectory to detect issues without invoking an LLM, and only when an issue is detected does it consult an advisor LLM for a high-level, next-step correction.
Shuyang Liu, Saman Dehghan, Ji Young Kim et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
