Skip to content

DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis

Sep 2026 · 0 citations · 30 references
Computer Science

TL;DR

DashAct is introduced, to the authors' knowledge the first benchmark to diagnose failures at a fine-grained level within the same dashboard task, and measures the minimum support an agent needs to recover rather than scoring isolated skills.

Abstract

Interactive dashboards require users to reveal and connect evidence across stateful interactions. Although graphical user interface (GUI) agents could automate this process, existing dashboard benchmarks primarily report final answers or task success. They provide limited insight into whether failures arise from maintaining the analytical process, selecting actions, or grounding visual targets. We introduce DashAct, to our knowledge the first benchmark to diagnose these failures at a fine-grained level within the same dashboard task. DashAct contains 357 human-verified interaction trajectories with milestone dependencies and hierarchical target annotations. Its progressive diagnostic cascade evaluates end-to-end execution, restores verified context for next-action prediction, and provides target semantics and a local view for visual grounding. By progressively restoring the conditions for success, DashAct measures the minimum support an agent needs to recover rather than scoring isolated skills. Experiments show that current models struggle even as support is added. The cascade outcomes reveal bottlenecks hidden by end-to-end scores and provide actionable guidance for improving GUI agents.

View source

Similar papers

Preprint Aug 2026

Reflection with Action-Induced Visual Differences for Desktop GUI Agents

Evidence-First Reflection (EFR), a two-stage reflector that explicitly decouples action-induced visual differences extraction from outcome verification, makes reflection better grounded in screen transitions, while reducing both visual search complexity and reasoning burden.

Yijie Ma, Chao-Yue Niu, Fan Wu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions

Understanding where and why Graphical User Interface (GUI) agents fail is essential for building more reliable systems, yet current evaluation relies on step accuracy, a metric that treats each screen independently and overlooks the underlying structure of GUI environments. This leads to two critical blind spots: (1) f...

Shao-Qing Zhang, Ke-Hai Chen, Xue-Feng Bai et al. · 0 citations

WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics

This work introduces WidgetVA, a widget-centric agentic VA framework that standardizes interactive components as structured widgets with unified action and perception-query APIs and presents WidgetVABench, a benchmark of single- and multi-widget VA tasks that require agents to perform multi-step interactions to uncover...

Yu-Tong Chen, Zhi-Ke Tang, Zhi-Hao Mai et al. · 0 citations
Review Aug 2026

MUSE: An Interactive Meta-Agent for Understanding and Steering LLM-powered Data Science Systems

MUSE is presented, an interactive meta-agent that enhances user understanding and control of agentic data science systems by dynamically restructuring low-level execution traces into multiple semantic levels that support navigation from high-level overviews to low-level implementation details.

Wei-Hao Chen, Weixi Tong, Yuan Tian et al. · 0 citations
#artificial intelligence Preprint Aug 2026

UI-Venus-2 Technical Report

UI-Venus-2 is presented, a general-purpose foundation GUI agent designed to operate across mobile, web, and desktop environments through a unified closed-loop reasoning-action framework that integrates safety-aware mechanisms to ensure controlled execution of consequential actions.

Venus Team, Zhuo-Hang Cai, Hao-Xin Chen et al. · 4 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.