PRIMA: Enabling User Agency and Control in Mobile GUI Agent Autonomy

Abstract

Advances in multimodal large language models have enabled mobile GUI agents that can interpret screen content and autonomously perform tasks across applications. While prior research has largely focused on task completion and agent autonomy, less attention has been paid to how users can remain involved in the delegation process and shape agent behavior. We present PRIMA (PReference-Informed Mobile Agent), a mobile GUI agent that supports user agency through six mechanisms: specifying preferences, defining execution steps, establishing approval conditions, clarifying requests, reviewing and revising generated plans, and allowing user interference. We further report findings from a preliminary field deployment. Results suggest that participants actively established delegation boundaries, developed trust through accumulated experiences, learned the agent’s capabilities and limitations through use, and adapted the agent through preferences. Participants also valued clarification when it reduced ambiguity but preferred it to be used selectively. These findings provide insights for more user-aligned mobile GUI agents.