The Ownership Cost of AI Goal-Setting: Empirical Evidence for a Quality-Motivation Dissociation and Differential Harm to Vulnerable Users (Extended Abstract)
Oct 2026· Proceedings of the AAAI/ACM Conference on AI Ethics and Society
Ethics and Social Impacts of AI
Abstract
AI systems are now deployed at scale to author personal goals on users' behalf in coaching platforms and productivity tools, a practice justified by the assumption that better-structured outputs produce better outcomes. We provide a preregistered experimental test of this assumption in personal goal-setting, a domain where outputs must drive sustained behavior rather than merely be delivered. In a between-subjects experiment (N = 470), LLM-authored goals scored dramatically higher on objective SMART criteria (d = 2.26), yet participants reported lower psychological ownership (d = 1.38), commitment (d = 1.19), and perceived importance (d = 1.13). At two-week follow-up, 72.8% of self-authoring participants had acted on two or more goals, versus 46.6% in the LLM condition, a behavioral gap that goal quality cannot explain. Mediation analyses identify psychological ownership, not structural quality, as the mechanism behind every motivational and behavioral outcome. The findings reveal an equity paradox: the ownership costs are largest for users lowest in self-efficacy, the population most likely to seek AI assistance. Optimizing for output quality in identity-relevant, behavior-dependent tasks can silently degrade human Ownership and widen inequity. Authorship preservation is an ethical requirement for AI tools deployed in self-regulatory and wellbeing contexts.
The method, ECCOLA, is presented, which aims at making the high-level AI ethics principles more practical, making it possible for developers to more easily implement them in practice.
Ville Vakkuri, Kai-Kristian Kemell, P. Abrahamsson· EUROMICRO Conference on Soft...· 64 citations· ⚡6
The goal is to not only refine the accuracy of the LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education.
Z. Rasheed, Malik Abdul Sami, Muhammad Waseem et al.· arXiv.org· 62 citations· ⚡3
The use of large language models to automatically improve the user story quality in Austrian Post Group IT agile teams is explored, with a reference model for an Autonomous LLM-based Agent System developed and implemented at the company.
Zheying Zhang, M. Rayhan, Tomas Herda et al.· International Conference on...· 48 citations· ⚡4
This paper introduces a novel multi-AI-agent system designed to fully automate SLRs, and demonstrates how it substantially reduces the time and effort traditionally required for SLRs while maintaining comprehensiveness and precision.
Abdul Malik Sami, Z. Rasheed, Kai-Kristian Kemell et al.· arXiv.org· 44 citations· ⚡2
The proposed LLM-based multi-agent system automates qualitative data analysis process, creating opportunities for researchers and practitioners, and future improvements focus on enhancing multilingual performance and integrating continuous expert feedback.
Z. Rasheed, Muhammad Waseem, Aakash Ahmad et al.· arXiv.org· 41 citations
A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.
MIT News · Artificial Intelligence· news.mit.eduOct 15, 2026