Humans in a Loop: Ethical Agency and Speculative Pathways in UX Practice Within the Generative AI Maelstrom
TL;DR
It is argued that expecting practitioners to bear ethical accountability for outcomes they cannot meaningfully influence is increasingly untenable, yet the practice of thinking otherwise—constructing alternative realities and engaging in self-authorship—remains available even when structural power is not.
Abstract
As generative AI (genAI) systems continue to be swiftly developed and widely deployed, UX practitioners—the humans responsible, in part, for designing people’s encounters with these systems—find themselves navigating complex ethical terrain with diminishing structural support. This research builds on prior work on the ethical dimensions of UX practice to investigate how UX practitioners experience ethical agency while working on genAI products in the current socioeconomic climate. Using a qualitative, interpretive design with an autoethnographic orientation, the study draws on the author’s own experience as a former UX practitioner and AI ethics team member at Microsoft, alongside semi-structured interviews with nine practitioners who had experienced values misalignment in their work on genAI. Interview data were analyzed through reflexive thematic analysis and complemented by a method of speculative microfictions: 175-word stories used as a reflexive analytic tool. Three themes emerged: participants viewed genAI with stronger ambivalence than opposition, recognizing benefits while fearing harms to cognition, labour, and well-being; they felt significantly ethically disempowered in their UX work due to structural barriers, the absence of ethical discussion in the workplace, and genAI-induced moral distress; and they employed “lean in” and “lean out” tactics to exert agency, oscillating between pessimism and cautious optimism. Framed through the metaphor of loops—cognitive cycles of dissonant beliefs and the sensation of going in circles while caught in the genAI maelstrom—this thesis argues that expecting practitioners to bear ethical accountability for outcomes they cannot meaningfully influence is increasingly untenable. Yet the practice of thinking otherwise—constructing alternative realities and engaging in self-authorship—remains available even when structural power is not. These threads culminate in a design prototype: HUMAN IN A LOOP: A UX Ethics Choose-Your-Own-Adventure Book, a branching narrative inviting readers to explore and reflect on ethical frictions and co-construct potential paths forward.
