A two-dimensional design space is introduced in which both dimensions are organised into five operational levels, making the coupling explicit and navigable, and six architectural tactics for adjusting a deployment’s position within it are proposed, offering a shared vocabulary for compliance-aware agentic AI design.
Abstract
Deploying agentic AI in regulated contexts requires knowing two things about a deployment: what the system can do—its agency—and how much it acts without human involvement— its autonomy. Though often treated independently, the two are coupled: at higher autonomy, human error correction is less available, so reliable operation requires constraining agency accordingly, and compliance rules reinforce this by mandating human involvement as the consequences of actions grow. Yet no established approach addresses them jointly as a design problem, leaving practitioners without a principled basis for deciding where oversight should sit and how errors can be caught before they propagate. We introduce a two-dimensional design space in which both dimensions are organised into five operational levels, making the coupling explicit and navigable, and we propose six architectural tactics—checkpoints, escalation, multi-agent delegation, tool provisioning, tool fencing, and write staging—for adjusting a deployment’s position within it. We ground the tactics in a public-sector document classification system, tracing a path from manual operation to near-full autonomy under realistic compliance constraints. Together they offer a shared vocabulary for compliance-aware agentic AI design in which responsibility, auditability, and reversibility are explicit design choices rather than retrofitted properties.
This primer draws on fieldwork in a computational biology laboratory to examine what human oversight of AI agents requires in practice and shows that effective oversight has four components: adequate knowledge of system capabilities and limitations, sufficient observation of system actions, meaningful control of system behavior, and timely intervention in system failures.
It is argued that governing such agents is a runtime problem -- not a model-alignment problem and not a build-time problem -- and five primitives are derived from the questions that must be answered before an action takes effect and after it has: discovery, identity, governance, attestation, and supply chain.
A growing literature on “agentic AI” — autonomous software agents that plan and execute multi-step actions on a principal’s behalf — has revived the thesis that such systems open a responsibility gap: because the deploying principal neither intends, foresees, nor controls the specific actions an autonomous agent selects, no human can be held fully responsible for resulting harms. This paper argues that the responsibility-gap thesis, as applied to principal-deployed AI agents, rests on a conflation called here the delegation illusion: the inference from a principal’s causal and epistemic remoteness from an outcome to a diminution of her answerability for it. Distinguishing attributability, answerability, and accountability, the paper argues that autonomy, opacity, and adaptivity bear on attributability while leaving intact an answerability grounded, through a standard tracing structure, in the deployer’s guidance control over the prior act of delegation. It defends a principle of responsibility conservation under delegation and operationalises that principle’s central criterion — foreseeability of harmful dispositions at the level of types — by specifying an epistemic base, a standard of diligence, and a constraint on type individuation that blocks vacuous redescription. Answerability is argued to be epistemically graded, but indexed to the deployer’s ex ante access to the dispositional profile of the deployment specification rather than to the particular act. The method is conceptual analysis and normative argumentation; the scope and limitations of the claims are stated explicitly.
Persistent AI assistants are intended to extend human attention, memory, and coordination across changing digital and physical environments. To be truly useful they must do more than just act when asked. They must decide on their own whether a situation warrants behavior at all, when it does and in what mode, whether to act, ask, monitor, defer or deliberately refrain. We call this the activation problem. Research on commitment, appraisal, mixed-initiative interaction and delegation each illuminates part of it, but none ties situated activation to continuing authorization and accountability. This paper develops a conceptual and formal framework for governed proactive agency, organizing behavior across time through perception, intent, affective-conative appraisal, constraint, and feedback. It distinguishes autonomous and delegated agency and defines symbiotic agency as delegation under a standing, revocable mandate, with continuing coupling to the principal's situation, calibrated inference of their condition, and bounded personalization. The distinctive contribution is an integrated account linking activation decisions to authorized perception, behavior selection, authority containment, traceable restraint, and constrained adaptation, with behavioral episodes as the unit of analysis. Through an agency classification method, an evaluation framework, proposed benchmark scenarios, and a reference architecture, the account provides a basis for specifying and assessing whether assistance is warranted, timely, authorized, and answerable beyond task completion alone. It is intended to guide the development and evaluation of always-present personal assistants and embodied support systems that augment human capabilities while preserving the principal's authority and judgment.
This is the first paper from POLIS, an ongoing research programme studying algorithmic institutions for multi-agent systems, and asks which parts of an AI institution produce safety and how they do it.
Charting these challenges provides a roadmap toward trustworthy autonomous agent deployment: security must become a verifiable property of the architectures, protocols, and runtimes that govern agent behavior, rather than an optional layer of guidance.
