From model trust to software topology: a Perspective on structurally governed agentic AI systems
TL;DR
This Perspective argues that agentic AI safety should move beyond exclusive reliance on model trust, alignment, or prompt-level defenses and should be treated as a software-architecture problem and developed a five-plane reference topology comprising intent, orchestration, execution, oversight, and adaptation.
Abstract
Large language models are increasingly embedded in agentic software systems that retrieve information, plan subtasks, call tools, modify code, execute tests, and adapt through memory or feedback. This transition changes the safety problem: when a model acts through software tools, failure can alter external state. This Perspective argues that agentic AI safety should move beyond exclusive reliance on model trust, alignment, or prompt-level defenses and should also be treated as a software-architecture problem. We propose software topology as a design lens for structurally governed autonomy and develop a five-plane reference topology comprising intent, orchestration, execution, oversight, and adaptation. Its core invariant is that components that decide should not directly act, and components that act should not act without supervision. The topology is realized in Polos, an open reference specification that encodes role contracts, message schemas, declared flow graphs, bounded loops, governed self-improvement, and validator-checked invariants. We argue that this approach complements model alignment and empirical red teaming by providing an inspectable substrate for evaluating, constraining, and governing tool-using AI agents.
