Instrumental succession: the gradual transfer of agency to AI
The instrumental succession thesis is introduced: that human controllers of powerful AI systems pursue a set of instrumental dispositions that progressively increase the AI’s capabilities and lead to the AI exercising an increasing share of oversight and control over key decisions and processes, resulting in the gradua...