Computation-bandwidth-memory trade-offs: a unified paradigm for AI infrastructure

Abstract

Abstract Large-scale artificial intelligence (AI) models are fundamentally transforming industries and redefining the paradigm of human–machine collaboration. While the technological revolution signals a new era of machine intelligence, the continued scaling of these models has exposed significant limitations in contemporary hardware architectures, manifesting as constraints on computational efficiency, interconnection bandwidth, and memory capacity. These three dimensions are inseparably intertwined, such that advances along any single axis often exacerbate bottlenecks in the others, rendering isolated optimizations increasingly ineffective. Achieving an optimal balance among them to maximize system efficiency therefore remains a central challenge in the design of scalable AI systems. To address this challenge, we introduce Computation-Bandwidth-Memory Trade-offs, termed the AI Trinity, a unified paradigm that positions computation , bandwidth , and memory as coequal pillars for next-generation AI infrastructure. Inspired by the device-edge-cloud collaboration principle from the AI Flow framework, we formulate AI Trinity as a resource-theoretic view of the computation-bandwidth-memory bottlenecks in distributed AI systems. Within this framework, AI Trinity identifies three fundamental trade-offs: (1) More Computation $$\rightarrow$$ → Less Bandwidth, wherein computational resources are exploited to reduce data transmission under limited bandwidth conditions, (2) More Bandwidth $$\rightarrow$$ → Less Memory, which exploits abundant communication capacity to populate or refresh memory when local storage resources are constrained, and (3) More Memory $$\rightarrow$$ → Less Computation, whereby storage capacity are utilized to mitigate redundant computation when computational costs are prohibitive. We illustrate its effectiveness through representative system designs spanning edge–cloud communication, large-scale distributed training, and model inference. The innovations embodied in AI Trinity advance a new paradigm for scalable AI infrastructure, providing both a conceptual foundation and practical guidance for a broad range of application scenarios.