Beyond Monoliths: Enabling Flexible and Composable AI Systems via Memory Disaggregation

Abstract

Modern state-of-the-art AI systems are increasingly built as monolithic supernodes integrating large numbers of specialized accelerators with proprietary high-bandwidth interconnects. These systems provision compute, memory, and networking resources in fixed ratios at design time. As AI workloads evolve, their resource demands increasingly diverge from these static configurations, leading to underutilization, limited scalability, and high operational cost. Composable systems based on disaggregated resources offer a more flexible alternative by allowing memory and compute capacity to be scaled independently without replicating an entire supernode. We introduce CAISA, a composable AI systems architecture that enables disaggregated memory expansion for large-scale AI workloads using CXL-based shared memory. CAISA goes beyond conventional memory disaggregation by jointly designing hardware support, runtime mechanisms, and workload mapping to provide contention-free shared-memory access across CPUs, accelerators, and CXL memory devices. Its key mechanism is workload-aware memory isolation, which maps shared memory through reserved address spaces to control data visibility, avoid hardware coherence overheads, and enable fine-grained pipelined data movement between compute and memory devices. By coupling memory isolation with software-managed data orchestration, CAISA decouples compute and memory resources while preserving efficient data movement, providing a scalable and cost-efficient foundation for future AI infrastructure.