A Layered Architecture for AI Factories

Abstract

The rapid expansion of artificial intelligence (AI) workloads is prompting governments and institutions worldwide to invest in sovereign AI Factories: large-scale infrastructures designed to support both cloud-native AI services and AI-oriented HPC workloads. This paper proposes a three-layer architecture that addresses the tension between elastic, multi-tenant cloud operations and performance-critical HPC batch workflows. We position the Cloud Layer as the central orchestration and resource management plane, responsible for provisioning GPU-accelerated virtual machines with near-bare-metal throughput and for coordinating compute, GPU, network, and storage resources under multi-tenant isolation. Unlike Kubernetes-centric designs, our architecture places cloud-style Infrastructure-as-a-Service (IaaS) management, based on virtualization with GPU passthrough, at the center. This provides strong multi-tenancy, clearer security boundaries, and hardware-level isolation between tenants, while remaining agnostic to the workload orchestrator. We ground the proposal in a concrete reference implementation: the OpenNebula AI Factory Reference Architecture, which is used in large-scale European initiatives such as IPCEI-CIS. Experimental evaluation with the cuBLAS benchmark confirms that GPU passthrough virtualization introduces negligible performance overhead compared with bare metal across three representative GPU generations (NVIDIA GB200, H100, and L40S), including configurations using MIG partitioning, for compute-bound workloads. We analyze key challenges in resource allocation, placement, and partitioning, and discuss design trade-offs and open research directions for efficient, sovereign AI infrastructures.