Semantic-Aware Scheduling and Resource Allocation for AI Inference Services in Enterprise Multi-Cloud Systems

Abstract

Multi-cloud and hybrid-cloud deployment has become a common architecture for enterprise artificial intelligence (AI) services, where inference requests may need to be processed across heterogeneous public, private, and regional cloud domains. Existing studies mainly model incoming requests as conventional resource-oriented tasks and focus on workload placement, latency reduction, or execution efficiency. However, such models are insufficient for enterprise AI services, because practical requests often carry richer service semantics, including business importance, service quality requirement, privacy sensitivity, compliance constraint, and feasible execution domain. To address this issue, this paper models each request as a semantic AI service request and investigates a semantic-aware scheduling and resource allocation problem in enterprise multi-cloud environments. To solve the formulated mixed-integer nonlinear problem, we develop a two-stage semantic-aware orchestration algorithm. In the first stage, the orchestrator selects the target cloud domain for each request and reserves the minimum feasible computation resource to satisfy its deadline. In the second stage, the remaining computation resource is allocated within each cloud domain to further reduce inference delay. Experimental results show that our proposed algorithm consistently outperforms representative baselines in terms of accepted request ratio, averaged delay, and mismatch penalty across different multi-cloud configurations.