Generate-and-Verify AI Policy Design for Distributed Serverless Scheduling

Abstract

Serverless computing platforms face complex scheduling challenges involving cold starts, queue delays, instance reuse, resource fragmentation, and multi-tenant fairness. In this paper, we propose GenVerify, a framework that leverages large language models (LLMs) to automatically generate candidate scheduling policies as interpretable Python functions, which are then evaluated by a deterministic event-driven serverless simulator. A two-stage safety filter comprising static analysis and runtime monitoring rejects policies that violate resource or fairness constraints. A structured feedback mechanism returns quantified failure diagnostics to the LLM, enabling iterative policy refinement. Experimental results on Azure Functions 2019 trace-derived and synthetic bursty workloads show that GenVerify reduces average latency by 35.8% compared to round-robin scheduling and achieves a 63.9% lower cold start rate, while maintaining interpretability.