Deploying a large language model (LLM) requires choosing the right accelerator, parallelization strategy, engine configuration, and replica count amongst ~140 other variables across clouds, regions, and markets. These decisions are intertwined, and the best cluster-wide plan continually changes as jobs contend for shar...
Mankeerat Sidhu, Gang-Muk Lim, Hetarth Chopra et al.· 0 citations
SafePar is introduced, a specification and monitoring framework for policies over concurrent microservice executions that constrains both the order of API calls and their series-parallel structure and shows that SafePar enforces rich concurrency-aware policies while incurring only millisecond-scale latency overhead.
Karuna Grewal, B. Godfrey, Justin Hsu et al.· arXiv.org· 0 citations
Many cloud-based applications are organized as loosely coupled microservices, where invoking a service's API triggers a cascade of APIs across many services and leads to inter-service exchange of API parameters and output responses. Current tools for monitoring microservice safety properties have limited expressiveness...
Karuna Grewal, B. Godfrey, Justin Hsu· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.