An Abstention-Aware, Calibrated Risk-Gated Controller for Canary Deployment under Low Observability

Abstract

Container deployment automation tools such as Kubernetes-native rollout, Argo Rollouts, Flagger, and Spinnaker/Kayenta gate promotion on operator-defined statistical thresholds do not interpret free-form intent. This paper studies whether coupling a Large Language Model (LLM) and retrieval with a quantitative, confidence-calibrated risk gate improves rollout reliability over such tools. A risk-gated canary controller is defined, in which, at each stage, a normalized composite risk score is computed from health, error-rate, and tail-latency signals and combined with an LLM decision confidence that is calibrated by Platt scaling (logistic calibration of the self-consistency agreement) and evaluated with Expected Calibration Error (ECE) and Brier score. A prototype (Python, Docker, GPT-4o, FAISS retrieval) is evaluated on a Kubernetes cluster under load generated with Locust over n = 20 repetitions against a manual baseline, Kubernetes-native rollout, and an automated canary-analysis baseline; differences are tested with the Mann–Whitney U test. An ablation study isolates the contribution of confidence, retrieval memory, and the risk score, and a weight-sensitivity analysis examines robustness.