A zeroth-order algorithm is developed that uses sampled losses to construct empirical CVaR estimates and their gradient estimates and establishes a finite-time expected suboptimality bound for the weighted ergodic iterate.
Abstract
Distributed systems often operate under uncertainty, where minimizing expected loss may overlook rare but severe events. This paper studies a distributed risk-averse convex optimization problem in which agents cooperatively minimize the average of local conditional value-at-risk (CVaR) objectives over a time-varying network. Each agent has access only to noisy evaluations of its local loss function, rather than to its CVaR objective or gradient. We therefore develop a zeroth-order algorithm that uses sampled losses to construct empirical CVaR estimates and their gradient estimates. At each iteration, agents combine neighboring decisions and perform a local update. Under convexity and Lipschitz continuity assumptions, we prove that the agents reach exact asymptotic consensus. We also establish a finite-time expected suboptimality bound for the weighted ergodic iterate. With diminishing step sizes and fixed sample sizes, the local last iterates converge almost surely to a common optimum, and their limiting expected CVaR gap is bounded in terms of the smoothing and finite-sample errors. This distributed bound matches the parameter dependence of the centralized benchmark provided in this paper. Finally, simulations on a distributed sensor network estimation problem illustrate the efficacy of the method.
The classical linear quadratic regulator (LQR) minimizes the expected cumulative return but fails to account for performance variability, rendering it inadequate for risk-aware applications. To address this, we introduce the variance of the cumulative return as a risk measure in LQR. We derive the first exact closed-fo...
We study finite-horizon LQR with independent disturbances whose laws may drift over time. We address uncertainty in this evolution through ex-ante distributionally robust regret optimization (DRRO), which minimizes worst-case excess expected cost relative to the optimal causal controller that knows the law sequence. To...
Lukas-Benedikt Fiechtner, José H. Blanchet· 0 citations
Sample complexity is a widely used metric in sequential decision-making problems, defined as the number of suboptimal decisions during the interaction between the agent and an environment. We study the sample complexity of stochastic multi-armed bandit problems and introduce the expected sample complexity performance m...
This work exploits the auxiliary-threshold representation of CVaR to establish the existence of an optimal strategy and strong duality without requiring market completeness, and proves that the resulting strategies converge to the optimal control as the number of iterations tends to infinity.
An-Ran Hu, Silvana M. Pesenti, Xiaofei Shi· 0 citations
It is shown that the approximate value functions obtained by value iteration and LP asymptotically converge to the optimal value function, and the policies generated by policy iteration converge in value to an optimal policy.
J. Zhang, Saumya Sinha, Mehdi Hemmati et al.· 0 citations
Motivated by many application problems, we consider Markov decision processes (MDPs) with a general loss function and unknown parameters. To mitigate the epistemic uncertainty associated with unknown parameters, we take a Bayesian approach to estimate the parameters from data and impose a coherent risk functional (with...