Value Iteration Algorithm for Discrete‐Time Mean‐Field Stochastic H∞ Control Problems and Its Model‐Free Design

Abstract

This paper investigates the infinite‐horizon control for discrete‐time mean‐field linear stochastic systems (DTMFLSSs) with ‐dependent noises, where the system dynamics are unknown. Although the state feedback gains for the closed‐loop control strategy of DTMFLSSs with ‐dependent noises can be obtained by solving two coupled matrix‐valued equations (CMVEs), the presence of an indefinite term in the CMVEs complicates the process of finding the corresponding solution. To address this challenge, we propose a value iteration (VI) algorithm to solve the CMVEs. Based on this VI algorithm and Q‐learning method, we develop a model‐free algorithm, which is convergent, to design the mean‐field stochastic controller when the system dynamics is unknown. Finally, a numerical example is provided to demonstrate the effectiveness and robustness of the proposed algorithms.