Hypergradient-based Bilevel Reinforcement Learning with Improved Sample Complexity
This work proposes a hypergradient-based bilevel RL algorithm using the optimality of the Boltzmann policy for the entropy regularized discounted RL objective function and obtains an iteration complexity of $O(\epsilon^{-1})$ and state-of-the-art sample complexity of $\tilde{O}(\epsilon^{-2})$ under mild regularity conditions.