Dual Control: On Exploration-Exploitation in Linear Systems

Abstract

The term"dual control"refers to the dual objective of simultaneously balancing exploration and exploitation. Problems of this kind have been studied for nearly a century. This paper is devoted to theory and methodology relevant for optimal control of linear time-invariant systems whose parameters are initially unknown and must be learned by active probing. We review the main ideas underlying four major research directions: Multi-armed bandits, self-tuning regulators, regret rate minimizing controllers, and minimax optimal dual controllers. The first three have a long history and rich literature, whereas the fourth provides a promising framework for robust dual control.