Bound-Optimized Task Choice for Path Integral Control
TL;DR
This work proves that every task provides an upper bound on the true cost-to-go and that BOTC minimizes this bound, and proposes Bound-Optimized Task Choice (BOTC), a framework that optimizes over the entire space of valid approximations, termed tasks, satisfying the PI coupling constraint.
Abstract
Path Integral (PI) control is a powerful sampling-based method for stochastic optimal control, but it requires a restrictive coupling between the noise covariance and the control cost matrix that is rarely satisfied in practice, particularly in aerospace and cyber-physical systems. We propose Bound-Optimized Task Choice (BOTC), a framework that optimizes over the entire space of valid approximations, termed tasks, satisfying the PI coupling constraint. We prove that every task provides an upper bound on the true cost-to-go and that BOTC minimizes this bound. We derive a change-of-measure formulation that enables evaluation of all candidate tasks from a single set of Monte Carlo samples, eliminating the need to resample for each candidate task. The resulting optimization is parameterized by a positive semi-definite matrix. Furthermore, we propose a novel Normal-Inverse-Wishart distribution-based importance sampling scheme to improve global optimization. We validate BOTC on a finite-horizon stochastic linear-quadratic regulator problem, demonstrating that it tracks the constrained optimum.