TRSOC is introduced, which augments standard stochastic optimal control with a Kullback--Leibler (KL) divergence between controlled and reference trajectory distributions, and the regularization parameter induces a trade-off between performance-driven and reference-preserving behavior.
Abstract
We introduce trajectory-regularized stochastic optimal control (TRSOC), which augments standard stochastic optimal control (SOC) with a Kullback--Leibler (KL) divergence between controlled and reference trajectory distributions. Using Girsanov's theorem, the trajectory KL reduces to a quadratic drift mismatch penalty, yielding a modified running cost that preserves the dynamic programming (DP) structure. We derive the corresponding Hamilton--Jacobi--Bellman (HJB) equation and characterize the optimal policy. In the linear-quadratic (LQ) setting, the formulation admits a closed-form solution with an augmented control cost. Experiments show that the regularization parameter induces a trade-off between performance-driven and reference-preserving behavior, including cases with reference dynamics learned from offline data.
This paper presents a data-driven stable manifold (DD-SM) method, which integrates Koopman operator representation learning with the geometric stable manifold approach to Hamilton-Jacobi-Bellman (HJB) equations, enabling end-to-end optimal feedback control synthesis from raw trajectory data without prior knowledge of s...
Empirical results demonstrate that PI-VM matches SOTA precision with an order-of-magnitude efficiency gain in low-dimensional settings, while effectively mitigating mode collapse in high-dimensional scenarios, and offers a scalable solution for solving complex SOC problems.
Bangyan Liao, Cheng-Lei Yu, Yuchen Yang et al.· 0 citations
On-policy and off-policy Q-learning algorithms that learn the optimal controller solely from online state trajectory data are developed, specifically developing on-policy and off-policy Q-learning algorithms that learn the optimal controller solely from online state trajectory data.
Wasserstein policy gradient (WPG) reduces exactly to a finite-dimensional ODE for the feedback gain and action covariance and it is proved that this ODE is globally well posed and converges exponentially from every admissible initialization.
A hybrid policy combining time-independent affine feedback with finite-horizon steering over a terminal layer is constructed, proving that its worst-case cost gap decays exponentially with the terminal-layer length uniformly in the horizon, which helps reducing the computation cost for long-horizon robust controls.
The resulting characterization reveals how control-dependent noise, policy-evaluation sensitivity, and update ordering govern local error propagation, and yields explicit sufficient conditions for local linear convergence.
Karl Handwerker, Felix Thömmes, Lucas Günther et al.· 1 citation
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 14, 2026
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
MIT News · Artificial Intelligence· news.mit.eduOct 13, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
Microsoft Research Blog· microsoft.comOct 12, 2026
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
