OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search
The Branch Aggregation Lemma shows that branch-weighted tree statistics recover chain expectations when branch choices and weights are fixed before outgoing transitions are sampled, and bound the gradient bias from adaptive expansion and show that max backup assigns prefix credit to actions leading to better discovered...