Jian-Hua Z. Huang

When Does Backpropagating Through Policy Memory Matter? Physical Credit, Optimizer Updates, and Observability

Policies with memory can learn along two backward paths: through the physical states their actions produce and through the representations they store. Transformer-XL and truncated backpropagation through time cut the second path at stored history while keeping its values. We ask when this cut matters. Holding the forwa...