This work considers agent memory as a temporally-extended abstraction over the agent’s observation-action history, and derives POMDP classes by applying traditional MDP state abstractions, such as model-preservation, optimal value Q ∗ preservation, and optimal policy π ∗ preservation, to the POMDP setting.
Hierarchical reinforcement learning improves long-horizon control by organizing primitive actions around persistent subgoals and assigning credit at multiple temporal scales. Recent hierarchical language agents bring these benefits to interactive tasks by explicitly separating subgoal planning from action execution. We...
This work introduces a surprisingly simple neural network architecture change: a learnable, state-independent attention mask applied to the inputs of the policy and value networks and trained end-to-end using only the RL objective.
In partially observable reinforcement learning (RL), a later observation can make stored information obsolete or change what it implies for the next decision. Memory architectures and benchmarks for RL mostly test retention, the ability to keep information unchanged until it is needed. We formalize two further requirem...
Oleg Shchendrigin, E. Cherepanov, A. Panov et al.· 0 citations
Partially observable environments pose a fundamental challenge in deep reinforcement learning, requiring agents to compress temporal information from observations and maintain a memory to make effective decisions. While there exist many approaches ranging from gated recurrence to attention mechanisms and model-based RL...
Sathya Kamesh Bhethanabhotla, Efstratios Gavves, André Biedenkapp· 0 citations
Memory is essential for long-horizon robotic manipulation, where successful actions may depend on past events that are no longer recoverable from the current observation. As episodes grow longer, however, retaining the full history becomes increasingly costly, creating a fundamental scalability challenge for memory-aug...
Hao-Xuan Wang, Geng-Yu Zhang, R. Kompella et al.· 0 citations
Recurrent models process long contexts efficiently by compressing their history into a fixed-size state, but modern architectures typically do so in a single causal pass over the sequence. Each input must therefore be compressed before the model knows how it will later be used, forcing a limited state to compromise acr...
Jyothish Pari, Ryan Bahlous-Boldi, Pulkit Agrawal· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
