Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning
TL;DR
A coreset-based task selection approach that selects a weighted subset of tasks based on how diverse they are in gradient space, prioritizing the most informative and diverse tasks is proposed, guaranteeing a faster adaptation to unseen tasks while focusing training on the most relevant tasks.
Abstract
We study how task selection can enhance sample efficiency in model-agnostic meta-reinforcement learning. Traditional meta-RL typically assumes that all available tasks are equally important, which can lead to task redundancy when they share significant similarities. To address this, we propose a coreset-based task selection approach that selects a weighted subset of tasks based on how diverse they are in gradient space, prioritizing the most informative and diverse tasks. Such task selection reduces the number of samples needed to find an ϵ-close stationary solution by a factor of $\mathcal{O}(1/ \in )$. Consequently, it guarantees a faster adaptation to unseen tasks while focusing training on the most relevant tasks. As a case study, we incorporate task selection to MAML-LQR [1], and prove a sample complexity reduction proportional to $\mathcal{O}\left( {\log \left( {1/ \in } \right)} \right)$ when task-specific costs also satisfy the gradient dominance property. Our theoretical guarantees underscore task selection as a key component for scalable and sample-efficient meta-RL. We numerically validate this trend across multiple RL benchmark problems, illustrating the benefits of task selection beyond the LQR baseline.
