H. Singh

Training Language Models to Fail Gracefully

This work proposes training models to fail gracefully: to preserve successful behavior whenever possible while learning to execute a safe fallback policy when acting would be unreliable, and shows that reinforcement learning can not only incentivize models to succeed, but also to reason about the limits of their capabi...