Beyond On-Policy vs. Off-Policy: At What Level Should LLM Agent Distillation Follow the Student?
It is found that student-induced histories paired with teacher-generated action prefixes generally provide the strongest standalone training signal, whereas following the student at both levels generally performs worst, and fully on-policy data becomes beneficial when incorporated into a mixture with other configuratio...