Xia-Ling-Zhang-Yu-Xin Xia

Learning from Interaction Experience via Localized Context Distillation

Learning from interaction experience is essential for enabling large language models to continually improve during deployment. Context distillation offers a promising path: privileged information extracted from experience is provided to a teacher, whose behavior is then distilled into a student model that has no access...