On-policy distillation (OPD) is increasingly studied as a cheaper alternative to reinforcement learning, yet it remains a form of knowledge distillation (KD) that relies on teacher supervision. We revisit OPD from the KD perspective by asking where the teacher's advantage lies along student-generated trajectories. Cont...
Yi-Ping Li, Fan Zhao, Zhi-Bo Hou et al.· 0 citations
Large Language Models (LLMs) are increasingly being used to summarize and verify claims in critical sources that inform consequential decisions. Such claims are full of plural definites. If three of five patients improved after taking drug A, the claim the patients improved after drug A is not true, since two did not,...
Duaa Nakshbandi, Zhi-Yu An, Yi-Ping Li et al.· 0 citations
A playbook for fine-tuning LLMs on de-identified clinical notes from patients with pancreatic cancer, spanning both pre-diagnosis and on-treatment settings is presented, and the use of machine-generated annotations to augment limited expert labels is examined, showing that balanced mixtures of synthetic and human data...
Li-Ching Chen, T. Zack, Divneet Mandair et al.· Proceedings of machine learn...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.