DRIFT: Data Selection for LLM Instruction Tuning via On-Policy Attribution
This work proposes DRIFT (Data Refinement via On-Policy Influence Functions for Supervised Fine-Tuning), a data attribution method built on influence functions that outperforms all evaluated data selection baselines on Olmo3-7B-Instruct-SFT and OpenR1-Distill-7B, raising average accuracy across eight benchmarks.
Zefan Wang, Lin-Cheng Li, Tianyu Yu et al.
· 1 citation