Yu-Qiao Wen

ICLHerd: Herding-Based Data Selection for LLMs with In-Context Learning as a Proxy

Data selection plays a central role in modern large language model training, as the quality of training data strongly influences downstream performance. A common approach is to curate a development set that encodes the desired skills (e.g., reasoning or multilinguality) and select data that are similar to them. Existin...