A comparative analysis of dataset distillation and sampling methods in the context of LtR indicates that, in contrast to previous studies in computer vision, the selected distillation methods do not outperform random sampling.
Dataset distillation (DD) condenses a large original dataset into a small distilled dataset with high training utility. Decoupled statistical matching methods substantially reduce distillation time and memory overhead while achieving strong performance. However, they typically supervise all distilled samples using runn...
Hong-Xu Ma, Guang Li, Shi-Jie Wang et al.· 0 citations
Multi-step matching models, including flow and diffusion models, produce high-quality outputs but incur substantial inference costs and may reproduce unwanted components of their training datasets. We introduce Inverse Distillation Unlearning (IDU), a unified framework that simultaneously distills a teacher multi-step...
Aleksei Leonov, N. Kornilov, Zhen-He Zhang et al.· 0 citations
This work test several proposed ``state-of-the-art''solutions from the literature and find that older methods are currently far superior for general use, and finds that a new and simple baseline, using LBFGS on a sub-gradient, is highly effective with minor tweaks, despite being dismissed in the literature for theoreti...
An exact finite-sample necessary-and-sufficient condition for two-stage training to strictly outperform real-only training under the same real-data budget and identical real-stage updates is established.
Backward Kernel Herding is introduced, an algorithm that addresses the problem of iteratively removing points from the dataset by iteratively removing points from the dataset, achieving results comparable to current state-of-the-art approaches while accelerating the subsampling process in realistic scenarios where the...
Blanca Cano-Camarero, Yago R. Aguado-Carrillo-de-Albornoz, Ángela Fernández-Pascual et al.· 0 citations
DataFlex-RL, an evaluation platform for comparing choices under a common GRPO recipe, is introduced, finding that changing the data policy measurably changes the training process but does not produce a reproducible improvement over uniform training.
Hao Liang, Ming-Rui Chen, Hengyi Feng et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
