Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning
CluSTER is proposed, a Cluster-aware balanced Sampling framework for Training Efficient data Reduction in DP instruction tuning that curates a representative reduced dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while preserving the...
Hyunjin Kim, Youngeun Nam, Jaemin Han et al.
· 0 citations