Form Over Content In Gradient-Based Data Attribution Methods
Data attribution methods using gradient similarity are widely used to analyze and select training data for large language models, but what gradient similarity actually measures is debated. Some interpret it as identifying task-relevant skills, while other work reports that surface form is the main factor. We resolve th...
Sunwoo Kim, Seokwon Jung, Sohyung Kim et al.
· 1 citation