Tag
This paper introduces Neuron-Activated Graph (NAG) Ranking, a training-free framework for selecting pretraining data aligned with target tasks by identifying and ranking candidate data based on similarity in neuron activation patterns. The approach achieves 4.9% average improvement over random sampling and demonstrates that sparse neuron patterns capture functional capabilities for target learning.
Introduces Neuron-Activated Graph Ranking, a training-free method that uses sparse high-impact neuron sets to select pretraining data for target tasks, boosting average benchmark performance by 4.9%.