Tag
This research explores how different data domains interact during pretraining, finding that adding code improves math performance while some combinations hurt each other, and proposes incorporating data synergy into scaling laws.