Tag
Concordia University and Mila researchers derive exact scaling laws for prediction regret in a bottlenecked bigram model with Zipfian token statistics, isolating capacity-driven scaling from optimization effects and matching predictions on WikiText2 and OpenWebText.