Tag
Yale researchers use influence functions to identify indirect evidence in language model training data across 15 BLiMP grammatical phenomena, showing that removing the top influential sentences lowers held-out accuracy significantly more than removing random controls, and characterizing how this evidence relates to linguistic structure.
This paper introduces an exposure-based framework to study grokking-like delayed generalization during LLM pre-training, using BLiMP minimal pairs and critical phrases. The authors observe delayed generalization across five grammatical phenomena and analyze internal changes such as concept vector predictability and attention head concentration.