developmental-approach

This paper uses a developmental approach to study how neural language models, specifically Transformers, learn statistical patterns from a synthetic grammar, finding that they first acquire global abstract statistics then local dependencies, with over-generalizations early on.