Tag
A new preprint shows that many neural language models exhibit abstract representation separation for grammaticality, addressing debates about confounds in measuring model grammatical knowledge.
A perspective on LLM harnesses, tracing their origins to early multi-step tool use research like HotPotQA and GoldEn, with commentary from Christopher Potts on debates about whether LLMs alone would suffice.
This paper uses a developmental approach to study how neural language models, specifically Transformers, learn statistical patterns from a synthetic grammar, finding that they first acquire global abstract statistics then local dependencies, with over-generalizations early on.