@swyx: co-sign. a very handy mental framework for what kinds of learning transformers do well today, and why it runs into limi…
Summary
The article discusses a mental framework for understanding what transformers learn well and their limitations, arguing that scaling current paradigms may be inefficient compared to approaches that hypothesize and seek truth, referencing the need for adversarial world models and reinforcement learning.
View Cached Full Text
Cached at: 05/23/26, 08:01 AM
co-sign. a very handy mental framework for what kinds of learning transformers do well today, and why it runs into limitations. when @ankit2119 and i wrote about the need for adversarial world models earlier this year, we were describing a couple of the functions of these rungs of thinking that bring us ever closer to the kolmogorov-limit generator of reality. throwing more params, more power, more everything at a demonstrably inefficient paradigm will be outclassed by the simple solution that can hypothesize and seek truth rather than backfit a house of cards - although the bitter lesson is it is simpler to scale and we may hit agi anyway because human intelligence just isn’t that smart nor plentiful
Rishabh Agarwal (@agarwl_): Very well written blog. I think of RL as learning from interventions, and it kinda explains why it’s more powerful as a paradigm than supervised learning.
Now learning from counterfactuals is something we haven’t been historically good at but maybe world modelling+ RL can get
Similar Articles
Agentic Transformers Provably Learn to Search via Reinforcement Learning
This paper theoretically studies how transformer-based policies acquire search capabilities from reinforcement learning training dynamics in a stochastic tree environment. It shows that a two-head transformer can implement depth-first search and that this mechanism emerges naturally from sparse reward signals under a depth-wise curriculum.
@qinzytech: https://x.com/qinzytech/status/2066585405479371092
A technical analysis of two approaches to building self-evolving AI agents: model-based (via architecture like SSMs or transformer with fast-weight updates, and training methods) and harness-based (via memory or meta harness that can rewrite itself). The author provides practical recommendations for different audiences.
A Mathematical Framework for Transformer Circuits (2021)
This paper presents a mathematical framework for reverse-engineering transformer models to enhance mechanistic interpretability and address safety concerns in AI systems.
@juleslogs: Want to understand modern AI? Start here: 1. Transformers → Illustrated Transformer 2. LLMs → Build a Large Language Mo…
A tweet curating foundational resources for understanding modern AI, covering topics from transformers to physical AI, including key papers and models.
Syntax vs. Semantics: How Transformers Learn Deep Dependencies
This paper introduces a mechanistic framework analyzing transformer learning dynamics, identifying gradient starvation as a barrier to deep semantic dependencies and validating chain-of-thought strategies for effective learning.