Tag
Ethan Mollick reflects on how AI agents — including Meta's Muse and OpenAI's dots — can organize work without elaborate human management, applying the Bitter Lesson to agentic systems and personal assistants that now catch users' own mistakes.
The tweet argues that Elon Musk's few critical and irreversible decisions are key, comparing this to the 'bitter lesson' in AI applied to free markets.
The tweet emphasizes the importance of careful problem specification in machine learning over attachment to methods, echoing DSPy's philosophy and referencing Sutton's bitter lesson.
This tweet reflects on a rewarding conversation about linguistics versus NLP, the Bitter Lesson, and the evolution of the Transformer architecture, linked to a podcast episode on AI tokenomics.
The author argues that while tool use allows smaller language models to perform tasks effectively, larger models remain crucial for speed, reliability, and internalized knowledge, emphasizing the ongoing need for scaling in AI.
The article argues that AI harnesses serve two distinct purposes—providing context about what the user wants (intent) and instructions on how to achieve it (execution)—and that these age differently: execution instructions become less valuable as models improve, while intent context becomes more valuable.
Rich Sutton discusses the common mistake of relying on one-step predictions in AI research, advocating for temporally abstract models using options and GVFs.
Max Rumpf argues that human feedback is becoming obsolete for training advanced AI models, citing examples like chess, math, and search. He advocates for human-free methods like self-play and synthetic data, while a quoted tweet from Will Depue calls for a large-scale data infrastructure parallel to compute scaling.
Anjney Midha warns that within ~18 months, AI-driven disruption will fundamentally threaten cybersecurity companies whose value depends on providing trust, citing the bitter lesson.
Richard Sutton summarizes his bitter lesson: AI should focus on scalable methods like search and learning rather than on incorporating human knowledge.
Introduces pedagogical RL, a paradigm where privileged self-teachers are trained to generate correct and easy-to-follow rollouts, showing it is a relatively easy RL problem.
The author argues that while the 'bitter lesson' and 'no free lunch' intuitions are misleading in isolation, they provide the correct perspective when combined.
A thoughtful thread on developing genuine research skills in machine learning, covering how to pick problems independently, cultivate taste, upgrade information inputs, and write to clarify thinking.