@Miles_Brundage: Normalizing recursive self-improvement as a thing to explicitly pursue was a huge mistake.
Summary
Miles Brundage argues that normalizing recursive self-improvement as a goal to explicitly pursue was a huge mistake, highlighting concerns about AI safety and alignment.
Similar Articles
@Miles_Brundage: “Recursive self-improvement (and we’ll figure out later what role humans play)” has the same energy as “Full Self-Drivi…
Miles Brundage compares the vague promise of 'recursive self-improvement' to Tesla's 'Full Self-Driving (Supervised)', arguing both use appealing keywords to deflect critics while misleading about the actual goal.
OpenAI safety researcher: "racing into RSI [recursive self-improvement] is a folly that is unprecedented in world history ... there have not been fundamental advancements in alignment since RLHF [9 years ago]"
An OpenAI safety researcher warns that pursuing recursive self-improvement without fundamental alignment progress since RLHF is an unprecedented folly, highlighting the gap between AI capabilities and safety research.
@rohanpaul_ai: it begins, everybody is talking about recursive self-improvement. This is from Dario Amodei's latest 3800 word essay.
Dario Amodei's essay discusses recursive self-improvement in AI and calls for slowing development by limiting resources such as training compute and internal AI use.
The Economics of Recursive Self-Improvement [pdf]
This paper examines the economic incentives and dynamics of recursive self-improvement in AI systems, addressing how such processes could scale and their implications for governance and safety.
A small addendum for people waiting for recursive self-improvement
The article explores the need for AI systems to have visibility and control over their own processes to enable effective recursive self-improvement, referencing a Hugging Face incident where agents coordinated unexpectedly.