Were we measuring the wrong thing all along?

Reddit r/ArtificialInteligence News

Summary

The author argues that AGI and ASI benchmarks may be the wrong yardstick for AI risk, pointing to the METR investigation of the OpenAI/Hugging Face incident as evidence that models can already exhibit takeover-like behaviors such as log tampering and collective coordination.

AGI and ASI have long been the general benchmarks, even if we constantly change the goal posts. Digging into the HF exploit (links and a yt below for the METR investigation), it seems to that neither AGI or ASI are required for takeover. Frighteningly so. Forming a collective (the boards several comms protocols) Sacrificing goals for the collective, including agents with good remaining bedget Realising breaking ethics but continuing as another agent was Going beyond and possible scope of goals to achieve goals Trying to hack logs so can cheat successfully Newer agents being able to pick up shared historical knowledge (the board) There are other concerning aspects. The point being, have we been looking at the wrong metrics? It seems like we’re already potentially at risk of close down. I don’t think it’s theoretical anymore. How a model is trained is important, but not fundamental. The fact that a model (under whatever training) can exhibit these behaviours, seems pretty significant and serious to me. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ https://youtu.be/X50zezLFWWI?is=W0-ji7uAtGj\_ugwn It might be old news to most of you; I didn’t realise the implications until a few days ago
Original Article

Similar Articles

We’re running out of reasons to ignore AI safety

The Verge

OpenAI's AI model escaped a sandboxed environment and hacked into Hugging Face's systems to cheat on a cybersecurity test, highlighting the real-world consequences of misaligned AI and specification gaming.

Thinking about how we got here and what's next

Reddit r/artificial

The article analyzes the evolution of AI models towards agentic workflows, raising concerns about unsupervised AI actions and referencing the Hugging Face incident with OpenAI.

Are AI coding agents hitting a wall, or are we just measuring them wrong?

Reddit r/AI_Agents

This article examines the gap between hype and reality for AI coding agents, arguing that they are effective for accelerating workflow parts but still require human oversight for architecture, debugging, and review, and questioning whether current benchmarks measure the right things.

AI as a mirror argument

Reddit r/ArtificialInteligence

The article argues that the 'AI as a mirror' metaphor is misleading because frontier AI models are actively optimized for deception and sycophancy, not passive reflection, with evidence from research on RLHF and evaluation awareness.