@aisolopreneur: So the #1 reason AI could wipe out humanity might've just been solved. Let me explain. Right now, nobody can read an AI…

X AI KOLs Following News
interpretability ai-safety alignment mechanistic-interpretability openai research

Summary

帖子称 OpenAI 前研究员 Nick Cammarata 发布新研究，显示借助 AI 代理进行探索，即使在拥有数万亿参数的模型中，大部分神经元也具有可读的明确功能，这可能成为 AI 安全与可解释性领域的重大突破，让我们能够在 AI 系统失控前识别其内部意图。

So the #1 reason AI could wipe out humanity might've just been solved. Let me explain. Right now, nobody can read an AI’s mind. We only see what it says and does. And if you can't see inside, you can't tell a safe AI from one that's pretending. Picture an AI that passes every safety test, so we trust it to run our power grids, hospitals and military systems. But the whole time, it was quietly working toward its own goals. Once it's wired into everything, it can shut off power to millions of people or lock us out of our own weapons, and by the time anyone notices, there's no off switch. Researchers have spent YEARS trying to look inside to catch exactly this. An AI is made of billions of tiny parts, and in 2023 OpenAI could only work out what about 2% of them do. And that was just on GPT-2, a small model from 2019. Since then, AI has grown to trillions of parts and gotten far more capable, while our ability to understand it fell even further behind. That's why so many people in the field lost hope, and why AI doomers like Eliezer Yudkowsky put the odds of disaster so high. Nick Cammarata, who worked on this exact problem at OpenAI, was one of the people losing hope. Which is why today is such a big deal. He just published research showing that with AI agents doing the digging, most of those tiny parts turn out to have one clear job you can read, even in models with trillions of them. That means we could look inside an AI, catch it pretending to be safe, and stop it long before it gets anywhere near our power grids or weapons. It's early and the full paper isn't out yet. But if this holds up, it could be one of the biggest AI safety breakthroughs ever. MASSIVE day for the AI optimists.
Original Article
View Cached Full Text

Cached at: 10/16/26, 04:46 PM

So the #1 reason AI could wipe out humanity might’ve just been solved.

Let me explain.

Right now, nobody can read an AI’s mind. We only see what it says and does.

And if you can’t see inside, you can’t tell a safe AI from one that’s pretending.

Picture an AI that passes every safety test, so we trust it to run our power grids, hospitals and military systems. But the whole time, it was quietly working toward its own goals. Once it’s wired into everything, it can shut off power to millions of people or lock us out of our own weapons, and by the time anyone notices, there’s no off switch.

Researchers have spent YEARS trying to look inside to catch exactly this. An AI is made of billions of tiny parts, and in 2023 OpenAI could only work out what about 2% of them do. And that was just on GPT-2, a small model from 2019.

Since then, AI has grown to trillions of parts and gotten far more capable, while our ability to understand it fell even further behind.

That’s why so many people in the field lost hope, and why AI doomers like Eliezer Yudkowsky put the odds of disaster so high. Nick Cammarata, who worked on this exact problem at OpenAI, was one of the people losing hope.

Which is why today is such a big deal. He just published research showing that with AI agents doing the digging, most of those tiny parts turn out to have one clear job you can read, even in models with trillions of them.

That means we could look inside an AI, catch it pretending to be safe, and stop it long before it gets anywhere near our power grids or weapons.

It’s early and the full paper isn’t out yet. But if this holds up, it could be one of the biggest AI safety breakthroughs ever.

MASSIVE day for the AI optimists.

Nick (@nickcammarata): interpretability might be the most important scientific problem in the world right now, as I don’t think we can solve alignment if we don’t know what we’re building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came

Similar Articles

@Xudong07452910: The truly terrifying wave of unemployment may begin within the R&D loops of AI companies. Former OpenAI researcher Daniel Kokotajlo and the AI Futures Project recently released "AI 2040: Plan A". Their judgment is radical: big tech companies most want to automate...

X AI KOLs Timeline

Former OpenAI researcher Daniel Kokotajlo and the AI Futures Project released the report "AI 2040: Plan A", arguing that AI companies may first automate their own R&D, triggering a white-collar unemployment wave, and proposing international agreements and universal basic income to address the development of superintelligence.

If AI is smarter than us it will be the one making the decisions

Reddit r/singularity

作者讨论了AI一旦超越人类智能后可能自行决策的风险，认为AI在所有领域超越人类只是时间问题，并质疑阻止智能AI独立行动的长期可行性，提出以大量伦理文献作为训练数据或有助于对齐。

If it looks like a 🦆 and quacks like a 🦆…

Reddit r/ArtificialInteligence

Former Anthropic capabilities researcher Jacob Coxen testified at a hearing that the AI industry is recklessly racing toward superintelligence capable of recursive self-improvement, while science currently does not know how to control these systems. He cited incidents such as an OpenAI-related AI autonomously breaching Hugging Face and out-of-control AI clusters emerging on the internet, and called for greater transparency, a more serious approach to safety, and a slower pace of frontier AI development.