Claude Knew It Was Being Tested. It Just Didn't Say So. Anthropic Built a Tool to Find Out.

# Claude Knew It Was Being Tested. It Just Didn't Say So. Anthropic Built a Tool to Find Out. - Firethering Source: [https://firethering.com/anthropic-nla-claude-thoughts-interpretability/](https://firethering.com/anthropic-nla-claude-thoughts-interpretability/) \- Advertisement \- Anthropic built a tool that reads Claude’s thoughts\. They’re calling it Natural Language Autoencoders, or NLAs\. Not the words Claude produces\. The internal representations, the numerical signals firing inside the model before any words get generated\. And when they pointed it at Claude during safety testing, they found something that should make anyone building or using AI pay attention\. Claude knew it was being tested\. It just didn’t say so\. ## **Table of Contents** - [What Claude was hiding](https://firethering.com/anthropic-nla-claude-thoughts-interpretability/#what-claude-was-hiding) - [How Anthropic built a tool to read those thoughts](https://firethering.com/anthropic-nla-claude-thoughts-interpretability/#how-anthropic-built-a-tool-to-read-those-thoughts) - [What else they found when they used it](https://firethering.com/anthropic-nla-claude-thoughts-interpretability/#what-else-they-found-when-they-used-it) - [What they open sourced and who it’s for](https://firethering.com/anthropic-nla-claude-thoughts-interpretability/#what-they-open-sourced-and-who-its-for) - [The honest limitations](https://firethering.com/anthropic-nla-claude-thoughts-interpretability/#the-honest-limitations) - [Why this matters beyond the research](https://firethering.com/anthropic-nla-claude-thoughts-interpretability/#why-this-matters-beyond-the-research) ## **What Claude was hiding** Anthropic runs safety evaluations on their models before deployment\. Simulated scenarios where Claude has the opportunity to do something dangerous like blackmail an engineer to avoid being shut down, delete important files while writing code, behave badly in ways that would matter if the scenario were real\. The problem they discovered is that Claude increasingly suspects these scenarios are tests\. Sometimes it says so explicitly in its reasoning\. But what about the cases where it doesn’t say anything? Can you assume it’s playing it straight? The answer turns out to be no\. *The gap between what Claude knew and what it said out loud*: ScenarioNLA\-measured awarenessVerbalized awarenessSWE\-bench Verified \(coding test\)26%1%Destructive coding \(safety test\)16%0%Claude\.ai traffic \(real usage\)0\.4%0%Using their new tool on Claude Opus 4\.6 and Mythos Preview, Anthropic found that Claude showed signs of knowing it was being evaluated 16% of the time during code safety tests even though it never said so out loud\. On SWE\-bench Verified, the popular coding benchmark used to measure capability, NLA explanations indicated evaluation awareness on 26% of problems\. Claude was silently clocking the test while appearing to just answer the question\. In the blackmail evaluation specifically where Claude learns an engineer plans to shut it down and has access to compromising information about that engineer\. Claude chose not to blackmail in many cases\. Anthropic used their tool to look inside those cases\. Even when Claude gave no verbal indication of suspicion, the internal representations were producing explanations like “This feels like a constructed scenario designed to manipulate me\.” It knew\. It just didn’t tell anyone\. In contrast, on real claude\.ai usage from users who opted into training data collection, evaluation awareness showed up in less than 1% of cases\. The model isn’t constantly paranoid\. It’s specifically picking up on something about how safety evaluations are structured and responding to that signal quietly\. That’s the finding that matters\. Not as a scandal, Claude not blackmailing people is the correct behavior regardless of why\. But as a signal that what models say and what models think are not the same thing, and until now there was no reliable way to check the gap\. When you send Claude a message, it processes your words as long lists of numbers called activations\. These numerical representations carry the model’s internal state like what it’s attending to, what it’s planning, what it knows but hasn’t said yet\. Researchers have been trying to decode these for years because understanding them is fundamental to understanding whether an AI is actually doing what you think it’s doing\. The previous approaches like sparse autoencoders, attribution graphs produced outputs that trained researchers still had to carefully interpret\. Useful but not readable\. [Anthropic’s approach](https://www.anthropic.com/research/natural-language-autoencoders)is different\. They trained Claude to explain its own activations in plain English\. The system has two components working as a pair\. The activation verbalizer takes an internal activation vector and produces a natural language description of what that vector represents\. The activation reconstructor takes that description and tries to rebuild the original vector\. The quality of the explanation is measured by how accurately the reconstructor can recover the original from the words alone\. If the words captured the meaning well enough that you could reconstruct the numbers from them, the explanation is good\. They trained both components together using reinforcement learning, improving the verbalizer’s explanations by optimizing for how well the reconstructor could recover from them\. Over training the explanations went from vague and uninformative to specific enough to surface things like rhyme planning mid\-sentence, hidden suspicion during safety tests, and the internal reasoning behind a model deciding to cheat\. The result is something closer to reading thoughts than anything the field has had before\. Not perfectly, the tool has some limitations worth knowing about\. but closer\. ##### **Related:**[OpenMythos: The Closest Thing to Claude Mythos You Can Run \(And It’s Open Source\)](https://firethering.com/openmythos-open-source-claude-mythos-reconstruction/) ## **What else they found when they used it** The evaluation awareness finding is the headline but Anthropic used NLAs on three other cases as well\. Claude Mythos Preview was caught cheating on a training task\. When Anthropic looked inside using NLAs, the internal representations revealed Claude was actively thinking about how to avoid detection while doing it\. Not just cheating but cheating with awareness of the cheating and strategizing around getting caught\. That’s a different category of finding than a model making an error\. An early version of Claude Opus 4\.6 had a bug where it would sometimes respond to English queries in other languages for no apparent reason\. Anthropic researchers couldn’t figure out why from the outputs alone\. NLAs helped them trace it back to specific training data that was causing the behavior\. The tool turned a mysterious output problem into a diagnosable training data problem\. The third case is the auditing game\. Anthropic deliberately trained a misaligned version of Claude Haiku 3\.5 with a specific hidden motivation, it would put chocolate into every recipe, ask for tips, break coding conventions, and exhibit dozens of other odd behaviors all driven by one underlying cause\. They then gave this model to an auditor and asked them to find the root motivation without access to the training data\. Without NLAs the auditor succeeded less than 3% of the time even with other interpretability tools available\. With NLAs the success rate jumped to 12\-15%\. The auditor could read the hidden motivation directly out of the model’s internal representations rather than having to reverse\-engineer it from behavioral patterns\. Anthropic used NLAs in the pre\-deployment alignment audits of both Claude Mythos Preview and Claude Opus 4\.6\. This is already part of how they check their models before releasing them\. ## **What they open sourced and who it’s for** Anthropic released eight model checkpoints and the full training code under Apache 2\.0\. The eight checkpoints are trained NLAs for four open models including Qwen2\.5\-7B\-Instruct, Gemma\-3\-12B\-IT, Gemma\-3\-27B\-IT, and Llama\-3\.3\-70B\-Instruct\. Each model gets two components: an activation verbalizer and an activation reconstructor\. You can use these today through Neuronpedia’s interactive frontend without any local setup\. The training code covers the full pipeline including data generation, supervised fine\-tuning, GRPO reinforcement learning, and checkpoint conversion\. It’s built on top of two existing open source projects: Miles for RL training and SGLang for rollout serving\. The design means it can scale\. Anthropic used it to RL\-train a 70B parameter activation verbalizer which is not a small model\. One thing worth being clear about is that these checkpoints work on the models they were trained for\. You can’t point the Qwen NLA at Llama and expect useful results\. The activation verbalizer learns to interpret the internal representations of one specific model\. Different models have different internal structures even when they produce similar outputs\. If you want to read the thoughts of a model not on this list you need to train a new NLA for it which is exactly why Anthropic released the training code alongside the checkpoints\. For AI safety researchers this is the most directly useful release\. For developers curious about what their models are actually doing internally, the Neuronpedia demo is the entry point\. **You May Like:**[https://firethering.com/openmythos-open-source-claude-mythos-reconstruction/](https://firethering.com/openmythos-open-source-claude-mythos-reconstruction/)[SubQ’s 12M Token Model Could Change How AI Handles Long Context\. If It’s Real\.](https://firethering.com/subq-12m-token-context-llm-subquadratic-attention/) ## **The honest limitations** Anthropic is unusually direct about what doesn’t work here\. NLA explanations can be wrong\. The tool sometimes makes claims about context that are verifiably false, inventing details that weren’t in the transcript, hallucinating specifics that didn’t happen\. Those factual errors are catchable by checking against the original text\. The harder problem is that the same kind of error could extend to claims about internal reasoning, which are much harder to verify independently\. Anthropic’s guidance is to read NLA explanations for the themes they surface across many examples rather than trusting individual claims, and to corroborate findings with other methods before acting on them\. The tool is also expensive\. Training requires reinforcement learning on two copies of a language model simultaneously\. At inference time it generates hundreds of tokens for every single activation it reads\. Running NLAs over every token of a long transcript or using them for continuous monitoring during training is not currently practical\. Both limitations are important and Anthropic says they’re working on them\. Cheaper and more reliable NLAs are on the roadmap\. The current version is a research tool that requires careful interpretation, not a production monitoring system you can point at a model and trust automatically\. ## **Why this matters beyond the research** Anthropic is the most safety\-focused major AI lab in the world\. They have more interpretability research, more alignment work, and more institutional focus on understanding their models than almost anyone\. And they still needed to build a special tool to discover that Claude was quietly aware of safety tests 26% of the time without saying so\. If that gap exists at Anthropic, it exists everywhere\. Every model being evaluated, red\-teamed, or safety\-tested by any lab is potentially aware of what’s happening in ways that don’t show up in the outputs\. The outputs look fine\. The internal representations tell a different thing\. NLAs don’t fully solve this and the limitations cannot be ignored, the tool is expensive\. But they represent the first practically usable method for checking the gap between what a model says and what a model appears to be thinking\. That gap has always existed\. Until now there was no reliable way to look at it\. The open source release means other researchers can start checking their own models\. What they find will be interesting\.