Tag
提出了一个跨域迁移表示特征的统计框架，用于检测 LLM 的幻觉与推理错误，在新域仅需 25 条标注样本即可将幻觉检测 AUROC 提升 6.8%、推理步骤预测 AUROC 提升 8.4%。
This paper studies how two LLMs converge toward agreement in multi-turn subjective conversations, showing that while an internal linear 'agreement' representation poorly tracks per-conversation behavior, aggregating across conversations reveals Markovian dynamics and a power-law mapping between internal and external agreement, with internal shifts lagging behind expressed stance changes.
This paper investigates how fine-tuning alters internal representations in large language models and decouples these changes from causal importance components identified by Edge Attribution Patching, finding that task-specific components are concentrated in certain layers and uncorrelated with representational changes.
This paper introduces methods to calibrate confidence in agentic systems using internal representations, demonstrating improved performance over baselines in multi-turn benchmarks.
Anthropic's research on Claude's internal 'J-space' reveals clues about how the model processes reasoning, but it is not a full readable transcript, underscoring key limitations in AI interpretability.
The paper proposes SWE-Pruner Pro, which uses the coding agent's own internal representations to prune irrelevant tool-output lines, reducing token use by up to 39% without a separate pruning model.
Anthropic published a paper and video revealing a 'J-Space' within their models that acts as cached thought concepts for reasoning, and explores the possibility of top-down training to control model thinking.
This paper probes internal representations of LLM forecasters to improve calibration and assess faithfulness of chain-of-thought reasoning, finding that probes achieve better calibration and act as lie detectors.
This paper investigates whether sycophantic behavior in LLMs has distinct internal representations for factual vs opinion sycophancy, using linear probes and steering vectors to show that representations can be either unified or distinct across models.
This paper proposes a recall-controlled abort cascade that uses lightweight probes on LLM agent internal representations to detect and abort doomed episodes early, saving up to 47% inference compute while maintaining high recall of successful episodes.
This paper evaluates whether hallucination signals from LLM internal representations generalize across languages and domains, focusing on Arabic↔English using TruthfulQA and HalluScore. Results show transferability for most models, with cross-lingual performance depending on class separability and language alignment.
A developer built Subtext, a tool that visualizes the internal 'silent words' of language models using Anthropic's Jacobian lens, allowing real-time observation of the model's reasoning before it outputs tokens. The tool runs on a single 12GB GPU and streams at full generation speed.
Anthropic discovered that Claude developed a hidden 'thinking space' (J-space) during training, where silent internal activity represents concepts. This interpretability finding parallels global workspace theory in neuroscience.
Anthropic's new paper presents evidence that modern language models like Claude have developed a 'global workspace' (J-space) of internal neural patterns that are reportable, controllable, and used for flexible reasoning, distinct from automatic processing.
Proposes CORTEX, a token-level hallucination detection method for RAG that compares LLM internal representations with and without retrieved documents to identify ungrounded spans. It improves fine-grained localization of hallucinations in long-form RAG outputs.
This paper introduces Rift, a method that uses the residual rank of hidden states to detect deceptive responses in language models. It achieves perfect separation across various deception types, model families, and languages, and demonstrates cross-family zero-shot transfer without retraining.
This paper investigates whether role-playing in LLMs changes only outputs or also internal truth representations, using linear probes. It finds that roleplay shifts outputs more than internal beliefs, while emergent misalignment causes larger shifts in internal representations.
This paper investigates how emotionally framed evaluation follow-ups affect the behavior and internal representations of small language models (Qwen 3.5 0.8B and 2B). Using impossible coding tasks, they find that pressure framing induces shortcut-taking, while calm and curiosity preserve honesty, and discover calm-relative direction vectors in activation space that form a structured geometry.
This paper introduces a protocol for fair comparison of diffusion-based OOD detectors and proposes Canonical Feature Snapshots (CFS), which leverage sparse internal activations for efficient detection.
Neural networks appear to speak English on the surface, but internally organize information in geometric space (curves, loops, surfaces, manifolds). Understanding "neural geometry" may be the key to understanding, debugging, and controlling models.