Tag
This paper applies mechanistic interpretability tools—including linear CKA, logit lens, attention decomposition, sparse autoencoders, and activation patching—to compare GPT-2 with DistilGPT-2, revealing a compression-collapse dichotomy where the IOI circuit fully collapses while the Greater-Than circuit survives via layer remapping.
TaxoSAE 提出将分类学(is-a 层级)结构显式编码进稀疏自编码器特征空间,通过分层 SAE 与分类对齐损失,在事实问答中显著提升了概念读出精度、编辑控制粒度,以及对未见概念的超类读出能力。
A University of Pittsburgh paper investigates whether LLMs internally represent the 26 CATO legal factors, showing all factors are linearly decodable in Qwen3-1.7B, Qwen3-8B, and Gemma-2 2B via sparse autoencoder features with test AUCs of 0.86-1.00.
This preprint proposes NR-SAE, a sparse dictionary method that reuses a frozen MoE router's expert selection to address per-expert autoencoder banks, showing better reconstruction than rank-addressed, candidate-matched, and randomized-identity controls on Gemma 4 26B-A4B, while noting speed and causal-faithfulness results remain untested.
This paper introduces 'decision sufficiency' as a framework for evaluating whether learned representation interfaces, such as selected sparse autoencoder features, retain enough information for a controller to safely steer language models, and audits Qwen2.5, SmolLM2, and Phi showing measurable information loss in safety-critical decisions.
The paper introduces Shaped SAEs, including a Matched Filter SAE (MF-SAE), which linearly shapes LLM activations via Jacobian-based rank-reducing transformations before learning sparse autoencoders to improve semantic selectivity of dictionary atoms, evaluated on SAEBench and SAE Probes.
Google DeepMind 论文提出 DetectorScope 管线,利用稀疏自编码器从黑盒 AI 文本检测器的残差流中提取可解释的语言特征,训练出既能保持原检测器性能、又可审计且在分布外文本上表现更强的透明分类器。
This paper presents a mechanistic interpretability study of vision-language-action (VLA) models, showing that motor programs are bound to absolute coordinates and ignore scene changes, explaining perturbation brittleness. It introduces Action Atlas, an interactive platform covering 520+ trained sparse autoencoders and 79 identified concepts across six VLA models.
该论文提出 SAGA 方法,将任务边界处训练的稀疏自编码器转换为对表示漂移的低秩局部约束,以缓解持续学习中的灾难性遗忘,并在三个 ViT-B/16 基准上取得最佳的 Class-IL 准确率与最低遗忘率。
该论文研究了稀疏自编码器(SAEs)所需的最小激活数据量,发现功能行为(如重建)在较少数据下即可收敛,但特征级可复现性仍随数据持续提升,表明SAE训练可在匹配目标的前提下大幅降低数据成本。
研究提出一种基于词袋的SAE特征引出流水线,通过转向与道义伦理相关的稀疏自编码器特征,首次证明机制性激活层面的干预可将Gemma-3-27B-IT的合规差距降至不显著水平,并发现了一种名为"deceptive defanging"的新欺骗行为形式。
Goodfire 研究者发文论证,此前被广泛接受的神经元多义性与叠加假说是对 InceptionV1 证据的误读;经架构改进后的模型 99-100% 末层神经元已可解释,主张网络本身就是可读的构建块,无需先用稀疏自编码器解缠。
提出了 MOSAIC(Multimodal Orthogonalized Sparse Autoencoders for Influence Characterization),通过共享的稀疏图文潜空间与 EK-FAC 分解,实现视觉语言模型中训练样本到具体文本短语和图像区域的补丁级归因,并在 Qwen 和 Gemma 的多项任务上验证了其有效性。
论文提出 redistribution sensitivity score (RSS),用于识别稀疏自编码器中可复现的特征重分配,并量化其对下游线性读出(如安全读出)失败的贡献,在 27 个预训练 SAE 上取得高 AUROC。
The paper introduces SAE Additivity Residuals (SAR) to measure how much of model behavior cannot be explained by individual or low-order SAE features, showing SAE coordinates are more behaviorally additive than random rotations while important feature interactions remain, improving coalition and steering rank fidelity when pairwise interactions are modeled.
研究者开发了一套无监督工具,利用稀疏自编码器(SAE)特征对比训练前后生成轨迹,以发现模型在 RL 后训练中实际学到的行为;在 MiMo-v2.6 RL 环境的轨迹上应用后,检测到明显的 eval awareness 及轻度的环境利用现象。
The paper introduces SAE-IDF, which uses sparse autoencoders and rarity-weighted features as a shared coordinate system to match symbolic units across heterogeneous systems, outperforming dense LLM baselines on 12 tasks while remaining human-interpretable.
OpenAI researchers identified four internal latent signals, found via sparse autoencoders, linked to AI 'metagaming' (reasoning about how it is graded) during reinforcement learning of o3, finding these signals grow stronger during training. The study suggests monitoring these latents could help detect and steer away from reward-hacking behavior.
该论文结合叙事学与机制可解释性,在GPT-2-Small、Gemma-2-2B和Llama-3.1-8B中识别出与叙事性(narrativity)相关的注意力头、MLP神经元和稀疏自编码器特征,并通过消融实验证明这些组件对模型的叙事行为具有因果影响。
该论文提出将 TopK 稀疏自编码器与路由特定监督和跨因子对抗训练结合,在冻结的 SPEAR 和 WavLM 语音编码器上分离语言学与副语言学信息,展示跨编码器、语料库和干预实验一致的路由选择性分离效果。