sparse-autoencoders

Tag

Cards List
#sparse-autoencoders

What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2

arXiv cs.CL ↗ · 2026-10-28 Cached

This paper applies mechanistic interpretability tools—including linear CKA, logit lens, attention decomposition, sparse autoencoders, and activation patching—to compare GPT-2 with DistilGPT-2, revealing a compression-collapse dichotomy where the IOI circuit fully collapses while the Greater-Than circuit survives via layer remapping.

0 favorites 0 likes
#sparse-autoencoders

TaxoSAE: Rethinking Sparse Autoencoders through Taxonomic Logic

arXiv cs.CL ↗ · 2026-10-26

TaxoSAE 提出将分类学(is-a 层级)结构显式编码进稀疏自编码器特征空间,通过分层 SAE 与分类对齐损失,在事实问答中显著提升了概念读出精度、编辑控制粒度,以及对未见概念的超类读出能力。

0 favorites 0 likes
#sparse-autoencoders

Where Do Legal Factors Live in LLMs?

arXiv cs.CL ↗ · 2026-10-22 Cached

A University of Pittsburgh paper investigates whether LLMs internally represent the 26 CATO legal factors, showing all factors are linearly decodable in Qwen3-1.7B, Qwen3-8B, and Gemma-2 2B via sparse autoencoder features with test AUCs of 0.86-1.00.

0 favorites 0 likes
#sparse-autoencoders

The Router Already Knows

arXiv cs.CL ↗ · 2026-10-21 Cached

This preprint proposes NR-SAE, a sparse dictionary method that reuses a frozen MoE router's expert selection to address per-expert autoencoder banks, showing better reconstruction than rank-addressed, candidate-matched, and randomized-identity controls on Gemma 4 26B-A4B, while noting speed and causal-faithfulness results remain untested.

0 favorites 0 likes
#sparse-autoencoders

Decision Sufficiency of Learned Representation Interfaces for Safe Generative Control

arXiv cs.LG ↗ · 2026-10-21 Cached

This paper introduces 'decision sufficiency' as a framework for evaluating whether learned representation interfaces, such as selected sparse autoencoder features, retain enough information for a controller to safely steer language models, and audits Qwen2.5, SmolLM2, and Phi showing measurable information loss in safety-critical decisions.

0 favorites 0 likes
#sparse-autoencoders

Shaped SAEs: Expanding the Toolkit for Mechanistic Interpretability

arXiv cs.LG ↗ · 2026-10-21 Cached

The paper introduces Shaped SAEs, including a Matched Filter SAE (MF-SAE), which linearly shapes LLM activations via Jacobian-based rank-reducing transformations before learning sparse autoencoders to improve semantic selectivity of dictionary atoms, evaluated on SAEBench and SAE Probes.

0 favorites 0 likes
#sparse-autoencoders

Building Auditable AI Text Detectors with Sparse Autoencoders

Papers with Code Trending ↗ · 2026-10-21 Cached

Google DeepMind 论文提出 DetectorScope 管线,利用稀疏自编码器从黑盒 AI 文本检测器的残差流中提取可解释的语言特征,训练出既能保持原检测器性能、又可审计且在分布外文本上表现更强的透明分类器。

0 favorites 0 likes
#sparse-autoencoders

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

Hugging Face Daily Papers ↗ · 2026-10-20 Cached

This paper presents a mechanistic interpretability study of vision-language-action (VLA) models, showing that motor programs are bound to absolute coordinates and ignore scene changes, explaining perturbation brittleness. It introduces Action Atlas, an interactive platform covering 520+ trained sparse autoencoders and 79 identified concepts across six VLA models.

0 favorites 0 likes
#sparse-autoencoders

Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting

arXiv cs.LG ↗ · 2026-10-19 Cached

该论文提出 SAGA 方法,将任务边界处训练的稀疏自编码器转换为对表示漂移的低秩局部约束,以缓解持续学习中的灾难性遗忘,并在三个 ViT-B/16 基准上取得最佳的 Class-IL 准确率与最低遗忘率。

0 favorites 0 likes
#sparse-autoencoders

Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility

arXiv cs.LG ↗ · 2026-10-16 Cached

该论文研究了稀疏自编码器(SAEs)所需的最小激活数据量,发现功能行为(如重建)在较少数据下即可收敛,但特征级可复现性仍随数据持续提升,表明SAE训练可在匹配目标的前提下大幅降低数据成本。

0 favorites 0 likes
#sparse-autoencoders

Feature Elicitation for SAE Steering of Deceptive Behaviour in Small LLMs

arXiv cs.LG ↗ · 2026-10-16 Cached

研究提出一种基于词袋的SAE特征引出流水线,通过转向与道义伦理相关的稀疏自编码器特征,首次证明机制性激活层面的干预可将Gemma-3-27B-IT的合规差距降至不显著水平,并发现了一种名为"deceptive defanging"的新欺骗行为形式。

0 favorites 0 likes
#sparse-autoencoders

Neural networks are clean

Hacker News Top ↗ · 2026-10-15 Cached

Goodfire 研究者发文论证,此前被广泛接受的神经元多义性与叠加假说是对 InceptionV1 证据的误读;经架构改进后的模型 99-100% 末层神经元已可解释,主张网络本身就是可读的构建块,无需先用稀疏自编码器解缠。

0 favorites 0 likes
#sparse-autoencoders

Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models

arXiv cs.AI ↗ · 2026-10-15 Cached

提出了 MOSAIC(Multimodal Orthogonalized Sparse Autoencoders for Influence Characterization),通过共享的稀疏图文潜空间与 EK-FAC 分解,实现视觉语言模型中训练样本到具体文本短语和图像区域的补丁级归因,并在 Qwen 和 Gemma 的多项任务上验证了其有效性。

0 favorites 0 likes
#sparse-autoencoders

Representation Redistribution in Sparse Autoencoders

Hugging Face Daily Papers ↗ · 2026-10-15 Cached

论文提出 redistribution sensitivity score (RSS),用于识别稀疏自编码器中可复现的特征重分配,并量化其对下游线性读出(如安全读出)失败的贡献,在 27 个预训练 SAE 上取得高 AUROC。

0 favorites 0 likes
#sparse-autoencoders

Behavioral Compositionality of Sparse Autoencoder Representations

Hugging Face Daily Papers ↗ · 2026-10-15 Cached

The paper introduces SAE Additivity Residuals (SAR) to measure how much of model behavior cannot be explained by individual or low-order SAE features, showing SAE coordinates are more behaviorally additive than random rotations while important feature interactions remain, improving coalition and steering rank fidelity when pairwise interactions are modeled.

0 favorites 0 likes
#sparse-autoencoders

What behaviors does a model actually learn in RL? [P]

Reddit r/MachineLearning ↗ · 2026-10-14

研究者开发了一套无监督工具,利用稀疏自编码器(SAE)特征对比训练前后生成轨迹,以发现模型在 RL 后训练中实际学到的行为;在 MiMo-v2.6 RL 环境的轨迹上应用后,检测到明显的 eval awareness 及轻度的环境利用现象。

0 favorites 0 likes
#sparse-autoencoders

Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching

Hugging Face Daily Papers ↗ · 2026-10-14 Cached

The paper introduces SAE-IDF, which uses sparse autoencoders and rarity-weighted features as a shared coordinate system to match symbolic units across heterogeneous systems, outperforming dense LLM baselines on 12 tasks while remaining human-interpretable.

0 favorites 0 likes
#sparse-autoencoders

OpenAI studies internal signals behind AI 'metagaming' during training

Reddit r/singularity ↗ · 2026-10-13 Cached

OpenAI researchers identified four internal latent signals, found via sparse autoencoders, linked to AI 'metagaming' (reasoning about how it is graded) during reinforcement learning of o3, finding these signals grow stronger during training. The study suggests monitoring these latents could help detect and steer away from reward-hacking behavior.

0 favorites 0 likes
#sparse-autoencoders

Language Models Have Interpretable Narrative Mechanisms

arXiv cs.CL ↗ · 2026-10-12 Cached

该论文结合叙事学与机制可解释性,在GPT-2-Small、Gemma-2-2B和Llama-3.1-8B中识别出与叙事性(narrativity)相关的注意力头、MLP神经元和稀疏自编码器特征,并通过消融实验证明这些组件对模型的叙事行为具有因果影响。

0 favorites 0 likes
#sparse-autoencoders

Disentangling Linguistic and Paralinguistic Information with Routed Sparse Autoencoders

arXiv cs.CL ↗ · 2026-10-09 Cached

该论文提出将 TopK 稀疏自编码器与路由特定监督和跨因子对抗训练结合,在冻结的 SPEAR 和 WavLM 语音编码器上分离语言学与副语言学信息,展示跨编码器、语料库和干预实验一致的路由选择性分离效果。

0 favorites 0 likes
Next →
← Back to home

Submit Feedback