ablation

A demo of chessformer_lens shows that ablating a single attention head in a chess transformer causes it to stop recognizing Morphy's queen sacrifice, demonstrating the concentration of specific capabilities in individual heads.

This paper presents a systematic comparative study of KV-cache compression schemes (TurboQuant and SpectralQuant), introduces a statistical validation methodology, and offers regime-specific recommendations for efficient transformer inference.

A new uncensored version of the Nex-N2-mini model, called Nex-N2-mini-ultra-uncensored-heretic, has been released. It achieves 93% fewer refusals while preserving quality with low KL divergence, and is available in safetensors and GGUF formats.

A new contrastive ablation operator called apostate is introduced that reduces model refusal from 96% to 5% while preserving harmless behavior with only 0.081 KL divergence, tested on Granite 3.3-8B.

A researcher shares an experimental plan for identifying causal dependencies between capability dimensions in a 31B model using contrastive targeted SFT and circuit tracing, seeking feedback on methodology and related work.

Tower-Plus-72B-Ultra-Uncensored-Heretic is a decensored version of Unbabel/Tower-Plus-72B, supporting 22 languages and excelling in translation tasks with minimal refusals.

This paper shows that attention heads meeting common criteria for mechanistic role claims (necessity, linear decodability, ablation reversibility) routinely fail to transfer computations across prompts, and introduces the KID (Knowing/Intent/Doing) framework and a three-stage pipeline for more rigorous role assignment.

Skill RSI is a free tool that recursively evaluates and improves AI skills via procedural evaluations and a research agent, supporting standalone or Codex plugin usage.

A blog post from Flyback demonstrates how a LightGBM feature that ranked #1 in importance actually worsened predictions due to target encoding leakage, highlighting the danger of relying solely on feature importance metrics.

This paper investigates how post-training of LLMs introduces AI-like stylistic regularities and proposes PASTA, a training-free method to localize and ablate these alignment signatures, reducing AI detection rates while maintaining coherence across 11 models and 6 detectors.

Nous Research released Contrastive Neuron Attribution (CNA), a method to steer LLM behavior by identifying and ablating sparse circuits in MLP neurons without training sparse autoencoders or degrading general benchmarks, validated on multiple large language models.