ablation-study

Nikita Pavlichenko, former head of pretraining for JetBrains Mellum, systematically evaluated LLM architecture changes since 2020 through 1,129 training experiments. He concluded that architectural gains had largely been exhausted by the DeepSeek-V3 generation, with fine-grained MoE and linear attention representing the most substantial advances, while the impact of training methods such as Muon and inference kernels is often underestimated.

该论文研究多智能体 LLM 谈判系统中的三部分控制栈：5 条支柱运行时宪章、4 层 swarm 架构（Director、多数投票、Monitor、schema 硬门控）以及 Cognitive Annealing（确定性死锁检测与原子上下文清除恢复），并针对一个有状态对抗性 Gatekeeper 进行消融实验，发现确定性恢复机制在死锁场景中显著优于纯 LLM 引导。

This paper instruments a minimal FunSearch-style loop with operator packages to test components of proposers in verified search for mathematical construction problems, finding that composition closes the gap and repulsion increases diversity.

This paper presents an exploratory ablation study of TALH, a hybrid language model combining MLA and SSM, showing that SSM integration is more critical for validation performance than MLA in the tested setup, with insights on memory usage and timing on consumer hardware.

The paper introduces MechaTerp-TRACE, a method for component ablation analysis in language models, finding that entity knowledge is largely attributable to generic generation machinery rather than localized components.

The paper proposes a Physics Informed Recurrent Neural Network (PIRNN) that predicts unobservable physical variables to improve time series forecasting in physical processes, demonstrated through groundwater level predictions.

This paper demonstrates that linear probes decoding concepts from language model activations do not necessarily identify causally relevant features, and introduces a feature-level diagnostic using sparse autoencoders to separate probe alignment from behavioral drivers.

This paper empirically studies harness design for coding agents, evaluating components like planning and context management to improve performance in software engineering tasks.

This paper introduces a multi-agent AI system for measuring and diagnosing competitive visibility in LLM-mediated e-commerce using Agentic Share-of-Search, with an ablation study showing feasibility.

This preregistered ablation study tests prompt-level context in a production speech transcription tool and finds no detectable change in side-level word error rate, contradicting earlier reports of gains from prompt conditioning.

This study investigates the benefits of frequency decomposition for Physics-Informed Neural Networks (PINNs) by proposing a dual-branch, spectrally-gated architecture (DBSG-PINN). Ablation experiments on 1D PDE benchmarks indicate that frequency decomposition is most effective on spectrally complex problems, reducing error by up to 59.2%.

This paper proposes ClaimGAT, a Graph Attention Network that encodes patent claim dependencies to predict litigation risk, achieving an AUC-ROC of 0.818 on USPTO utility patents.

This paper formalizes and empirically measures whether semantic retrieval via Language Server Protocol saves tokens for coding agents, finding that benefits are conditional and often negative, suggesting an adaptive tool-routing approach.

A controlled ablation study of LLM self-reflection in conflict forecasting finds that typed action routing drives performance gains, while diagnostic scaffolding and taxonomy vocabulary add no measurable value, with replication on GPT-4o.

This paper shows that four seemingly minor architectural choices—normalization, GQA, pretraining context length, and sliding window attention—have a compoundingly negative effect on long context extensibility, dropping performance by up to 47% when combined. The authors release OlmPool, a set of 26 comparable 7B models, after 170,000 GPU hours of controlled ablations.

Introduces CT-HEG, a continuous-time heterogeneous EHR graph schema for ICU mortality prediction, with an ablation study showing bidirectional connectivity and time-attentive edge features matter; surprisingly, a simplified homogeneous graph outperformed the full heterogeneous model on the MIMIC-IV cohort.

This paper presents a controlled ablation study across Claude Code and Codex, 17 real tasks, and 288 runs, finding that context files like AGENTS.md/CLAUDE.md do not measurably improve correctness; agents fail on implementation skill, not missing repository knowledge.

A pre-registered ablation on a 4B model classification task shows that harness design (rule placement, evidence order, turn structure) swings accuracy from 60% to 82%, demonstrating that poor harness design is often mistaken for poor model capability.

This paper compares cascaded and joint multi-task modeling for hierarchical offensive language detection, finding that cascaded architectures achieve higher accuracy at the cost of increased parameters and inference latency, and that class-imbalance handling strategies should be verified via ablation.

Loopie introduces looped Mixture-of-Experts Transformers that outperform vanilla transformers under the same compute budget, achieving gold-medal performance at the 2025 IMO and IPhO without tools.