layer-routing

MACRO is a framework that learns task-specific execution routes over frozen LLM layers using Markov chain-based routing, improving reasoning accuracy without modifying model weights. It outperforms prior routing approaches while reducing search time significantly.

本文介绍了一种问题级审计框架，用于区分LLM基准测试中的“已实现”答案与“可达”答案，表明总体得分提升往往来自生成已经可达的答案，而非扩展真实能力，并且在匹配预算下，随机层路由与结构化搜索表现相当。

Delta Attention Residuals 通过关注特征变化（增量）而非累积隐藏状态，改进了Transformer模型中的逐层路由，在220M到7.6B参数的规模上实现了1.7-8.2%的验证困惑度提升。