Tag
This paper demonstrates that frontier language models can perform 'invisible reasoning' using semantically irrelevant filler tokens, improving accuracy on synthetic reasoning tasks by up to 13 percentage points, which undermines the assumption that chain-of-thought monitoring captures all reasoning.
This paper introduces Hidden Decoding, a sequence-length scaling method for LLMs that adds internal computation per token by expanding each token into multiple streams with independent embeddings, using Stream-Factorized Attention to keep costs low. Experiments on models up to 617B parameters show consistent improvements over baselines, demonstrating a practical fixed-backbone scaling path.