Tag
This paper introduces a method to control recurrent dynamics in neural networks for reliable test-time depth, analyzing dynamical regimes like settling, marginal, or drifting to improve performance on algorithmic tasks such as Sudoku and carry propagation.
This paper introduces Bifocal Attention, which decouples positional encoding into geometric (standard RoPE) and spectral (learnable harmonic operators) modalities to address the 'Spectral Rigidity' of fixed RoPE, improving algorithmic generalization beyond the training window.
This paper introduces RecurrReason, a difficulty-controlled benchmark of four symbolic logic puzzles to evaluate multi-step reasoning in sequence models. Fine-tuning experiments on T5 and GPT-2 show that architecture determines success more than scale, and that pre-training transfer depends on local transition structure.
This paper introduces PyLang, a programming language absent from all pretraining corpora, and shows that LLMs fine-tuned on it can learn syntax but fail to transfer algorithmic reasoning, resulting in an 'implementation fidelity gap' where models understand algorithms but cannot express them in an unfamiliar language.