Tag
This paper proposes ODE-based Transformer decoders using Runge-Kutta integration methods to improve iterative refinement for sign language translation, achieving better BLEU scores than the IPSLT baseline without increasing model size.
This research introduces a technique to loop frozen, off-the-shelf transformer checkpoints at inference time by using damped Runge-Kutta substeps, treating transformer layers as Euler steps in a residual ODE. This allows extra latent compute without fine-tuning, architecture changes, or new weights, showing gains on knowledge tasks like MMLU-Pro, GPQA, and ARC.