Tag
This paper critiques conventional depth truncation methods for evaluating recursive language models and introduces the Depth Control Protocol (DCP) to disentangle and isolate factors affecting depth utilization, improving evaluation accuracy.
RecurrentGPT introduces a recurrent depth transformer that uses gated modulation to iteratively reuse shared layers, achieving competitive accuracy with fewer parameters and improved memory efficiency compared to standard transformers.