Tag
A new transformer variant, the full-bandwidth transformer, feeds back top-layer hidden states through a gated linear unit to improve reasoning and efficiency without altering the core architecture. Trained up to 400B tokens, it matches standard transformers trained with 1.5x more data while producing shorter reasoning traces.