Tag
Extropic introduces Z1T, a sparse transformer-like model optimized for their probabilistic Z1 hardware to enhance energy efficiency in transformer inference through algorithmic-hardware co-design.
This paper presents Token-Selective Attention (TSA), a differentiable token routing mechanism that learns to skip unnecessary computations per token in transformer layers, reducing token-layer operations by 14–23% with minimal quality loss on language modeling tasks.