Tag
SAS introduces a gated sparse attention mechanism that optimizes context ranking end-to-end with language modeling loss, improving performance in reasoning and long-context tasks under tight attention budgets.
This paper explores the synergy between visual understanding and generation in unified multimodal models, showing that task-decoupled architectures and end-to-end optimization can enhance performance by turning coexistence into synergy.
Researchers from MIT and Harvard introduce Role Anchor, a technique to mitigate role drift in compound AI systems by forcing modules to adhere to their assigned roles during end-to-end optimization, as terminal accuracy can hide underlying failures.