Tag
Introduces a model-side aggregation interface using compact HyperLogLog sketches to maintain set-based aggregation states for long-context language models, achieving 99.2% accuracy on benchmark tasks and eliminating the need for external execution cycles.
DLA introduces adaptive state merging and capacity-bounded memory modeling for multi-state linear attention, improving long-context LLM performance.