causal-attention

Hy-Nomas是一个8B参数的视频世界模型，通过action-aware因果注意力与结构化字幕，支持多轮自由文本指令和连续相机控制的交互式世界生成，并开源了模型权重、代码与基准。

该论文提出因果潜在工作区 Transformer（causal latent workspace Transformer），通过在共享深度的循环块之间加入可读写的前缀专属潜在槽位，提升了循环 Transformer 的语言建模困惑度，并在三个字节级语料上的多数设置中优于五个基线对照。

提出了 Continuum Attention，一种用连续分段线性记忆曲线上的积分替代离散键值查找的因果 Transformer 注意力算子，在语言建模困惑度上优于标准 softmax 与 focal attention。

StreamErase 实现了亚秒级延迟的实时流式视频物体擦除，通过滑动窗口因果注意力与注意力蒸馏将双向模型的擦除能力迁移到单向流式生成器，在单 GPU 上可达 26 FPS，并提出 LongErase 基准。

This article explains in detail why the KV cache stores K and V but not Q in large language models, and discusses prefill, prefix cache, and architectural innovations in models like DeepSeek-V4.1-Flash.

The paper proposes a data-driven constitutive modeling framework using neural operators with causal attention to predict stress response in path-dependent materials, focusing on plasticity and damage, with accurate and parallelizable predictions.

This paper studies the mismatch between sequence locality and attention-graph reachability in fixed block-sparse causal attention, formalizing boundary artifacts and proposing diagnostic coverage functions and a minimal repair called Boundary Bridge Attention.

This paper introduces FFDC, a lightweight verifier for World Action Models that enables adaptive action chunk sizes by checking consistency between predicted and actual observations, improving efficiency and robustness in robotic manipulation.