Tag
HuggingFaceBio 发布开源 14.8B DNA 理解与生成模型 Carbon-X，基于 Qwen3-14B 微调，融合 DNA 序列与自然语言训练，在五项 DNA 基准上平均得分 66.7%，接近前沿闭源模型但推理成本约为其 1/1800。
提出了运动等变卷积 LSTM（MELSTM），将序列数据中的运动视为规范对称性（gauge symmetry），通过速度模型作为规范场实现对任意运动的可证明等变性，在未来帧预测、运动数字分类和动作识别任务上以更少状态超越 ConvLSTM 与流量等变基线。
提出 First Passage Attention（FPA），用路径极值替代单调衰减累积，从而在压缩 KV 缓存的同时实现完美的长程检索；在 110M 语言模型上以约 25% 缓存达到 8 倍训练长度的 passkey 检索。
作者介绍 Event2Vec，一个通过自监督下一个事件预测学习离散事件序列加性嵌入的小型库，使任意线性评分可精确分解为逐事件项之和，并支持双曲、球面及非交换几何空间；论文见 arXiv 2509.12188，代码以 MIT 协议开源。
该论文提出一种基于非线性时间延迟系统的序列建模框架，通过可微语义延迟估计器、对角状态空间的闭式 O(1) 解以及可解释的模糊语义调制机制，在文本分类和生成任务上超越了现有最先进方法。
This paper proposes Seq-Drifting, a framework that learns a map from Gaussian noise to text in a single forward pass by applying drifting to sequences of continuous token representations, achieving a generative perplexity of 36.01 on LM1B with large speedups over few-step and multi-step diffusion language models.
该论文提出了一种由离散化偏微分方程驱动的空间场替代传统神经元间通信的二阶循环模型，实现等价于无限阶RNN的无界感受野，并通过边缘稳定性约束消除梯度消失与爆炸问题，在长序列基准上以更少参数超越其他循环模型。
论文提出 MaRK（Markov-adapted Recurrent Kernels），一种将上下文向量直接映射为冻结 SSM 递归参数有界调制的动态算子条件化框架，用于将预训练 SSM 适配到迭代扩散生成任务，在 111M Hydra 骨干上仅需 6.3–11M 可训练参数，其中 Chebyshev 变体表现最佳。
Empirical Variational Autoencoder (EVA) is a general generative framework for continuous-valued sequences that learns autoregressive latent priors empirically from training data, alleviating the prior-posterior gap of conventional VAEs and achieving competitive generation quality with autoregressive diffusion baselines at much faster inference speed.
提出 NC-SSM、SO(3)-SSM 和 Cayley-SSM，将状态空间模型的状态转移提升到非交换紧李群上，以精确跟踪非阿贝尔群并规避对角 SSM 的代数局限，在多项实验中达到 100% 群跟踪准确率。
SMat-Attention introduces structured causal masks with tunable VC-dimension to bridge softmax attention and linear attention, achieving subquadratic prefill, constant-time decoding with O(T^{1-1/d}) cached states, and improved recall over Mamba-2 and Gated DeltaNet backbones.
The paper introduces triadic linear attention, which generalizes linear attention by writing a triadic outer product of two keys and a value into a third-order tensor state, enabling an E-fold state-size increase with only two extra projections. Applied to Gated DeltaNet and scalar-gated linear attention, it improves long-context language modeling and recall, outperforming alternative state-enlargement methods.
FRAC是基于分数阶动力学的新型选择性状态空间模型架构，用幂律长记忆替代传统SSM的指数遗忘，通过log间距指数模式和近似实现高效循环模块，并在1.3B参数语言模型上显著提升长上下文性能。
This paper introduces Pruned CTC, a method that reduces memory usage in Connectionist Temporal Classification training for large-vocabulary automatic speech recognition by restricting alignment computation, proving equivalence to full-vocabulary CTC and demonstrating strong performance with LLM adaptations in offline and streaming settings.
BoundInk introduces a writer-conditioned framework for online handwriting generation that explicitly models inter-character boundaries to improve cursive continuity and spatial alignment, outperforming existing methods with significant reductions in error and high human preference.
ALPHABET is a compact linear-time sequence model that compresses temporal history into stable complex pole modes, achieving competitive performance with fewer parameters and faster inference.
WorldToken presents a time-first sequence modeling method for robotic imitation learning, using causal Transformers and diffusion action heads to fuse heterogeneous observations, with performance evaluations on RoboCasa and RMBench benchmarks.
This paper uses mechanistic interpretability to analyze how LLaMA 3.1 8B models numerical sequences, revealing that it internally computes and stores first differences to extrapolate patterns.
The paper explores using complex-valued states inspired by quantum theory in sequence models, showing it can reduce optimization steps in Mamba and attention-based models with better sample efficiency.
This paper introduces Modular TTT, a framework that represents test-time training inner learners as directed acyclic graphs, enabling systematic ablation and composition of components. The authors train 410M and 1.45B parameter models on 100B tokens, achieving performance comparable to GatedDeltaNet.