sequence-modeling

Tag

Cards List
#sequence-modeling

Kernel Ridge Attention: Nonlinear Memory with Selective Forgetting

arXiv cs.LG ↗ · yesterday Cached

Kernel Ridge Attention (KRA) is a recurrent sequence mixer that uses recursive ridge regression with covariance-guided memory updates and an Ornstein-Uhlenbeck transition for selective forgetting, achieving constant-size inference state and improved language-modeling performance over attention and recurrent baselines.

0 favorites 0 likes
#sequence-modeling

HuggingFaceBio/Carbon-X (DNA understanding and generation, Qwen 3 finetune)

Reddit r/LocalLLaMA ↗ · 2026-10-22 Cached

HuggingFaceBio 发布开源 14.8B DNA 理解与生成模型 Carbon-X,基于 Qwen3-14B 微调,融合 DNA 序列与自然语言训练,在五项 DNA 基准上平均得分 66.7%,接近前沿闭源模型但推理成本约为其 1/1800。

0 favorites 0 likes
#sequence-modeling

Motion Equivariant Convolutional LSTM: When Motion Becomes a Gauge Symmetry

arXiv cs.LG ↗ · 2026-10-22 Cached

提出了运动等变卷积 LSTM(MELSTM),将序列数据中的运动视为规范对称性(gauge symmetry),通过速度模型作为规范场实现对任意运动的可证明等变性,在未来帧预测、运动数字分类和动作识别任务上以更少状态超越 ConvLSTM 与流量等变基线。

0 favorites 0 likes
#sequence-modeling

First Passage Attention

Hugging Face Daily Papers ↗ · 2026-10-15 Cached

提出 First Passage Attention(FPA),用路径极值替代单调衰减累积,从而在压缩 KV 缓存的同时实现完美的长程检索;在 110M 语言模型上以约 25% 缓存达到 8 倍训练长度的 passkey 检索。

0 favorites 0 likes
#sequence-modeling

Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]

Reddit r/MachineLearning ↗ · 2026-10-14

作者介绍 Event2Vec,一个通过自监督下一个事件预测学习离散事件序列加性嵌入的小型库,使任意线性评分可精确分解为逐事件项之和,并支持双曲、球面及非交换几何空间;论文见 arXiv 2509.12188,代码以 MIT 协议开源。

0 favorites 0 likes
#sequence-modeling

Time Delay System for Sequence Modeling

arXiv cs.CL ↗ · 2026-10-12 Cached

该论文提出一种基于非线性时间延迟系统的序列建模框架,通过可微语义延迟估计器、对角状态空间的闭式 O(1) 解以及可解释的模糊语义调制机制,在文本分类和生成任务上超越了现有最先进方法。

0 favorites 0 likes
#sequence-modeling

One-Step Text Generation by Seq-Drifting

arXiv cs.AI ↗ · 2026-10-12 Cached

This paper proposes Seq-Drifting, a framework that learns a map from Gaussian noise to text in a single forward pass by applying drifting to sequences of continuous token representations, achieving a generative perplexity of 36.01 on LM1B with large speedups over few-step and multi-step diffusion language models.

0 favorites 0 likes
#sequence-modeling

Learning infinite context windows in recurrent architectures via spatial neural computing

arXiv cs.LG ↗ · 2026-10-09 Cached

该论文提出了一种由离散化偏微分方程驱动的空间场替代传统神经元间通信的二阶循环模型,实现等价于无限阶RNN的无界感受野,并通过边缘稳定性约束消除梯度消失与爆炸问题,在长序列基准上以更少参数超越其他循环模型。

0 favorites 0 likes
#sequence-modeling

MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models

arXiv cs.LG ↗ · 2026-10-08 Cached

论文提出 MaRK(Markov-adapted Recurrent Kernels),一种将上下文向量直接映射为冻结 SSM 递归参数有界调制的动态算子条件化框架,用于将预训练 SSM 适配到迭代扩散生成任务,在 111M Hydra 骨干上仅需 6.3–11M 可训练参数,其中 Chebyshev 变体表现最佳。

0 favorites 0 likes
#sequence-modeling

Empirical Variational Autoencoder

Hugging Face Daily Papers ↗ · 2026-10-05 Cached

Empirical Variational Autoencoder (EVA) is a general generative framework for continuous-valued sequences that learns autoregressive latent priors empirically from training data, alleviating the prior-posterior gap of conventional VAEs and achieving competitive generation quality with autoregressive diffusion baselines at much faster inference speed.

0 favorites 0 likes
#sequence-modeling

Beyond Diagonal State Space Models: Exact Non-Abelian Group Tracking, Solvability Barriers, and Geometric Physical Manifolds

arXiv cs.LG ↗ · 2026-10-03 Cached

提出 NC-SSM、SO(3)-SSM 和 Cayley-SSM,将状态空间模型的状态转移提升到非交换紧李群上,以精确跟踪非阿贝尔群并规避对角 SSM 的代数局限,在多项实验中达到 100% 群跟踪准确率。

0 favorites 0 likes
#sequence-modeling

SMat-Attention: Structured Long-Context Sequence Modeling

arXiv cs.AI ↗ · 2026-09-30 Cached

SMat-Attention introduces structured causal masks with tunable VC-dimension to bridge softmax attention and linear attention, achieving subquadratic prefill, constant-time decoding with O(T^{1-1/d}) cached states, and improved recall over Mamba-2 and Gated DeltaNet backbones.

0 favorites 0 likes
#sequence-modeling

Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling

Hugging Face Daily Papers ↗ · 2026-09-29 Cached

The paper introduces triadic linear attention, which generalizes linear attention by writing a triadic outer product of two keys and a value into a third-order tensor state, enabling an E-fold state-size increase with only two extra projections. Applied to Gated DeltaNet and scalar-gated linear attention, it improves long-context language modeling and recall, outperforming alternative state-enlargement methods.

0 favorites 0 likes
#sequence-modeling

Fractional State Space Transition for Long Sequence Modeling

Hugging Face Daily Papers ↗ · 2026-09-28 Cached

FRAC是基于分数阶动力学的新型选择性状态空间模型架构,用幂律长记忆替代传统SSM的指数遗忘,通过log间距指数模式和近似实现高效循环模块,并在1.3B参数语言模型上显著提升长上下文性能。

0 favorites 0 likes
#sequence-modeling

Pruned CTC for Memory-Efficient Large-Vocabulary ASR Training

Hugging Face Daily Papers ↗ · 2026-09-27 Cached

This paper introduces Pruned CTC, a method that reduces memory usage in Connectionist Temporal Classification training for large-vocabulary automatic speech recognition by restricting alignment computation, proving equivalence to full-vocabulary CTC and demonstrating strong performance with LLM adaptations in offline and streaming settings.

0 favorites 0 likes
#sequence-modeling

BoundInk: Boundary-Aware Online Handwriting Generation

Hugging Face Daily Papers ↗ · 2026-09-20 Cached

BoundInk introduces a writer-conditioned framework for online handwriting generation that explicitly models inter-character boundaries to improve cursive continuity and spatial alignment, outperforming existing methods with significant reductions in error and high human preference.

0 favorites 0 likes
#sequence-modeling

ALPHABET: A Laplace-Pole History Aggregator with Banked Exponential Transport

arXiv cs.LG ↗ · 2026-08-26 Cached

ALPHABET is a compact linear-time sequence model that compresses temporal history into stable complex pole modes, achieving competitive performance with fewer parameters and faster inference.

0 favorites 0 likes
#sequence-modeling

WorldToken: Time-First Sequence Modeling for Robotic Imitation Learning

Hugging Face Daily Papers ↗ · 2026-08-23 Cached

WorldToken presents a time-first sequence modeling method for robotic imitation learning, using causal Transformers and diffusion action heads to fuse heterogeneous observations, with performance evaluations on RoboCasa and RMBench benchmarks.

0 favorites 0 likes
#sequence-modeling

Mechanistic Interpretability of Structure-Aware Numerical Reasoning in LLaMA 3.1 8B

arXiv cs.LG ↗ · 2026-08-20 Cached

This paper uses mechanistic interpretability to analyze how LLaMA 3.1 8B models numerical sequences, revealing that it internally computes and stores first differences to extrapolate patterns.

0 favorites 0 likes
#sequence-modeling

The Quantum Shortcut: Complex Phase-State Dynamics Reduce the Optimization Steps of Sequence Models

arXiv cs.LG ↗ · 2026-08-18 Cached

The paper explores using complex-valued states inspired by quantum theory in sequence models, showing it can reduce optimization steps in Mamba and attention-based models with better sample efficiency.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback