Fractional State Space Transition for Long Sequence Modeling
Summary
FRAC是基于分数阶动力学的新型选择性状态空间模型架构,用幂律长记忆替代传统SSM的指数遗忘,通过log间距指数模式和近似实现高效循环模块,并在1.3B参数语言模型上显著提升长上下文性能。
View Cached Full Text
Cached at: 09/30/26, 08:18 PM
Paper page - Fractional State Space Transition for Long Sequence Modeling
Source: https://huggingface.co/papers/2609.36314
Abstract
StateSpaceModels(SSMs)compresssequencehistoryintoaboundedrecurrentstate,makingtheresultingmemorylawacentralarchitecturalchoiceforlong-contextperformance.MostmodernSSMsrelyonODE-baseddynamicsthatleadtoexponentialforgetting,limitingtheirabilitytoretaininformationoverbroadtemporalranges.WeintroduceFRAC,aselectiveSSMarchitecturederivedfromfractionaldynamicsthatreplacesthisexponentialdecaywithpower-lawlongmemory.Tomakefractionaldynamicspractical,FRACapproximatestheheavy-tailedtargetkernelwithafinite-state,log-spacedsumofexponentialmodes.Thisconstructionturnsfractionalmemoryintoanefficientrecurrentmodulewithparalleltrainingandprefill,whileretainingbounded-stateautoregressivedecoding.Extensiveexperiments,including1.3B-parameterlanguagemodeling,demonstratethatFRACconsistentlyimproveslong-contextperformanceoverstate-of-the-artSSMbaselineswhilestayingcompetitiveonshort-context.Theseresultsshowthatfractionaldynamicsprovideapracticalandeffectivepriorforlong-contextSSMs.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2609\.36314
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.36314 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.36314 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.36314 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
The Cost of Long Memory: State, Context, and Stability Complexity in Sequence Models
本文研究序列模型中的长程记忆资源问题:给定预测记忆律,证明了指数与有限状态模式在预测风险下的匹配上下界,得到达到误差τ所需的 Θ(log²(1/τ)) 状态复杂度定律,并分析了分数长记忆、非线性递归的临界性条件及其与状态空间、门控循环和注意力模型的实验验证。
TIDES: Implicit Time-Awareness in Selective State Space Models
TIDES 提出一种选择性 SSM 变体,将输入依赖性从步长移到对角状态矩阵上,从而在保持每 token 表达能力的同时原生处理不规则时间戳,并在 UE 分类与 PhysiomeODE 回归基准上取得最佳平均排名。
Time Delay System for Sequence Modeling
该论文提出一种基于非线性时间延迟系统的序列建模框架,通过可微语义延迟估计器、对角状态空间的闭式 O(1) 解以及可解释的模糊语义调制机制,在文本分类和生成任务上超越了现有最先进方法。
SpecFold: Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models
SpecFold 论文提出一种算法-系统协同设计,通过识别并折叠多分支投机验证中的计算冗余(token级残差门控与折叠注意力/FFN复用),在扩散大语言模型上实现最高1.64x吞吐提升(相比Spiffy),且保持任务性能相当。
An Input-Dependent Readout Phase for Resonant Sequence Models
该论文提出让谐振序列模型(ResonatorLM)的读出相位依赖当前 token,在保持 FFT 卷积训练路径的同时引入输入依赖性,使在 256 token 训练、1024 token 测试时困惑度从 12.81 降至 3.76,显著改善长度泛化。