@HuggingModels: Ever heard of a state space model that runs in linear time and constant memory? Meet OrthoSSM-130M, a Phantom SSM that'…
Summary
OrthoSSM-130M is a new Phantom state space model optimized for linear-time, constant-memory text generation, positioned as a fast and efficient alternative to Mamba.
View Cached Full Text
Cached at: 09/06/26, 08:53 PM
Ever heard of a state space model that runs in linear time and constant memory? Meet OrthoSSM-130M, a Phantom SSM that’s shaking up the text generation scene. It’s like Mamba’s cool cousin, built for speed and efficiency. Dive in! https://t.co/rF7gb7EYxc
Similar Articles
@rshia_afz: 1/ SSMs struggle on recall benchmarks due to their fixed-size state. But are current models actually storing context “w…
The article introduces Raven, a new State Space Model (SSM) with selective memory allocation that achieves state-of-the-art performance on recall tasks and demonstrates superior length generalization compared to existing models like SWA.
ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models
ChronoSSM introduces an autoregressive State Space Model that jointly models events and timestamps, showing that joint training improves temporal recoverability without degrading content generation quality.
Looped State-Space Language Models with Adaptive Exit-State Selection
This paper explores looped (recurrent) state-space language models using Mamba and hybrid Mamba-Transformer backbones, showing they outperform non-looped baselines on reasoning tasks and remain competitive under iso-parameter and iso-FLOPs pretraining, with adaptive exit-state selection improving intermediate-depth performance.
I built a Mamba1 variant I call SM1 with d_state=1 that runs on Blackwell in pure PyTorch [P]
The author presents SM1, a variant of Mamba1 with d_state=1, using two native PyTorch ops to replace the selective scan, reducing memory by 16x compared to d_state=16. The closed-form solution eliminates the state dimension, enabling efficient inference with constant memory per token.
PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift
This paper proposes Physics-Informed Multi-Scale Mamba (PIMSM), a state-space architecture that aligns model memory with physical timescales to improve robustness under distribution shift in scientific time series, demonstrating improvements on fMRI and weather forecasting tasks.