mamba

The article compares the routes of two AI model architecture camps, Mamba and RWKV: Mamba targets cloud-side enterprise AI platforms, while RWKV targets on-device and edge devices, each adapting to different hardware and inference scenarios.

LinearKV is a training-free framework that enables position-independent caching for hybrid LLMs by using a single cached state to initialize linear layers, outperforming exact prefix-state composition and remaining compatible with existing PIC methods.

Introduces AraSSM, a bidirectional Mamba state-space encoder pretrained via masked language modeling on Arabic corpora, achieving competitive results on Arabic NLU benchmarks while training from scratch on consumer GPUs.

NVIDIA released Nemotron 3.5 Lightning 30B-A3B-NVFP4, a hybrid MoE LLM with 3B active parameters, up to 1M context, and speculative decoding support for efficient single-GPU inference.

A developer trained a tiny Mamba-based autoregressive language model and built an inference engine to run it on the 6502 processor inside a BBC Micro, demonstrating modern machine learning on 1975 hardware with only 25KB of user memory.

Proposes MaLoRA and MaRA, two adapter families that introduce selective state-space recurrence for token-level and context-level adaptation in frozen language models, achieving significant gains on multi-hop reasoning benchmarks like MuSiQue and 2WikiMultihopQA.

The paper proposes MambaLSTM, a framework combining Mamba state-space models and LSTM for spatio-temporal traffic accident risk prediction, addressing noise in feature fusion and global spatial correlation.

Introduces AdaSurvMamba, an adaptive framework for multimodal survival analysis that uses a dual-scale importance-aware reconstruction module and semantic aggregation scanning to improve integration of whole slide images and genomic profiles, achieving consistent gains across five TCGA cohorts.

This paper proposes Samba, a hybrid state-space architecture for audio-visual navigation that uses a Mamba State Encoder to replace GRUs and an Audio Mamba Encoder to better capture global time-frequency dependencies, achieving an 11.3% improvement in navigation success rate on the Matterport3D dataset.

German AI consortium releases Soofi S, an open 30B mixture-of-experts model that achieves top scores on English and German benchmarks, using a hybrid Mamba-Transformer architecture for efficient long-context inference.

This paper proposes a hybrid Mamba-attention architecture for block diffusion language models that restricts reverse Mamba scans to the active denoising block, enabling exact caching across blocks and achieving high throughput for long-context generation.

QuantFlow introduces a federated Mamba-based foundation model for time-series forecasting that combines inverted sequence embedding, bidirectional state-space decoders, and quantile regression to achieve strong results on benchmarks while preserving data privacy.

This paper evaluates the Mamba state space model for ASR on seven South African languages, finding it matches Conformer accuracy with fewer resources, and explores multilingual training strategies and low-resource settings.

This paper proposes a self-supervised Mamba-based model to learn effective representations from electronic health records for improved patient subtyping, demonstrating better performance than baseline models on real-world datasets.

NVIDIA's Nemotron-3-Super-120B-A12B, a hybrid Mamba and mixture-of-experts model, achieves perfect needle-in-haystack retrieval at 504K tokens using only four RTX 3090 GPUs.

NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B, a compressed version of Nemotron-3-Super with improved inference efficiency and strong benchmark performance.

SPDM introduces a geometry-aware state-space model that uses manifold constraints on the symmetric positive definite manifold for time series forecasting, achieving state-of-the-art performance on 11 benchmarks.

Proposes the Mamba-Assisted Closure (MAC) framework, a Mamba-based sequence model for non-Markovian closure in reduced-order modeling of high-dimensional dynamical systems, outperforming GRU-based and Markovian methods on Burgers' equation and Lorenz '96 systems.

Nemotron 3 Ultra is an open-weight release with an impressive capability-to-efficiency ratio, using a Mamba-2-attention hybrid stack and LatentMoE, and is larger than the previous Super variant.

Researchers propose a lightweight autoregressive framework for graph generation that uses structure-guided topological ordering to achieve near log-linear complexity, addressing scalability and novelty limitations of existing diffusion and autoregressive methods. The approach supports both LSTM and Mamba-style backbones and shows improved novelty while maintaining validity and uniqueness on molecular and non-molecular benchmarks.