sensenova/SenseNova-Loop-9B-Preview · Hugging Face

Source: https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview English|简体中文

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#%F0%9F%93%A3-news📣 News

• \[2026\.10\.30\]We releaseSenseNova-Loop-9B-Preview, a looped multimodal model for long-context and agentic workflows.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#%F0%9F%9A%80-sensenova-loop-9b-preview🚀 SenseNova-Loop-9B-Preview

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#%F0%9F%8E%AC-showcase-created-by-raccoon-work🎬 Showcase (Created by Raccoon Work)

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#motion-designMotion Design

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#ppt-designPPT Design

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#data-analysisData Analysis

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#overviewOverview

SenseNova-Loop-9B-Previewis a multimodal language model developed for long-context and agentic workflows. It accepts text and images and generates text for applications that combine document understanding, coding, and interaction with external tools.

The model combines alooped decoder with two-stream manifold-constrained hyper-connections (mHC). A contiguous block of decoder layers is executed twice with shared weights, while learned mappings manage information flow between two residual streams in the looped section. Together, these components provide a deeper execution path without duplicating the repeated block’s parameters.

Documented training recipes combine long sequences, multimodal inputs, and supervised task trajectories, with a maximum training sequence length of 262,144 tokens. For the best model performance, we recommend using our documented deployment environment and configuration parameters.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#model-highlightsModel highlights

• **Shared looped block:**24 decoder layers are applied twice in sequence, producing 56 decoder-layer executions from 32 distinct layers.
• **Two-stream mHC:**learned mappings route and combine information across two residual streams in the looped section.
• **Hybrid attention:**the decoder interleaves linear and full attention in a repeating four-layer pattern.
• **Long-context multimodal training:**documented recipes include text, images, code, and tool trajectories at sequence lengths up to 256K tokens.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#%F0%9F%93%8A-key-benchmarks📊 Key Benchmarks

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#model-specificationsModel specifications

PropertySenseNova-Loop-9B-PreviewArchitectureAutoregressive multimodal decoder; looped block with two-stream mHCBackbone scale~9B total parameters; ~14.6B inference parametersInput / outputText and images / textDistinct decoder layers32Looped layers / passes[4, 28) / 2, with shared weightsDecoder layer executions56 per forward passmHC streams2Training contextUp to 262,144 tokensExisting export precisionBF16 backbone; FP32 mHC routing parameters

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#model-architectureModel architecture

SenseNova-Loop-9B-Preview builds on a 32-layer multimodal decoder that combines shared-weight looped computation, hybrid attention, and two-stream manifold-constrained hyper-connections (mHC). The following sections describe how these components extend the computation path and manage information flow through the model.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#looped-decoderLooped decoder

The decoder first processes the input through layers 0–3. The same block of layers 4–27 is then applied twice in sequence, after which layers 28–31 and the output head produce the model’s output. IfBrepresents the repeated block andhits input, the looped computation can be written asB(B(h)).

During inference, the model performs56 decoder-layer computations per forward pass while storing only 32 sets of decoder-layer weights. The looped block reuses the same weights across both passes, with the second pass taking the first pass’s output as input.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#hybrid-attentionHybrid attention

The decoder interleaves linear-attention and full-attention layers. Each four-layer group contains three linear-attention layers followed by one full-attention layer. The loop boundaries align with complete four-layer groups, so the attention pattern is preserved across both passes.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#two-stream-mhcTwo-stream mHC

Inside the looped section, mHC carries two residual streams around the decoder-layer computations. Learned mappings govern how a layer reads from the streams, mixes stream information, and incorporates the layer output into the residual state. Sinkhorn normalization constrains the stream-mixing map; the documented configuration uses 20 iterations.

The loop range, pass count, and mHC routing parameters form part of the checkpoint’s execution configuration. A compatible inference implementation must load these parameters and reproduce the same execution order.

ConfigurationValueDistinct decoder layers / executions32 / 56Repeated block / passes[4, 28) / 2Hidden dimension4,096Feed-forward dimension12,288Vocabulary size248,320Full-attention heads / KV heads16 / 4mHC streams / Sinkhorn iterations2 / 20

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#inference-accelerationInference acceleration

SenseNova-Loop-9B-Preview supportsMTPfor accelerated inference.DsparkandDflash2are under development, with training currently in progress.

MethodStatusMTPAvailableDsparkTraining in progressDflash2Training in progress Practical speedups depend on the workload, hardware, and serving configuration.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#models-and-resourcesModels and resources

ResourceStatusModel weightsHugging Face model pageTransformers inference implementationPublication details to be announcedTechnical reportTBD

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#getting-startedGetting started

Use Transformers to load the model and its custom implementation from a Hugging Face repository or a local model directory. The example below runs text generation on a single GPU.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#requirementsRequirements

The checked inference environment usesTransformers 5.18.0andPyTorch 2.11.0 with CUDA 13.0. The model directory must include the configuration, tokenizer, chat template, model code, and accessible weight shards.

First install aPyTorch buildcompatible with your CUDA environment, then install the remaining dependencies:

python -m pip install "transformers==5.18.0" accelerate safetensors

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#text-generation-with-transformersText generation with Transformers

Setmodel\_idto the Hugging Face repository ID or a local model directory, then run:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sensenova/SenseNova-Loop-9B-Preview"
device = "cuda:0"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map={"": device},
).eval()

messages = [{"role": "user", "content": "What is 13 times 7? Reply with the number only."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        use_cache=True,
    )

response = tokenizer.decode(
    outputs[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=True,
)
print(response)

The example uses greedy decoding and generates up to 256 new tokens. Setenable\_thinking=Trueto enable thinking mode, or changedeviceto select another GPU.

trust\_remote\_code=Trueloads the custom SenseNova Loop model implementation.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#trainingTraining

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#model-trainingModel training

To fine-tuneSenseNova-Loop-9B-Previewwith mHC on a single node with 8 GPUs, run the following commands in your configured training environment:

git clone https://github.com/OpenSenseNova/torchtitan.git
cd torchtitan
MODEL_PATH=/path/to/SenseNova-Loop-9B-Preview DATASET=/path/to/train.jsonl \
  bash examples/train/SenseNova-Loop-9B-Preview/mhc2_loop2_titan.sh

Replace/path/to/train\.jsonlwith your SFT dataset. See thetraining guidefor configuration options, DPO, and MTP training.

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#dspark-trainingDSpark training

To train a DSpark draft model forSenseNova-Loop-9B-Previewon a single node with 8 GPUs, run the following commands in your configured training environment:

git clone https://github.com/OpenSenseNova/SenseSpec.git
cd SenseSpec
bash examples/SenseNova-Loop-9B-Preview-dspark/run.sh

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#dflash2-trainingDFlash2 training

git clone https://github.com/OpenSenseNova/SenseSpec.git
cd SenseSpec
bash examples/SenseNova-Loop-9B-Preview-dflash2/run.sh

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#deploymentDeployment

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#lightllmLightLLM

Four-GPU example; replace the local model path:

docker pull ghcr.io/modeltc/lightllm:main

docker run --rm --gpus all --ipc=host -p 8000:8000 \
  -v /path/to/SenseNova-Loop-9B-Preview:/model \
  -e LOADWORKER=4 -e LIGHTLLM_TRITON_AUTOTUNE_LEVEL=1 \
  ghcr.io/modeltc/lightllm:main \
  python -m lightllm.server.api_server \
    --model_dir /model --model_name SenseNova-Loop-9B-Preview \
    --tp 4 --host 0.0.0.0 --port 8000

Example request:

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "SenseNova-Loop-9B-Preview", "messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 128}'

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#sglangSGLang

Use a build containing the SenseNova Loop implementation in this PR, and replace the path below with your local canonical checkpoint directory:

git clone https://github.com/sgl-project/sglang.git
cd sglang
git fetch origin pull/47281/head
git checkout -b sensenova-loop FETCH_HEAD
python -m pip install -e ./python

sglang serve --model-path /path/to/SenseNova-Loop-9B-Preview

https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview#license-and-citationLicense and citation

License:Apache License 2.0. Citation:

@misc{sensenova2026loop,
  title        = {{SenseNova-Loop-9B-Preview}},
  author       = {{SenseNova}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview}
}