poolside/Laguna-S-2.1-NVFP4 · Hugging Face

Source: https://huggingface.co/poolside/Laguna-S-2.1-NVFP4

Use on OpenRouter·Use on Vercel AI Gateway·Release blog post

Laguna S 2.1-NVFP4 is a 117.6B total parameter Mixture-of-Experts model with 8.5B activated parameters per token designed for agentic coding and long-horizon work on a local machine. It uses Sliding Window Attention with per-head gating in 36 out of 48 layers for fast inference and low KV cache requirements.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#highlightsHighlights

• Mixed SWA and global attention layout: Laguna S 2.1 uses softplus gating with per-layer rotary scales, enabling mixed SWA (Sliding Window Attention) and global attention layers in a 3:1 ratio (across 48 total layers)
• KV cache in FP8: KV cache quantized to FP8, reducing memory per token
• Native reasoning support: Interleaved thinking between tool calls with support for enabling and disabling thinking per-request
• Local-ready: At 117.6B total parameters and 8.5B activated, the NVFP4 weights are roughly 71 GB. Available onOllamaandllama.cpp(BF16 and Q4_K_M only)
• OpenMDW-1.1 license: Use and modify the model and associated materials freely for commercial and non-commercial purposes (learn more about OpenMDW)

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#model-overviewModel overview

• Training: pre-training, post-training and reinforcement learning stages
• Number of parameters: 117.6B total with 8.5B activated per token
• Optimizer: Muon
• Layers: 48 layers (12 layers with global attention, 36 layers with sliding window attention)
• Experts: 256 experts with 1 shared expert
• Sliding Window: 512 tokens
• Modality: text-to-text
• Context window: 262,144 tokens
• Reasoning support: interleaved thinking with preserved thinking

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#benchmark-resultsBenchmark results

ModelSizeTerminal-Bench 2.1SWE-bench MultilingualSWE-Bench Pro (Public Dataset)DeepSWESWE Atlas (Codebase QnA)Toolathlon VerifiedLaguna S 2.1118B-A8B**70.2%****78.5%****59.4%****40.4%****46.2%**49.7%Tencent Hy3295B-A21B71.7%75.8%57.9%---Inkling975B-A41B63.8%-54.3%--45.5%*Nemotron 3 Ultra550B-A55B56.4%67.7%---34.3%*DeepSeek-V4-Pro Max1.6T-A49B64.0%*76.2%55.4%9.0%*27.2%*55.9%*Kimi K32800B-A50B88.3%--69%--Qwen 3.7 Max-74.5%*78.3%60.6%---Muse Spark 1.1-80%-61.5%53.3%42.2%*75.6%Claude Fable 5-88%-80.3%70%-- Benchmarks as of 21 July 2026. Laguna S 2.1 inbold; a dash (-) marks a benchmark a model was not evaluated on. Scores marked * are as reported by third parties: Terminal-Bench 2.1 and DeepSWE via Artificial Analysis, SWE Atlas via Scale AI’s official leaderboard, and Toolathlon Verified via its official leaderboard. Full evaluation trajectories:trajectories.poolside.ai.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#usageUsage

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#context-lengthContext length

This checkpoint ships configured for a 262,144-token (256K) context window. This is the configuration we recommend for best output quality.

The weights are native 1M checkpoints: training included a long-context extension stage up to 1,048,576 tokens, and quantization was calibrated at the 1M configuration. If you need more than 256K of context, restore the 1M configuration by editingconfig\.json:

"rope_parameters": {
  "full_attention": {
    "factor": 128.0,
    "attention_factor": 1.4852030263919618
  }
},
"max_position_embeddings": 1048576

You may experience quality degradation with the 1M configuration. If you use it, we recommend sampling withtemperature <= 0\.7andtop\_p <= 0\.95.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#recommended-samplingRecommended sampling

For the best balance of quality and reliability we recommend sampling withtemperature 0\.7andtop\_p 0\.95.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#local-deploymentLocal deployment

Laguna S 2.1-NVFP4 is supported in vLLM, SGLang and Transformers, and TRT-LLM thanks to the support of the team at NVIDIA. Use Laguna-S 2.1 with Ollama (with MLX support) or Llama.cpp (BF16 and Q4_K_M only) for the best results on your local machine.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#vllmvLLM

Serving requires vLLM 0.25.0 or later. See the mainLaguna S 2.1 model cardfor the full recipe.

The full vLLM recipe is on the mainLaguna S 2.1 model cardand on thevLLM recipes page. Quantization is detected automatically fromquantization\_configin this checkpoint, so the same command works withpoolside/Laguna\-S\-2\.1\-NVFP4substituted for the model ID. No extra flags required.

**Optional: speculative decoding with DFlash.**Pair with the quantization-matched draft modelpoolside/Laguna-S-2.1-DFlash-NVFP4by adding\-\-speculative\-config '\{"model":"poolside/Laguna\-S\-2\.1\-DFlash\-NVFP4","num\_speculative\_tokens":15,"method":"dflash"\}'to the serve command.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#dgx-spark–mac-studioDGX Spark & Mac Studio

Ollama is the quickest way to run Laguna S 2.1 on a single 128 GB box, whether that is an NVIDIA DGX Spark (GB10) or an Apple Silicon Mac Studio:

ollama run laguna-s-2.1

It pulls the Q4_K_M GGUF (about 75 GB) and runs on the GB10 GPU on the Spark or on Metal on the Mac, with no build step. We measured about 12.6 tokens/s on the Spark and 17.6 on Apple Silicon.

• You need about 128 GB of unified memory; the Q4_K_M weights are around 75 GB on their own.
• The first load reads 75 GB off disk and can run past Ollama’s 5-minute default timeout. If it does, setOLLAMA\_LOAD\_TIMEOUT=20m.
• Reasoning is on by default. Pass"think": falsein the request to turn it off.
• The higher-fidelity tags need a bigger box:laguna\-s\-2\.1:q8\_0is about 128 GB (so a 192 GB Mac Studio) andlaguna\-s\-2\.1:f16is about 235 GB (256 GB or more). On the Spark’s 128 GB, stick to Q4_K_M. Apple Silicon can also run it through MLX.
• Ollama is built on llama.cpp. To build the runtime yourself, poolside’slagunabranch of llama.cppserves the same GGUFs.

Maximum performance on the DGX Spark (native NVFP4 + DFlash)

For the fastest setup on the Spark, serve this checkpoint with vLLM instead. That gives you the native NVFP4 kernels and DFlash speculative decoding.

One-time setup:

# Python headers: Triton JIT needs them and DGX OS ships without them.
# Without this, the first server start dies in triton/runtime/build.py.
sudo apt install -y python3.12-dev

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv ~/venvs/vllm025 -p 3.12

# vLLM 0.25.1 with CUDA-13 torch (aarch64 wheels are on PyPI)
uv pip install -p ~/venvs/vllm025 vllm==0.25.1 --torch-backend=cu130

# FlashInfer nightly trio: without flashinfer-python the NVFP4 path is not
# native; the jit-cache wheel avoids most first-start JIT compilation.
uv pip install -p ~/venvs/vllm025 \
  "flashinfer-python==0.6.15.dev20260712" \
  "flashinfer-cubin==0.6.15.dev20260712" \
  "flashinfer-jit-cache==0.6.15.dev20260712" \
  --extra-index-url https://flashinfer.ai/whl/nightly/ \
  --extra-index-url https://flashinfer.ai/whl/nightly/cu130/ \
  --index-strategy unsafe-best-match

hf download poolside/Laguna-S-2.1-NVFP4
hf download poolside/Laguna-S-2.1-DFlash-NVFP4   # 73 GB total

Serve:

export CUTE_DSL_ARCH=sm_121a          # arch string for FP4 kernel JIT
export PATH=/usr/local/cuda/bin:$PATH # nvcc for JIT
export MAX_JOBS=4                     # cap JIT fan-out; see warning below
source ~/venvs/vllm025/bin/activate

vllm serve poolside/Laguna-S-2.1-NVFP4 \
  --speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash-NVFP4","num_speculative_tokens":15}' \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1 \
  --override-generation-config '{"temperature":0.7,"top_p":0.95}' \
  --max-num-seqs 32 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.85 \
  --host 0.0.0.0 --port 8000

• You do not need backend flags: auto-selection picks FlashInferCutlass, which runs natively onsm\_121. Do not set\-\-linear\-backend flashinfer\_b12xon 0.25.1; the opt-in is broken there and it is slower anyway.
• Keep the\-\-override\-generation\-config. Many clients send no sampling parameters, and the raw defaults degrade output on the NVFP4 quantization. The model’sgeneration\_config\.jsonsetstop\_k 20(eval-certified truncation), which handles that. Do not addmin\_p: vLLM rejectsmin\_pandlogit\_biasunder speculative decoding, so putting it in the defaults returns a 400 on every sampled request.
• \-\-max\-num\-seqs 32is required: DFlash crashes vLLM at the default of 256.
• The first start takes about 15 minutes (weight load from NVMe, JIT, and graph capture).

Never dropMAX\_JOBS=4on a cold~/\.cache/flashinfer. An uncapped nvcc fan-out can exhaust the 128 GB unified memory and take the whole machine down. A warm cache makes it a no-op, but the cache is cold again after any config or shape change.

Prefill runs 600-800 tok/s, decode is around 15 tok/s on prose and 22-24 on code, and DFlash accepts 2.9-3.1 tokens per step. At the 256K setting you get 830-870K KV tokens. Two requests at once roughly double total throughput. Without speculation, decode sits at 13-14 tok/s on every engine we tried on the GB10, which is the memory-bandwidth ceiling for this model. Speculation is how you get past it.

pool CLI against the box:

POOLSIDE_STANDALONE_BASE_URL=http://<spark-ip>:8000/v1 \
POOLSIDE_STANDALONE_MODEL=poolside/Laguna-S-2.1-NVFP4 \
POOLSIDE_STANDALONE_CONTEXT_LENGTH=262144 \
POOLSIDE_API_KEY=dummy pool

Use the box’s IPv4 address rather than the\.localmDNS name: Go’s resolver can pick the link-local IPv6 and fail with “no route to host”. On macOS the terminal app needs Local Network permission, and the system curl is exempt from that check, so if curl works but pool does not, it is the permission and not the network.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#sglangSGLang

Laguna S 2.1 is supported in SGLang viasgl-project/sglang#24204. Quantization is detected automatically fromquantization\_config, so no extra flags are required. See theSGLang cookbook entryand the mainLaguna S 2.1 model cardfor a serving recipe.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#transformersTransformers

The full Transformers recipe is on the mainLaguna S 2.1 model card. Substitutepoolside/Laguna\-S\-2\.1\-NVFP4for the model ID; quantization is detected automatically fromquantization\_config.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#trt-llmTRT-LLM

Laguna S 2.1 support ships in TensorRT-LLM\>=1\.3\.0rc16; see the install recipe on the mainLaguna S 2.1 model card. Substitutepoolside/Laguna\-S\-2\.1\-NVFP4for the model ID; quantization is detected automatically fromquantization\_config, no extra flags required.

from tensorrt_llm import LLM

llm = LLM(model="poolside/Laguna-S-2.1-NVFP4", trust_remote_code=True)

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#ollamaOllama

Available on theOllama library.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#controlling-reasoningControlling reasoning

Laguna S 2.1-NVFP4 uses the same reasoning controls (interleaved thinking, preserved reasoning, and theenable\_thinkingflag) as the base model. See theControlling reasoningsection of the main Laguna S 2.1 model card.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#licenseLicense

This model is licensed under theOpenMDW-1.1 License.

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4#intended-and-responsible-useIntended and Responsible Use

Laguna S 2.1-NVFP4 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna S 2.1-NVFP4 is subject to theOpenMDW-1.1 License, and should be used consistently with Poolside’sAcceptable Use Policy. We advise against circumventing Laguna S 2.1-NVFP4 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.

Please report security vulnerabilities or safety concerns to[email protected].