poolside/Laguna-S-2.1-GGUF · Hugging Face

Source: https://huggingface.co/poolside/Laguna-S-2.1-GGUF

Use on OpenRouter·Use on Vercel AI Gateway·Release blog post

GGUF conversions ofLaguna S 2.1for llama.cpp, plus the DFlash speculative-decoding draft model. See thebase model cardfor architecture details, license, and usage guidance.

https://huggingface.co/poolside/Laguna-S-2.1-GGUF#filesFiles

FileSizeNoteslaguna\-s\-2\.1\-F16\.gguf235 GBfull precisionlaguna\-s\-2\.1\-Q8\_0\.gguf128 GBlaguna\-s\-2\.1\-Q4\_K\_M\.gguf75 GBimatrix-quantizedlaguna\-s\-2\.1\-DFlash\-BF16\.gguf2.2 GBDFlash drafter for speculative decodinglaguna\-s\-2\.1\.imatrix0.4 GBimportance matrix used for the K-quants

https://huggingface.co/poolside/Laguna-S-2.1-GGUF#servingServing

Serve with Poolside’s llama.cpp fork, branchlaguna, which carries full Laguna support including DFlash speculative decoding. (Base Laguna support is also in upstream review:ggml-org/llama.cpp#25165.)

git clone --branch laguna https://github.com/poolsideai/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf --jinja --port 8000

# with DFlash speculative decoding:
./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf \
  -md laguna-s-2.1-DFlash-BF16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 15 -fa on --jinja --port 8000

https://huggingface.co/poolside/Laguna-S-2.1-GGUF#context-lengthContext length

These GGUFs ship configured for a 262,144-token (256K) context window. This is the configuration we recommend for best output quality.

The weights are native 1M checkpoints: training included a long-context extension stage up to 1,048,576 tokens. To use more than 256K of context with llama.cpp, override the rope configuration at load time:

--ctx-size 1048576 --rope-scaling yarn --rope-scale 128 --yarn-orig-ctx 8192

You may experience quality degradation with the 1M configuration. If you use it, we recommend sampling with\-\-temp 0\.7 \-\-top\-p 0\.95.

This release also corrects the embeddedyarn\_attn\_factormetadata (now 1.0; llama.cpp derives the YaRN attention scaling internally).