Laguna-S-2.1 runs on my 6 years old gaming PC!

Reddit r/LocalLLaMA Models
laguna-s-2.1 local-inference llamacpp gguf quantization gaming-pc

Summary

Laguna-S-2.1, a quantized AI model, runs on a 6-year-old gaming PC with an RTX 3080, achieving 10 t/s decode and using 8.3 GB VRAM and 52.2 GB host RAM.

Laguna-S-2.1 runs on my 2020 PC (RTX 3080 10GB, Ryzen 9 3950X, 64 GB RAM). Realistically only usable overnight though. I must not breathe too hard or I run out of both VRAM and host RAM. There's just enough RAM left to run the compilation and unit tests of whatever coding project I hand to it. IQ4_XS, 128k q8/q8 context, all dense tensors on VRAM, no DFlash (won't fit). 120 t/s prefill, 10 t/s decode 8.3 GB VRAM, 52.2 GB host RAM I did not test it thoroughly yet, but thinking works out of the box. When support lands in beellama (a month from now?) and I can switch to kvarn, VRAM pressure will become much more manageable (hopefully. kvarn does not work with Gemma; no idea about this model). llamacpp CUDA @ https://github.com/poolsideai/llama.cpp/pull/3 hf = unsloth/Laguna-S-2.1-GGUF:UD-IQ4_XS ngl = 99 n-cpu-moe = 99 ctx-size = 131072 jinja = true flash-attn = on cache-type-k = q8_0 cache-type-v = q8_0 mlock = true no-mmap = true
Original Article

Similar Articles

I'm impressed by Laguna S 2.1

Reddit r/LocalLLaMA

Laguna S 2.1, a 120B-class model, impressed by solving a complex coding problem in Julia with long thinking tokens, outperforming Qwen models on a memory-constrained rearrangement task.

Laguna S 2.1 looping fix incoming

Reddit r/LocalLLaMA

Poolside released Laguna S 2.1, their most capable model for long-horizon tasks, along with multiple quantized variants (FP8, NVFP4, INT4, DFlash, GGUF) on Hugging Face.

poolside/Laguna-S-2.1-GGUF

Hugging Face Models Trending

Poolside releases GGUF quantizations of the Laguna S 2.1 AI model, including a DFlash speculative decoding draft model, enabling efficient local inference with llama.cpp.