Laguna S 2.1 GGUF Q4_K_M went from 68GB to 96GB?

Reddit r/LocalLLaMA Models
laguna-s gguf quantization model-size llama-cpp

Summary

A user notices that the Q4_K_M quantized version of Laguna S 2.1 increased from 68GB to 96GB, likely due to using more FP16 layers, and discusses potential issues with quantization and context looping.

I've been occasionally checking Laguna S 2.1 to see if there's any updates/fixes to the issues they've been having. I just noticed that they recently updated their Q4_K_M and it's now ballooned to 96GB, bumping up 8 layers to FP16 while leaving the rest in 4-bit. Does anyone know why they would do this? I'm guessing because the previous Q4_K_M had issues at that level of quantization, but just wanted to see if anyone else has better insight into this. I tried using Unsloth's Q4_K_M, but I find that it starts to loop repeatedly at higher contexts. I haven't tried their latest updated version from yesterday where they fixed the YaRN config, but I've tried manually setting those config values in llama.cpp and even then I have the looping, so I think I'll get the same result. Will give it another spin though just in case.
Original Article

Similar Articles

Unsloth Quantization of Laguna S 2.1 Is Out

Reddit r/LocalLLaMA

Unsloth released GGUF quantizations of the Laguna S 2.1 Mixture-of-Experts model, a 118B parameter coding model with 8B active parameters, 1M context window, and agentic capabilities. The quantized versions enable efficient local deployment.

Laguna S 2.1 looping fix incoming

Reddit r/LocalLLaMA

Poolside released Laguna S 2.1, their most capable model for long-horizon tasks, along with multiple quantized variants (FP8, NVFP4, INT4, DFlash, GGUF) on Hugging Face.

poolside/Laguna-S-2.1-GGUF

Hugging Face Models Trending

Poolside releases GGUF quantizations of the Laguna S 2.1 AI model, including a DFlash speculative decoding draft model, enabling efficient local inference with llama.cpp.