@ggerganov: llama-server -hf ggml-org/Qwen3.6-27B-GGUF --spec-default
Summary
Georgi Gerganov shared a one-liner to launch the quantized 27B Qwen3.6 model with llama-server using default speculative-decoding settings.
View Cached Full Text
Cached at: 04/22/26, 05:02 PM
llama-server -hf ggml-org/Qwen3.6-27B-GGUF –spec-default
Similar Articles
Best config for Qwen3.6 27b / llama.cpp / opencode
Community thread sharing optimized llama.cpp launch commands for running the 27B Qwen3.6 GGUF model with long 100K-512K context on multi-GPU setups.
Qwen 3.6 27B on 24GB VRAM setup: backend comparisons, quant choice and settings (llama.cpp, ik_llama.cpp, BeeLlama, vllm)
The article compares llama.cpp backends for running Qwen 3.6 27B on an RTX 3090 24GB, finding ik_llama.cpp with IQ4_KS quantization yields the best performance (1261 tok/s prefill, 72.9 tok/s decode).
Qwen3.6 27B more dumb in vLLM compared to llama.cpp
A user reports that the Qwen3.6-27B model performs better and more reliably with llama.cpp than with vLLM, citing tool call errors and 'lobotomized' behavior in vLLM despite extensive configuration.
@WaleedAhmad1a10: Check out the Qwen 3.5 27B MoQ GGUFs :
A Hugging Face repository (kaitchup/Qwen3.6-27B-GGUF-MoQ) provides GGUF quantized weights for the Qwen3.6-27B MoQ model, enabling local inference with tools like llama.cpp and Ollama.
@MiaAI_lab: If you're looking for a simple start/stop script to run your Qwen3.6 27B/35B, check this out. It's optimized for speed …
MiaAI-Lab provides a simple Bash start/stop script for running Qwen3.6 27B/35B GGUF models via llama-server, optimized for speed and coding performance.