PSA: llama.cpp -cram should be increased for agentic workflows (default is 8192)
Summary
PSA advising to increase the -cram parameter in llama.cpp for better performance in agentic workflows with long contexts, based on personal experience with Qwen 27B 3.8.
Similar Articles
After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)
The article shares an optimal llama.cpp configuration for running the Qwen 3.8 27B model on 16GB VRAM with 73k context, demonstrating its performance in agentic coding workflows through a real-world software engineering project.
PSA: If you haven’t updated Llama.cpp for a couple of days and find MTP to not be performing well, update llamacpp.
Update Llama.cpp for a significant token generation speed boost, up to 1.5-1.8x, and improved prompt processing.
Help optimizing llama.cpp + Qwen 27B on RTX PRO 6000 Blackwell for coding agents
A user details their setup running Qwen 27B with llama.cpp on an RTX PRO 6000 Blackwell for local coding agents, compares performance to Claude models, and asks for help resolving frequent crashes and malformed response issues.
PSA: Test your "threads" argument in llama.cpp (+80% performance in my case)
A user benchmarks thread count for hybrid CPU-GPU inference with Gemma 4 in llama.cpp, discovering a 80% performance uplift by using 16 threads instead of 6 on a hybrid core CPU, and shares the optimal command configuration.
Pipeline parallelism in llama.cpp may be wasting your VRAM
Testing shows that default pipeline parallelism in llama.cpp wastes VRAM with no speed benefit; compiling with GGML_SCHED_MAX_COPIES=1 saves significant VRAM while maintaining identical inference speed.