Would you consider 5t/s usable for a local model?
Summary
User discusses the usability of running the Qwen3.8 27b model locally at 5 tokens per second, comparing performance on different hardware setups and noting that lower speed can still be acceptable if system resources are managed.
Similar Articles
What speed is everyone getting on Qwen3.6 27b?
User benchmarks Qwen3.6-27B-Q8_0 at ~13 tokens/sec on 3 mixed GPUs with 128k context via llama.cpp, asking if performance is typical.
How do you plan to run Qwen3.8-2.4T-A95B locally?
A community thread asking how enthusiasts plan to run the large Qwen3.8-2.4T-A95B model locally, sharing the struggle of running huge AI models on personal hardware.
Run Qwen3.8 27B locally: real numbers from my Mac Studio
The article provides real-world performance benchmarks for running the Qwen3.8 27B AI model locally on a Mac Studio, comparing it to its predecessor and discussing hardware requirements and quantization effects.
Qwen 3.8 27B is faster than expected
A user reports that the Qwen 3.8 27B model achieves 50-60 tokens per second on dual 5060 TI cards, showing unexpected speed improvements over previous versions like Qwen 3.6.
Local agent workspace on a 4GB laptop GPU (RTX 3050 Ti): the tok/s and where a small model struggles once it has to call tools, build artifacts, and RAG
The author benchmarks local Qwen models of various sizes on a 4GB RTX 3050 Ti laptop GPU within the Bike4Mind workspace, finding the 2B model at Q4_K_M quantization is the sweet spot for fitting in VRAM, achieving 96 tok/s. Smaller models struggle with tool selection, artifact generation requiring multiple models, and RAG embeddings causing model swap overhead.