Stop asking what model to run. There are literally only two.
Summary
A tech enthusiast argues that only two local AI models (Qwen 3.6 35b a3b and Qwen 3.6 27b) are worth running, dismissing smaller models and recommending heavy quantization of larger models.
Similar Articles
Building a local AI server for Qwen3 30B with Q8 is this hardware a good fit?
A discussion about building a local AI server for the Qwen3 30B model with Q8 quantization, questioning whether the chosen hardware is a good fit.
@jtdavies: Coding on small models... My default model for my 4xDGX Spark cluster is @UnslothAI's Qwen3.6-35B-A3B-NVFP4. I get exce…
A user tests various small AI models for coding tasks, finding Qwen3.6-27B-NVFP4 to be the best balance of speed and accuracy, and notes poor Java performance in these models.
I tested 9 local models on the same flight sim prompt, all Q8, different Q providers, MLX
Benchmark of 9 quantized local LLMs running MLX on a flight-combat HTML prompt shows quant provider choice and model quirks matter more than parameter count or bit-width for usable code output.
Running local models on an M4 with 24GB memory
A guide on running local AI models like Qwen 3.5-9B on an M4 MacBook with 24GB RAM using tools like LM Studio, Ollama, and pi, including specific configuration tips for optimal performance.
Qwen 3.6 27B is the sweet spot for local development
Qwen 3.6 27B is praised as a powerful local AI model that outperforms expectations for general intelligence, suitable for practical tasks like code generation, and runs easily with llama.cpp.