Qwen 3.8 Flash Next locally on simple mobile phone at 3.5 tok/s
Summary
Demonstrates that Qwen 3.8 Flash Next can run locally on a mid-range Android phone at 3.5 tokens per second with optimizations and low quantization.
Similar Articles
Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀
The article provides memory estimates for the Qwen3.8-Flash-Next model, suggesting it could be local-friendly with quantization techniques.
Qwen3.8-Flash-Next at 170K context on a single 96 GB card. ~110 tok/s.
The article describes a method to run the Qwen3.8-Flash-Next model with quantized n-grams to achieve over 170K token context on a single 96GB GPU card, with performance up to 110 tokens per second using INT4 quantization and memory-mapped disk access.
TQwen 3.8 flash next ud1s on 6gb vram and 16 gb system ram
A user shares their experience running the Qwen 3.8 flash next model on a system with 6GB VRAM and 16GB RAM using llama.cpp, achieving 6-7 tokens per second with 1-bit quantization, and asks for recommendations on quantization variants.
I ran Qwen3.5-0.8B on a sub-$20 CPU chip in under 512MiB of memory
The author ran Qwen3.5-0.8B on a $10-20 Amlogic A113X CPU chip with a custom C runtime, achieving 1.82 tok/s decode and under 490 MiB peak RSS, demonstrating that small LLM inference can run on deployed edge hardware without a GPU.
Qwen3.8-Flash-Next turns 4xR9700 into a local AI powerhouse! 120 t/s TG and 12k t/s PP single request with optimized vLLM
The article reports that the Qwen3.8-Flash-Next model achieves 120 tokens/second generation speed and 12k tokens/second prefill on a system with 4x AMD R9700 GPUs using optimized vLLM and a custom Docker image.