Qwen3.8-Flash-Next optimised for Macs

Running on a M1 Max 64 GB: - SSD streaming for tensors - SSD streaming for engrams - SSD streaming for MTP How is it possible? * Custom Q4 quant: benchmarked all metal tensors then picked and spliced tensors from multiple Unsloth and AtomicChat quants to achieve best performance/bit. * Developed custom metal-optimized sparse attention mechanism, with almost linear degradation instead of the standard llama.cpp quadratic attention. * Using Q4_0 MTP - same acceptance rates as unsloth Q8_0 at half the RAM. * Dynamic MTP speculation size - leads to disabling MTP at the point where context size makes MTP a negative. * Various fixes to metal kernels, qwen graph and qwen indexer. https://github.com/mihailescu2m/llama.cpp Special thanks to Claude - three weeks worth of tokens and some extra out of pocket usage credits made it all possible. Feedback appreciated. Note: enabling MTP uses more RAM, which means less cache for tensors, leading to prefill going from 180 tps to g170 tps (at 4K). For 256K context, more RAM is needed for KV cache, prefill goes down to 150 tps. But with MTP, decode gains +70%, going up to 22 btps. So if you need highest prefill, disable MTP. A