Tag
A user reports that the latest LM Studio update (0.4.17) eliminated the multi-token prediction speed boost, reverting to previous performance on an RTX 5090 setup.
Benchmarks of DFlash speculative decoding combined with KV cache compression on RTX 5090 show up to 3.26x speedup on Qwen3.6-27B with minimal perplexity degradation, with q4_0/turbo4 providing the best balance.
SANA Streaming enables video-to-video generation on a single NVIDIA RTX 5090 GPU.
A tweet demonstrates that Multi-Token Prediction (MTP) achieves significant speedups for Qwen models on dual RTX 5090 hardware, suggesting that local inference can now rival cloud-model performance.
llama.cpp release b9235 introduces speculative n-gram tuning, achieving up to ~7x throughput improvement on Qwen3.6 27B on an RTX 5090, with the k4v96 configuration showing the best sustained performance in 10k and 70k token tests.
A technical test of llama.cpp's new Multi-Token Prediction (MTP) support using Qwen3.6 models on an RTX 5090, comparing performance with and without MTP across different prompts and GGUF quantizations.
Unsloth Qwen3.6 27B Q6_K achieves over 100 tokens per second with MTP on RTX 5090, up from 45-50 t/s without MTP.
Open-dLLM adapts Qwen3.6 to use diffusion-based generation, achieving over 3,000 tok/s on an RTX 5090 for short sequences, with code released on GitHub.
NVIDIA is reportedly planning to increase the price of its upcoming RTX 5090 graphics card due to rising costs of GDDR7 memory.
Tracking 15 EU GPU stores shows RTX 5090 prices rising 3% due to AI/workstation demand while other GPUs drop 7-9%, suggesting sustained high pricing for AI inference hardware.
User asks whether purchasing an RTX 5090 and high-end PC for ~$5500 is worth it for LLM experimentation and learning, compared to cloud compute alternatives.
A benchmark shows that using vLLM with DFlash speculative decoding boosts Gemma 4 26B inference to ~578 tokens per second on a single RTX 5090, achieving a 2.56x speedup over baseline.
User reports surprisingly usable coding performance from Qwen3-27B-UD-Q6_K_XL.gguf running locally on RTX 5090 at ~50 tok/s with 200K context, marking a significant leap in local model quality.
User reports a satisfying experience running Claude Code locally via LM Studio on an RTX 5090, achieving 64k context length and 200+ tokens per second.