Tag
Users report that AI models, including Qwen 3.6 35B, enter infinite loops when integrated with Copilot Chat or Hermes, generating excessive tokens or incorrect tool calls.
Build 9254 of llama.cpp fixes a token generation regression and adds Programmatic Dependent Launch (PDL) support for NVIDIA GPUs, yielding up to 10% speedup in token generation on newer hardware.
A user benchmarks the Nvidia 5090 RTX GPU for LLM inference using llama.cpp, measuring prompt processing and token generation at various power levels, finding that prompt processing is more sensitive to power limits than token generation, and noting differences from the 4090 RTX.
atomic.chat has optimized Gemma 4 26B inference in LLaMA.cpp, achieving ~40% faster token generation on MacBook Pro M5 Max using Multi-Token Prediction (MTP) speculative decoding. This is a notable win for local AI users running desktop apps, coding agents, and private on-device assistants.