Tag
The user experimented with the Qwen 3.8 27b model to generate a 3D graphic game using a PI agent, processing a large design document on CPU over 12 hours.
A user shares how they achieved a 2x performance boost in AI inference by splitting a large MoE model between a Strix Halo APU and an R9700 GPU, detailing configurations and code modifications.
The article presents a benchmark tool for evaluating local LLM configurations, focusing on VRAM usage, performance metrics, and hardware optimization to assist developers in optimizing setups.
A discussion thread sharing configurations and tips for running AI models like Qwen3.8-27B on 16 GB VRAM Windows systems, focusing on memory optimization techniques.
A showcase of using the Qwen 3.8 27B model with 128k context to create a fully playable GTA Vice City-style game, including performance benchmarks and configuration tips.
An observation of Ling-3.0-flash (124B) on one DGX Spark generating 15,128 tokens in a single response with stable decode throughput around 35.6 tok/s, highlighting long-context decoding performance.
A user shares their attempts and configurations to achieve up to 115K context on a Q8-quantized Qwen3.6-27B model using 32GB VRAM on an RTX 5090, with benchmark results and trade-offs between context length and kv-cache quantization.
Ornith 35B shows 30-40% token generation speedup when paired with Qwen3.6 35B DFlash speculative model in llama-server, achieving 80% acceptance rate on mixed code and text, though prompt processing suffers.
A troubleshooting guide showing how changing OpenClaw's contextInjection setting from 'always' to 'continuation-skip' fixes prompt cache reloading on every turn when using llama-server, resulting in a 100x speed improvement for long sessions.
A guide for running Pi AI agent securely inside a Docker Sandbox while running llama-server on the host machine for local GPU inference.
The author upgraded their Hermes agents with TencentDB Agent Memory, using a local Qwen 3.5-4B model via llama-server for structured JSON extraction and multi-step tool use, implementing a resilient layered memory pipeline with cursor-based checkpointing.
Llampart 1.0.0 is a standalone local web UI for llama-server with translations, extended settings, and a polished conversation sidebar, released under MIT license.
Discussion about how the Pi coding agent controls thinking verbosity of Qwen 35B A3B model on llama-server, while other clients fail to do so.
Llama-Studio is a WebUI for managing llama-server sessions, allowing configuration, monitoring, and control of multiple instances for local development and experimentation.
Georgi Gerganov shared a one-liner to launch the quantized 27B Qwen3.6 model with llama-server using default speculative-decoding settings.
Author shares a working llama-server config to run the 35B-MoE Qwen3.6 model on an 8GB RTX 4060, highlighting a max_tokens trap caused by unconstrained internal reasoning and the fix using per-request thinking_budget_tokens.