Tag
PS5LM is an open-source project that runs llama.cpp on a jailbroken PS5's RDNA 2 GPU, turning the console into a private model server that serves GGUF models via OpenAI and Anthropic compatible APIs to any device on the local network.
Drama3 模型现已支持在 audio.cpp 中运行，在 RTX 5090 上全速运行时最快可达实时速度的 5 倍，或在低于 8 GB 显存下以约 2 倍实时速度运行，仍需进一步优化与修正。
A community-compiled ranking of 76 GPUs benchmarked with llama.cpp's Vulkan backend for local LLM inference, sorted by price-to-performance on the used market. Older wide-bus and HBM2 cards like the P102-100, MI50, and Radeon VII dominate value, while modern mid-range cards are bottlenecked by narrow memory buses.
Infermeld is an experimental open-source Linux companion kit (v0.1.0) for running one GGUF model across mixed AMD and NVIDIA GPUs using llama.cpp, packaging device selection, build instructions, preflight checks, and a thermal guard. Tested on an RX 6000 XT + RTX 3080 with Qwen3.6-35B-A3B, with the maintainer seeking other hardware combos for reproduction.
A community-curated list of budget and legacy GPUs (NVIDIA GTX/Maxwell/Pascal, AMD Radeon and Instinct MI-series) that can run LLM inference through the Vulkan backend on Linux, bypassing the latest CUDA/ROCm support requirements. The AMD Instinct MI50 16GB at around $150 is highlighted as the top value pick.
Apex Compute is developing an open-source Mesa Vulkan driver for its own AI hardware, enabling native open-source graphics software support on their accelerator platform.
Janus 是一个单文件 Go 二进制工具，通过 Vulkan 在 AMD/Intel/NVIDIA GPU 或 CPU 上运行 GGUF 模型，并提供 OpenAI 兼容 API，无需 Python、Docker 或 Ollama。
Follow-up update on a native Rust + Vulkan Transformer training backend: 14 architectures now pass strict parity-verified full-training gates against a pinned Hugging Face Transformers oracle, and full PEFT workflows (LoRA, saved modules, adapter switching, exact resume) are verified across 32 architecture surfaces.
OpenDLSS-NR is an open-source, bit-exact Vulkan reimplementation of NVIDIA's DLSS 5 generative neural rendering network (71-block Swin/ViT U-net), with a second independent WebGPU port that runs in the browser without tensor cores or FP8.
This pull request fuses a chain of operations in the Vulkan backend for the qwen4exp model to optimize inference performance in llama.cpp.
This update to llama.cpp introduces a Vulkan-based implementation of int8 cooperative matrix multiplication for AMD RDNA3 and RDNA4 GPUs, enhancing inference performance on these hardware platforms.
A user shares their local AI setup using two BC-250 ex-mining APUs to run the Qwen3.6-35B-A3B model with llama.cpp, achieving 60 tok/s and 64k context for under $300.
The author demonstrates forcing AMD MI50 GPUs to work with ROCm 6.4.3 and building a Vulkan-based neural network training stack, challenging the consensus on legacy hardware support.
VoxGen is a new lightweight native inference engine for VoxCPM2 models, optimized for AMD cards using Rust and Vulkan compute to enhance performance and remove Python/PyTorch dependencies.
Android Studio now natively supports running the Gemma 4 AI model using llama.cpp, with features like multi-GPU support and a 128k context length.
A user reported slower inference speeds with MTP on the Gemma 4 12B QAT model when using the Vulkan backend, but discovered that switching to ROCm significantly improved performance, achieving 60t/s with MTP.
Benchmark results for running Qwen3.8-Flash-Next with MTP on Strix Halo hardware using the Vulkan backend in llama.cpp, including performance metrics and observations on output quality.
The user fixed crashes when running large AI models on an AMD GPU in headless Linux by disabling runtime power management, preventing model weights from being dumped into limited RAM and causing OOM errors.
This blog post details optimizations to the meshoptimizer tool for processing billions of triangles, inspired by NVIDIA's RTX Mega Geometry and hierarchical clustered level-of-detail techniques.
Solo is a .so loader that enables fully static Linux binaries to dynamically load host GPU drivers at runtime, using a glibc ABI bridge on musl to avoid containers or AppImages.