exllamav3

Tag

Cards List
#exllamav3

Qwen3.8 flash next + exllamav3 + hermes is amazing

Reddit r/LocalLLaMA ↗ · 2026-09-26

User shares their positive experience using Qwen3.8 model with ExLlamaV3 on 6x3090 GPUs, achieving 80-120 tokens per second, and controlling Hermes agent via Matrix for daily use.

0 favorites 0 likes
#exllamav3

Success running Qwen 3.8 27B EXL3 on RTX 3060 + 5060 Ti

Reddit r/LocalLLaMA ↗ · 2026-09-20

A user successfully runs the Qwen 3.8 27B AI model on a mixed setup of RTX 3060 and 5060 Ti GPUs using tensor parallelism with exllamav3, achieving around 50 tokens per second with MTP enabled.

0 favorites 0 likes
#exllamav3

ExLlamaV3 is underrated

Reddit r/LocalLLaMA ↗ · 2026-09-07

The author advocates for ExLlamaV3, an underrated local AI inference tool that outperforms llama.cpp on NVIDIA hardware, with recent updates like CPU MoE offload enhancing its performance.

0 favorites 0 likes
#exllamav3

exllamav3 comfortably beats llama.cpp running CPU-offloaded Qwen-3.8-Flash-Next on my setup!

Reddit r/LocalLLaMA ↗ · 2026-09-07

A user benchmarks exllamav3 against llama.cpp for CPU-offloaded inference, showing exllamav3 is faster for Qwen models but slower for others, depending on hardware and model architecture.

0 favorites 0 likes
#exllamav3

I ported EXL3 to run well on Apple Silicon - PonyExl3

Reddit r/LocalLLaMA ↗ · 2026-06-15

Ported the EXL3 LLM codec to run on Apple Silicon via Metal, achieving high prefill and generation speeds on M5 Max (e.g., ~600 tok/s prefill, 17-80 tok/s gen on various models).

0 favorites 0 likes
#exllamav3

@0xSero: Here's everything you need to know about inference and hosting LLMs. Have you ever seen: - vllm - sglang - llama.cpp - …

X AI KOLs Timeline ↗ · 2026-04-20 Cached

An overview of popular open-source inference engines including vLLM, SGLang, llama.cpp, and ExLlamaV3 for hosting and running large language models.

0 favorites 0 likes
← Back to home

Submit Feedback