gpu-performance

Tag

Cards List
#gpu-performance

Making cudaMemcpy 5x faster on a B300 [P]

Reddit r/MachineLearning ↗ · 6d ago

A developer released cudamemcpy-tuner, an open-source tool that speeds up pageable cudaMemcpy transfers up to 5x on NVIDIA B300 GPUs, boosting 1 GB CPU-to-GPU copies from 10.2 GB/s to 51.5 GB/s via a `cmt tune` command.

0 favorites 0 likes
#gpu-performance

Today's Strata update boosts performance for R9700 users in Flash Next

Reddit r/LocalLLaMA ↗ · 2026-11-03

Strata's latest update significantly boosts inference performance for AMD R9700 users running Flash Next, roughly doubling token output from ~70 tok/s to 140-160 tok/s while still trailing Radiance. The user notes quantization limits (Q5 falling just short of VRAM for split-engine mode) and hopes future updates close the gap further.

0 favorites 0 likes
#gpu-performance

@jiayq: A hand-typed analysis for optimizing LLM inference with agent fleets. There has been a lot of recent news about that - …

X AI KOLs Timeline ↗ · 2026-10-28 Cached

A technical analysis of optimizing LLM inference using agent fleets, based on DeepSeek 4.1 Flash optimization reaching 2300 tok/s, highlighting that most gains come from configs/graphs rather than new kernels, questioning megakernel approaches, and emphasizing verification and convergence in automated optimization runs.

0 favorites 0 likes
#gpu-performance

Qwen3.8-27B and Flash-Next on consumer hardware: engine numbers I could verify, and what they cost with 2026 RAM prices

Reddit r/LocalLLaMA ↗ · 2026-10-19

A detailed comparison of inference engines running Qwen3.8-27B and Flash-Next on consumer hardware, with verified decode/prefill numbers and cost-per-euro analysis showing two mid-range GPUs can beat a single RTX 5090 by 3–10× per euro, while DDR5 RAM now makes up ~40% of experts-in-RAM builds.

0 favorites 0 likes
#gpu-performance

Why is Speculative Decoding Fast? (4 minute read)

TLDR AI ↗ · 2026-10-09 Cached

一篇技术博客深入解释了投机解码为什么快:并非因为总FLOPs减少(实际上反而增加),而是因为它改变了GPU的工作类型——在低批量场景下把原本空闲的算力从内存瓶颈(memory-bound)转为计算瓶颈(compute-bound),用单序列的'深度'换取了批量解码的'广度'。

0 favorites 0 likes
#gpu-performance

@no_stp_on_snek: 9070xt on Butter. @waffuruai . not too shabby. still some rough edges to clean up Gemma 4 12B https://localmaxxing.com/…

X AI KOLs Following ↗ · 2026-10-05 Cached

User benchmarked Gemma 4 12B IT (GGUF) on a Radeon RX 9070 XT GPU, reaching about 85.3 tokens per second via localmaxxing.com, noting solid performance with some rough edges remaining.

0 favorites 0 likes
#gpu-performance

95+ TPS through 100K generated for qwen3.8 27b, 262K ctx, on a single 3090

Reddit r/LocalLLaMA ↗ · 2026-09-28

LlamAmpere v0.4 is released with Ampere-specific improvements, achieving over 95 tokens per second and supporting 262K context for the Qwen3.8-27B model on a single NVIDIA 3090 GPU.

0 favorites 0 likes
#gpu-performance

Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine (15 minute read)

TLDR AI ↗ · 2026-09-28 Cached

Modal and Carnegie Mellon University's Full Stack Data Lab developed Quail, an inference engine that integrates query planning with inference to achieve over a billion tokens processed per minute on a single H100 GPU, outperforming vLLM by 10x in specific scenarios.

0 favorites 0 likes
#gpu-performance

Got 85.6tok/s MTP with Qwen3.8:27b. Single RTX 5090

Reddit r/LocalLLaMA ↗ · 2026-09-26

Achieved 85.6 tokens per second using the Qwen3.8:27b model on a single RTX 5090 GPU.

0 favorites 0 likes
#gpu-performance

@BenjaminDEKR: Here are my tests of Qwen-Image-2.1 on a NVIDIA 5070 Ti GPU (16gb vram) It fits, and it's fairly quick. ~25s for a 1024…

X AI KOLs Timeline ↗ · 2026-09-20 Cached

Tests of Qwen-Image-2.1 on an NVIDIA 5070 Ti GPU demonstrate it fits within 16GB VRAM, with generation times around 25s for 1024² images, and highlight excellent prompt adherence and typography capabilities.

0 favorites 0 likes
#gpu-performance

This draft model is OP on 16 GB cards for Qwen 3.8 27b

Reddit r/LocalLLaMA ↗ · 2026-09-12

A draft model for Qwen 3.8 27B is highly effective on 16 GB GPUs, achieving around 60 tokens per second on an RX 9070 XT and offering better VRAM efficiency than built-in MTP.

0 favorites 0 likes
#gpu-performance

@superalesha: Most of you dont know what an 8.4GB model can do on a 12GB RTX 3080 Ti. Qwen3.8-27B made this voxel Japanese pagoda gar…

X AI KOLs Timeline ↗ · 2026-09-02 Cached

Showcases the performance of the Qwen3.8-27B AI model running on a 12GB RTX 3080 Ti GPU, achieving 47 tokens per second at 128K context using llama.cpp with specific quantization settings.

0 favorites 0 likes
#gpu-performance

Ninfer and a 5090 with 3.8 27B is making me cry tears of joy it's so good.

Reddit r/LocalLLaMA ↗ · 2026-08-28

A user reports achieving high token throughput with the Ninfer tool on an NVIDIA RTX 5090 GPU using a Qwen 3.8B model, significantly outperforming llama.cpp.

0 favorites 0 likes
#gpu-performance

Compared Qwen 3.8 27B community quants on RTX 6000 vs Claude Opus 4.6

Reddit r/LocalLLaMA ↗ · 2026-08-26

A benchmark comparison of community quantized Qwen 3.8 27B models on RTX 6000 GPUs versus Claude Opus 4.6, evaluating token generation speed and efficiency for creating HTML games.

0 favorites 0 likes
#gpu-performance

LLM4LLM: Bridging Kernel Benchmarks and Real Deployment via Closed-Loop Agentic Optimization

arXiv cs.AI ↗ · 2026-08-25 Cached

LLM4LLM introduces a deployment-aware closed-loop optimization framework to bridge kernel benchmarks and real LLM inference, achieving up to 6.98x speedups on H100 GPUs.

0 favorites 0 likes
#gpu-performance

I hosted Kimi K3 (2.8T parameters) using 8 B300s. 92 tok/s, $190 per million tokens

Reddit r/LocalLLaMA ↗ · 2026-08-23

The article describes hosting the Kimi K3 AI model with 2.8 trillion parameters using 8 B300 GPUs, achieving 92 tokens per second and costing $190 per million tokens, while comparing it with Unsloth's dynamic GGUF quantization method.

0 favorites 0 likes
#gpu-performance

@Yuchenj_UW: UC Berkeley open-sourced FreeToken. Wild results: A single RTX PRO 6000 runs the 753B GLM-5.2 at 14.9 tok/s! An 8GB RTX…

X AI KOLs Following ↗ · 2026-08-21 Cached

UC Berkeley has open-sourced FreeToken, a tool that significantly speeds up local AI inference on consumer GPUs, achieving up to 39.3 tok/s on an 8GB RTX 4060 laptop.

0 favorites 0 likes
#gpu-performance

Qwen3.8 27b just exceeded my expectations on svg generation :D

Reddit r/LocalLLaMA ↗ · 2026-08-20

A user tested the Qwen3.8 27B model on SVG generation with a complex prompt and found the results exceeded expectations, highlighting the model's capability in creative tasks.

0 favorites 0 likes
#gpu-performance

I pushed Qwen3.8-27B to 99 tps single request and 1150 tps with a batch request on a RTX 3090

Reddit r/LocalLLaMA ↗ · 2026-08-17

The author optimized the Qwen3.8-27B model inference on an RTX 3090 GPU, achieving up to 99 tokens per second for single requests and 1150 tps with batch processing through various quantization and optimization techniques, and released the updated code on GitHub.

0 favorites 0 likes
#gpu-performance

@akshay_pachaar: how do you know whether your GPU is compute-bound or memory-bound? here's a simple explanation: your model weights sit …

X AI KOLs Following ↗ · 2026-08-16 Cached

The article explains how to determine if a GPU workload is compute-bound or memory-bound by analyzing operations per byte fetched from HBM, using NVIDIA's H100 as an example, and discusses how batching and prompt length affect performance.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback