gpu

Rgpu 是一个开源工具，可通过 TCP 让 PyTorch 张量和现有 CUDA 程序在远程 NVIDIA GPU 上执行计算，客户端应用保持在本地运行，提供 PyTorch 设备接口和 CUDA shim 两条路径。

An in-depth explainer covering parallelism strategies for LLM inference—data, tensor, pipeline, context, and expert parallelism—explaining how to split inference across GPUs without losing performance to communication overhead. It walks through prefill vs. decode phases, memory and communication bottlenecks, and how to choose a defensible layout given model, hardware, and workload constraints.

The author reassesses hardware choices for running Qwen Flash Next locally under the Strata streaming architecture, finding that memory bandwidth and PCIe speed have an enormous impact on older platforms, while newer platforms can get close to 5090-level performance with just two 5070 Ti cards. The value-for-money calculus flips completely.

Meta has consolidated PyTorch's media processing stack over two years, making TorchCodec the single home for CPU and CUDA decoding/encoding of images, video, and audio, while TorchVision and TorchAudio now focus strictly on high-performance media transforms.

The post reports that Strata's boost technology has completely reshaped the hardware landscape for locally running Qwen Flash Next on a single machine: NVIDIA cards with large memory (RTX 3090/4090/5090, even mid-range options like 5060/3060 X2) now clearly outperform formerly competitive Apple Mac M5-series configurations, with demand surging for the 3090, 4090, and 5090.

Comment on the launch of 'Core,' a personal AI computer, highlighting that its GPU alone is worth $3,200 alongside 64GB DDR5, NVMe storage, and pre-configured software.

A tweet discussing the reasons behind the $6,000 price tag on GPUs, linking to external content explaining the cost drivers.

Gitframes is an open-source npm package that packs Photoshop-, After Effects-, and Blender-class video capabilities into a single code-first tool, rendering directly on WebGPU (Dawn/Metal/Vulkan) for AI coding agents to compose and export MP4s via TypeScript.

The author highlights the growing buzz around Strata, an open-source local LLM inference engine featuring a quantized 125B Qwen3.8-Flash-Next, and compares it with antirez's ds4 (DwarfStar) engine, noting both support speculative decoding, CPU/GPU collaboration, and integration with tools like Claude Code and Codex.

SparkTrace LLM 是一个新基准，用 Blackwell GPU 的真实计时结果评估 LLM 阅读 Nsight Compute 性能剖析输出后，能否正确选择最快的 CUDA kernel 重写版本。实验发现包括 GPT 5.4 nano 在内的一些模型会被性能剖析信息误导而选错 kernel。

The author believes that after the Strata revolution, prices for memory above 32G and mid-range graphics cards above 16G are very likely to rise, and suggests that a configuration of 64G RAM + 16G VRAM can run Strata for just over a thousand yuan. The quoted post shows that DDR5 prices (especially 16G and 32G) have already deteriorated significantly.

Prime Intellect 今日发布 Prime Inference，提供面向前沿开源模型的低延迟、高可靠的推理服务，涵盖无服务器端点与预留容量，已在 NVIDIA Blackwell GPU 集群上为 GLM-5.3 等模型提供近乎零工具调用错误率和 100% 可用性的生产级部署。

A post recommends grabbing an RTX 5060 Ti before prices rise, arguing that a native FP4 GPU paired with 128GB DDR4 can run a state-of-the-art Qwen model locally for around $1.5k — an accessible first real local LLM setup.

A developer (coffeecup2020) built a custom inference engine called tqllm that runs Qwen3-27B (TQ3_4S-v2 quantization) on an RTX 3090 at an impressive 273 tokens per second — the post contextualizes that speed against human reading rates and notes a broader recent wave of lightweight LLM inference releases, including Strata running quantized Qwen3.8-Flash-Next on 12GB VRAM.

The author shares a ready-to-paste local AI model installer script, built with the Muse model in mind but designed to work with most models, and tested using Hermes for the PC agent portion of the install process.

This long post is a real-world test of running Qwen3.8-27B on two second-hand V100 32G GPUs. By rewriting the kernel of the inference engine NInfer, adding dual-GPU support and speculative decoding, generation speed on a 200K-token code task jumped from 36 tok/s to roughly 100 tok/s, beating the 80.9 tok/s of a single 4090 48G. The post also analyzes how VRAM bandwidth, quantization, and NVLink affect inference performance.

Valve Linux graphics engineer Timur Kristóf presented at XDC 2026 on his work migrating legacy AMD GCN 1.0/1.1 GPUs from the Radeon driver to the modern AMDGPU kernel driver, unlocking RADV Vulkan support, better performance, and improved power management for decade-old cards.

Vx is a systems programming language for heterogeneous computing (CPU, GPU, NPU) that encodes device memory topologies directly in its type system, using machine files and formal verification to catch data-movement and memory bugs at compile time.

Gufo is a vertical local inference engine optimized for AMD Strix Halo (Ryzen AI MAX+ 395 with Radeon 8060S, up to 128 GiB unified memory), with a new Windows pre-package release supporting Qwen3.8, DeepSeek V4 Flash, ASR/TTS, image and video models at high throughput benchmarks.

mlsubgen is an open-source CLI tool that generates video subtitles in 45 languages entirely on a local machine, using CUDA-based ASR (Qwen3-ASR, Whisper large-v3) and Ollama-hosted LLM translation with hardware profiles down to 8 GB GPUs.