Tag
The author details how they built an in-browser poker solver by using LLMs to generate custom WebGPU kernels instead of relying on a general tensor library, achieving over 10x speedup and demonstrating a paradigm shift where cheap generation can replace library abstraction.
在自定义Rust内核和Swift调度逻辑的优化下，Qwen3.6-35b-A3B模型的预填速度在2k提示下从255 tok/s提升到1058 tok/s，实现了约4倍的加速，解码和困惑度未受影响。
LFM2.5 230M model achieves 1,400 tokens per second in-browser using custom WebGPU kernels, demonstrating efficient local inference.
A developer successfully runs DeepSeek-V4-Flash (284B total, 13B active) locally on four RTX 2080 Ti GPUs with a $2,500 budget, achieving 255 prefill tokens/s using custom Turing CUDA kernels, W8A8 quantization, and heterogeneous inference. The implementation is open-sourced.