A linter for PyTorch 'torch-preflight' [P]
Summary
torch-preflight is a new linter for PyTorch code that catches common GPU-wasting bugs without executing code or requiring a GPU, plus estimates VRAM usage to check if a training run fits before paying for an instance.
Similar Articles
Built an LLM training framework that actually runs on older GPUs without crashing
Introduces Picotron, a clean-room rewrite of Nanotron that eliminates mandatory GPU-specific dependencies, enabling LLM training on older GPUs like T4 and V100. It defaults to standard PyTorch SDPA but supports FlashAttention-2 at runtime.
PreFT: Prefill-only finetuning for efficient inference
PreFT proposes applying adapters only to prefill tokens, discarding them during decode, which increases throughput for multi-adapter serving with minimal performance loss.
A hackable compiler to generate efficient fused GPU kernels for AI models [P]
The author presents a custom, hackable ML compiler written in Python that lowers LLMs to optimized CUDA kernels through a multi-stage IR pipeline, achieving performance competitive with or superior to PyTorch on specific operations. The article details the compiler's optimization passes, lowering rules, and CLI usage for generating efficient fused GPU kernels.
@anyscalecompute: LLM post-training is the new baseline. Picking the wrong method or GPU config is how you waste a 36-hour run. Introduci…
Anyscale introduces a new Agent Skill for LLM post-training that automatically selects the optimal fine-tuning method (SFT, DPO, GRPO, etc.) and generates ready-to-launch configs, helping avoid wasted GPU runs.
PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX
PTXBench is introduced as a benchmark to evaluate and adapt large language models for optimizing GPU kernels using architecture-specific PTX, showing uneven performance and fine-tuning insights.