serving-engines

AutoTuneBench presents a benchmark and measurement protocol for trustworthy agent auto-tuning of LLM serving engines, addressing failure modes like strawman baselines and infrastructure defects to ensure reliable results.

A detailed guide on learning AI inference engine internals, covering serving engines like vLLM and SGLang, low-level GPU kernel programming with Triton and CUTLASS, and a sequence of mini-projects to build hands-on expertise.