@barrowjoseph: My full review of @philipkiely's Inference Engineering. TL;DR: I desperately wish I could ship this book back to myself…
Summary
A review of Philip Kiely's book 'Inference Engineering', recommending it to avoid common mistakes in AI inference engineering.
View Cached Full Text
Cached at: 06/17/26, 08:03 PM
My full review of @philipkiely’s Inference Engineering.
TL;DR: I desperately wish I could ship this book back to myself in 2023. But you should read it today to avoid making the same mistakes I did!
Similar Articles
A Guide to AI Inference Engineering (17 minute read)
This guide explains the discipline of AI inference engineering, covering the split between prefill and decoding phases, the shift from closed to open models, and optimization techniques for latency, throughput, and cost.
@polydao: This Stanford lecture on AI inference will teach you more about how LLMs work in production than most ML courses > Clau…
A Stanford lecture on AI inference emphasizes practical bottlenecks like KV-cache and techniques like speculative decoding and continuous batching, offering more real-world insight than typical ML courses.
The Final Bottleneck
A reflective blog post on how AI acceleration in code generation overwhelms review processes, creating a new bottleneck in software engineering. Draws parallels to historical industrial bottlenecks and suggests throttling input as a necessary response.
Reviewed 250+ real AI implementations, a few things surprised me...
The author shares insights from reviewing 250+ real-world AI implementations, highlighting that Engineering and Finance are leading adoption while most outcomes focus on speed rather than cost reduction or revenue growth.
AI is eating the AI Engineering Loop (5 minute read)
The article discusses how the AI engineering loop can be fully automated but argues that handing over the entire loop produces 'agent slop' due to imperfect evals. It recommends automating certain steps while keeping human judgment for nuance.