@bcherny: LLMs still produce bugs, but those bugs are different than what they used to be. It’s less off-by-ones and more about s…
Summary
A developer observes that LLM-generated bugs have shifted from off-by-one errors to higher-level design and context issues, and recommends using adversarial code review (e.g., Claude's /code-review) to catch them.
View Cached Full Text
Cached at: 08/13/26, 03:31 PM
LLMs still produce bugs, but those bugs are different than what they used to be. It’s less off-by-ones and more about system design, ui usability, missing broader context. Some kinds of coding has been solved, but not all.
While models continue to improve, adversarial code review has been an incredibly powerful tool to catch many of these kinds of bugs.
It can be as simple as a one line prompt - “use a dynamic workflow to adversarial test every edge case in an iOS simulator”, or use Claude’s built in /code-review (or /code-review low, /code-review medium, etc.)
Jarred Sumner@jarredsumner·Aug 11: doesn’t happen anymore x.com/jarredsumner/s…
Similar Articles
LLMs are bad at vibing specifications
Hillel Wayne discusses how LLMs, while popular for writing formal specifications like TLA+ and Alloy, often produce shallow, tautological properties that fail to capture subtle bugs, based on analysis of community projects.
@msimoni: So far, I haven't been able to make LLMs significantly help me with the hard parts of programming - designing code that…
The author finds that LLMs are not significantly helpful for the hard parts of programming (designing simple, general, clear code) but are so helpful for other parts that they rate them as a 10x productivity improvement.
@mattpocockuk: LLM's are notoriously bad at following negative instructions That's why code review works so well, it turns all the neg…
Matt Pocock observes that LLMs struggle with negative instructions, and explains how code review converts those into positive, actionable directives.
2x, not 10x: coding with LLMs in 2026
The author argues that LLMs currently provide about a 2x productivity boost for coding due to their ability to handle easily verifiable tasks, but fundamental limitations prevent a 10x improvement; further gains will come from retooling around existing capabilities rather than model improvements.
The Patchwork Problem in LLM-Generated Code
This paper formalizes the 'patchwork problem' where LLM-generated code is locally correct but structurally incoherent across a codebase, proposes a taxonomy of eight failure categories and a hybrid verification framework, and demonstrates that many failures evade current tools.