Agentic Code Quality

The creator of Chrome DevTools, Addy Osmani: The center of gravity for code quality has shifted from code review to the harness — humans can’t read all the code agents produce, so quality is now backstopped by constraints.

“Agentic Code Quality”

Agentic Code Quality

Author: Addy Osmani (Google Engineering Lead, creator of Chrome DevTools)

For most of human history, we’ve evaluated code quality through code review: someone reads the code you wrote and confirms it’s clean, thoughtful, fast, understandable, and well-tested. For agents, this approach doesn’t scale — there’s simply too much code, and no human can read it all. As a result, more and more quality checks have to move into the harness, environment, and operating system around the agent. I still read code and do reviews myself, but I’m very deliberate about where I’m willing to let constraints act as the check.

Software quality now depends on the constraints you set for your agent.

Image: Guillermo Rauch’s checklist.

If you don’t read the code — either personally, or through agent-style questioning — at least one of the following is true:

• You’re a beginner • The software is throwaway • You’re prototyping • You have no users, no revenue • You’re taking on debt and risk • Your problems are trivial

By the way, all of these are fine. But the reality is, models aren’t at the “fully autonomous” stage yet.

Guillermo’s checklist is a good test for “can you afford not to read the code.” Notice that every “yes” is really about how low the stakes are — no users, throwaway code, prototypes. Once the stakes go up, something has to read the code. If it’s not you personally reading every diff, then it has to be constraints doing the reading.

Constraints define what the system is allowed to do: throwing tests and deterministic constraints at every proposal the agent makes. It’s by setting and maintaining these constraints that we build a loop capable of reliably delivering high-quality production software, even when agents generate hundreds of thousands or even millions of changes every day.

We call these constraints quality gates, and they come in many forms.

They include regular unit tests, property tests, and acceptance tests. They include mutation testing: generating mutations of the code, running the same test suite against them, and making sure no one sneaks in a bug we’d otherwise miss. They include code quality metrics like cyclomatic complexity and line length, which help keep code readable.

Image: Uncle Bob Martin’s original words.

I’m much older than you all — I started writing code in the late ’60s. My current strategy is: I don’t read a single line of code my agents write. It’s the only way I can enjoy their productivity. I do the exact opposite: I surround the agent with extreme constraints. Unit tests, Gherkin tests, QA processes, quality metrics, mutation testing, test coverage, and a bunch of others. In the end, I have very high confidence in the code they produce, because it has to run through an entire gauntlet of my constraints and tests.

Two people can disagree on “whether to read the code” while agreeing completely on the mechanism. Guillermo reads. Bob doesn’t read a single line. Both of them are describing the same gauntlet — the only difference is whether a human sits inside it. (I don’t agree with Bob’s other views)

Constraints also matter in another place: which proposals the system accepts and applies as code changes. When a change proposal travels from the running agent’s interpreter, through the agent controller, all the way to production, we’ve already run enough checks on it to be confident it’s safe to ship, and that its impact is tightly scoped to the agent’s area of responsibility.

An agent can propose anything. Your constraints determine whether a proposal is safe, correct, well-scoped, and useful enough for you and your team to ship it.

This model gives a lot, but it also misses a lot — and these gaps are worth thinking hard about today. One of those issues is autonomy: agents can execute intent well, but they can fail when information is missing or when what they’re trying to do is ambiguous. This applies both to the task itself and to the way the task is parameterized by the harness, environment, and other components.

Agents run into many of the same reasons humans can’t ship good code: fragile environments that buckle under script-driven pressure, nondeterministic builds, missing permissions, weak tests. This forces us to build a better environment: give agents feedback they can trust, allow low-destructiveness failure modes, and let success accumulate step by step.

The environment we want is one where agents can do real work, get feedback they can trust, and not cause much damage when they fail.

Another important issue is trust. We can’t naively hand intent to something — no matter how smart and robust it is, like a modern agent — without checking correctness. We start from trust, but trust has to be earned through capability.

Some constraints shape the work before it begins. Some give the agent feedback while it works. Some determine whether its output can cross the production boundary at all.

There are many ways to model the verification structure you wrap around a system.

Image: The autonomy ladder.

Autonomy is earned through verification loops, not granted by the model.

• Routine, already-validated changes (we’ve seen this kind of work before) → agent proceeds independently → ship → high autonomy • Non-trivial changes (real blast radius) → automated checks + targeted review → ship after review → conditional autonomy • Novel, high-risk, thin-evidence changes → human decides (credentials, permissions, funding, migrations, irreversible operations) → human decision

Autonomy is a property of the task, the evidence, and the harness — not of the model’s reputation, and not a permanent setting. Prove a change is routine and it moves up. Novelty, risk, and weak evidence hit resistance early. Consistent success earns more autonomy.

In my experience: pair your constraints with a broader but deliberately chosen set of checks, rather than relying on unit tests alone. The idea is that each check has its own distinct responsibility, from type safety and performance all the way to security scanning later on. You can also define your own constraints, including architecture rules enforced by lint tools like ESLint. Most of these tools have built-in hooks to pull in an agent or human when something goes wrong.

For now, the difference between agents producing something useful versus a pile of junk still largely comes down to the skill of the team operating the loop.

AI gives us massive code generation and speed, but that also means it’s getting harder and harder for humans to review every change. You have to do the opposite: deliberately plan where human attention goes. If you inject a manual review step into a system that otherwise runs at machine speed, don’t be surprised when productivity suffers. Human attention is scarce and precious, and we should actively steer it toward the most subtle, judgment-heavy problems. Downstream humans should only be pulled in when the automated guardrails fail.

The “code review” of the future will look very different

Correctness is one important dimension, but there are other things you care about: maintainability, performance, security, efficiency, and comprehensibility. Just as correctness can be decomposed into many signals, so can the rest of quality. And it’s not just about how many constraints we set — it’s whether they’re challenging enough to reach the bar we’ve set for quality and production readiness.

Software quality is not a single metric. Think of it as a set of signals, where each signal matters differently to you and your team.

Image: The constraint wheel. Set your constraints around your agent:

• Comprehensibility: review, answerability • Cost efficiency: token/compute budgets • Maintainability: coverage, complexity • Correctness: unit, property, mutation tests • Security: SAST, dependencies, secrets • Performance: performance budgets, load • Accessibility: axe, contrast, keyboard

Agent → ship: the exit gate. Too much code to read. Back-pressure, resisting bad work.

Back-pressure can be implemented through many tools: the compiler rejecting illegal code, failing tests, security policies blocking bad practices, CI refusing deployments. Ideally, it should run through the entire loop, not as a single review bolted on after all the work is done.

Image: Dex Horthy’s map of the same loop, from “Why Software Factories Fail.” The green boxes in the diagram are his argument: for now, human review should be brought back into the loop, not replaced by it. The flow in the diagram is roughly: user feedback enters monitoring and production, becomes a stream of tickets flowing into the issue system, orchestration hands tasks to agents building in a sandbox, artifacts pass through CI/CD checks, unit tests, static scanning, security checks, and agentic code review, then get handed back to a human for code review before shipping.

Constraints and back-pressure let agents catch bad work before it becomes a problem

What happens when the volume of changes is too much for the tools to digest and constraints can’t be applied? We end up with a queue backed up behind a validation system running at human speed. To scale, we need to push as many checks as possible into the validation loop itself, not leave them until the end. If we can scale within the automated checks, we can increase the speed and throughput of the entire delivery system. If there’s no room left in the validation loop, we have a few options.

First, scale the validation system to create more capacity to constrain and push back against incoming changes. Second, reduce the rate at which agents generate new changes so validation can keep up with the workload. Third, lower our quality bar so validation doesn’t have to push back as hard. From a scaling perspective, we need to be prepared for all of these. At the same time, we shouldn’t ignore that loosening constraints in some directions actually lets us do more work. Maybe we can increase the rate at which agents produce changes by using swarms of agent developers or automated software factories, letting them produce changes directly instead of waiting for us to review them one by one.

And in some places, we might want to give them more freedom, as long as we tighten elsewhere. Tighten constraints where it matters most, and you can maximize throughput without sacrificing quality. There are a lot of options in these decisions. The most obvious one: we have to make trade-offs between different dimensions of quality. As we’ve said repeatedly, security is incredibly important, but we’ve also had to choose between “shipping secure” and “shipping on time.” One end of this spectrum is innovation-driven, the other is quality-driven. At some point, we have to choose where we stand on that spectrum.

We want to feed the clear feedback from the environment and system back to the agent or team, so people can focus their energy on more subjective things: taste, intent, and architecture. If we can help humans stay within the safe bounds of the constraints, they won’t have to burn effort figuring out where things went wrong.

Software quality isn’t just correctness. Software quality also means maintainability, good performance, security, efficiency, and ease of understanding. All the constraints that help us meet those standards and keep production running will create back-pressure in our delivery pipeline.

We need to be deliberate about where we apply strong constraints and where we remove or relax them. Apply strong constraints where they serve both goals at once. If they don’t serve either goal, don’t keep them around. Be ready to raise or lower the bar as circumstances demand. And remember: it’s precisely these constraints at different points in the software system that make software quality enforceable.

We should apply strong constraints where they best serve this dual purpose, and consider removing or relaxing those that serve neither goal well. We should also be ready to move the quality bar up or down as needed. In fact, it’s these constraints at various points in the software system that give quality its teeth. In many cases, we can create more back-pressure and more constraints by deploying new tools or strengthening existing ones. All of this can be used to push back against the majority of change requests. We need to build them into the entire pipeline.

Image: The software factory. Intent comes in, agents implement, evidence determines what ships.

• Intent: issues, specs, goals • Shaping (before work begins): break into bounded subtasks, risk boundaries, intent + context • Feedback (while the agent works): sandbox + reproducible builds, tests, types, diagnostics, mechanical back-pressure • Boundaries (can it cross into production?): acceptance + QA, security scanning, release gates (CI) • Production: continuous monitoring

Humans only review exceptions: weak evidence, novelty, risk get escalated to them. Constraints define the boundaries of the work; agents iterate on their own; a passing test is a statement, not a verdict; every incident leaves behind a new test, monitor, or policy. Some constraints shape the work, some give feedback during it, and some determine whether it can cross into production. Generation and validation must scale together. Reject bad work fast, and vouch for what remains — that’s the advantage.

We don’t want to wait until the end of the pipeline for the CI system to tell us: don’t deploy until these issues are fixed. We want to use these signals as early as possible, through every possible channel. The ultimate constraint in this system is the one we put on ourselves: taking responsibility for every decision and action involved in building and operating the system. But like all other constraints, we need to make deliberate trade-offs: how much constraint and back-pressure we want our own judgment to exert, and whether we want it to serve as the final check.

Quality lives in the constraints we set for our agents. So when you’re thinking about quality for your own application, pick up this challenge and build your own constraint-driven approach.

Speaking of quality — agents are writing your code. Sonar gives you quality gates that make code shippable. It runs the same complete suite of checks on every commit: deep cross-file analysis, a risk distribution map, and a quality gate that holds every human and agent to the same standard.

This article was rated 100% human-written by Pangram 4.

Original: https://x.com/i/article/2087205551038230528… #AgenticCoding #CodeQuality #AIEngineering