AI Agent Engineer in 2026: 12 steps roadmap - Loops, Graphs, Evals, Context, Harness (Full-Course)

This is the complete 12-step roadmap through the seven pillars that actually decide whether an agent works: context, tools, memory, loops, graphs, harness, evals - each with the Claude workflow that proves it.

Frontier models went from 30% to over 80% on SWE-bench Verified in a single year. Coding agents got dramatically, measurably better.

Meanwhile only 17% of executives say they have fully adopted AI agents across their company.

Follow my Substack to get fresh AI alpha:movez.substack.com

Agents fail because context bloats. Because loops never converge. Because nobody can measure whether last week’s change made anything better. The demo works on your machine. Production is a different animal.

This is the 12-step roadmap through the seven pillars that separate the two - each with its own failure modes, its own Claude workflow, and its own way of quietly killing your agent if you skip it.

Here’s the reframe that makes the rest coherent. These are not seven separate skills. They are one skill in seven projections.

• Bad context breaks the loop.

• A loop without evals never converges on anything you can trust.

• A graph without evals scales your error instead of your throughput.

• Memory without a harness evaporates the moment a session ends.

• Tools without context management drown the window before the work begins.

Which means there’s no useful way to learn them in isolation - but there is a dependency order, and it runs foundation first, execution second, reliability last. That’s how these twelve steps are arranged.

01. Context - read what actually loads

Karpathy’s framing is the one to hold: the model is the CPU, the context window is the RAM. Context engineering is the art of filling that RAM with exactly what the next step needs - no more.

The number that reframes everything: in Claude Code, roughly 7,850 tokens load before you type a single character - system prompt, auto memory, skill descriptions, CLAUDE.md, environment info, MCP tool names.

Your actual prompt is around 45. **Everyone optimizes the 45 and never opens the 7,850. **

/context prints your real breakdown by category, tells you which memory files loaded, and flags which tool call ate the most tokens.

Two numbers to watch: memory files (heavy means an overweight CLAUDE.md) and free space. Pair it with /memory to see the exact files in play.

02. Context - cut it, then layer it

Anthropic deleted over 80% of Claude Code’s system prompt for the Claude 5 generation and measured no loss on their coding evals.

Most context isn’t wrong - it’s guidance written for a weaker model that now just costs tokens and forces Claude to reconcile contradictions before it can start.

Two rules make the cutting safe:

• Delete in blocks, not lines - one sentence sits inside the noise floor of your evals and tells you nothing.

And convert absolutes into principles: instead of “never write multi-line comments,” say write code that reads like the surrounding code - match its comment density, naming, and idiom.

The rule gives a fixed answer, the principle gives Claude a way to find the right one by reading your repo.

• What survives goes into a tree, not a scroll. Anthropic’s guidance is a hard number: keep the project CLAUDE.md under 200 lines, holding only gotchas Claude can’t infer.

Everything else becomes a skill (description loads at startup, body on invoke) or a path-scoped rule (loads only when a matching file is read).

03. Tools & MCP - what the agent can reach

Tools are how an agent touches the world, and their descriptions are context - which makes this the hinge between what the agent can do and what it can afford to know about.

The old advice was to teach tools with examples.

That inverted: with current models, examples constrain Claude to the exploration space they describe. Design expressive parameters instead.

A status enum of pending | in_progress | completed teaches an entire lifecycle without a single example - the type is the documentation.

Anthropic reached state of the art on SWE-bench Verified partly through precise refinements to tool descriptions, not model changes.

And deferring beats dumping. Claude Code loads MCP tool names only - around 120 tokens - fetching schemas on demand through tool search.

Applying retrieval to tool descriptions rather than loading all of them improves selection accuracy roughly threefold.

Which makes the description the discovery surface: say when the tool applies, not just what it does. A tool Claude can’t find is a tool you didn’t ship.

04. Memory - what survives the window

Every long task eventually exceeds one window.

What happens at that boundary is a design decision most people never make - they let automatic compaction guess, then wonder why the agent forgot a constraint stated an hour ago.

The mechanics are specific and worth memorizing. Project-root CLAUDE.md and auto memory are re-injected from disk.

But rules scoped with paths: and nested CLAUDE.md files live in message history - they get summarized away and don’t return until a matching file is read again.

Invoked skill bodies come back, capped at 5k per skill and 25k total, oldest dropped first and truncated from the end - so anything critical belongs at the top of a SKILL.md.

The reliable move is the oldest one in computing: write it to a file.

A plan file, a progress log, notes the agent rewrites as it goes. It persists without occupying the window and survives compaction because it lives on disk.

Compress deliberately too - /compact focus on the auth bug keeps what you chose, and /clear is badly underused when the next task doesn’t depend on the last twenty messages.

05. Loops - when to stop

An agent loop is act → observe → decide → repeat. The entire engineering problem lives in that last step: how does it know it’s finished?

Left to itself, a model answers badly in two directions - declaring victory on half-built work, or grinding forever on something that was done three iterations ago.

Neither is fixed by a better prompt, because both are structural. The stopping condition belongs in code that sits outside the model’s judgment. For bounded work that’s a test gate or a schema check.

For discovery of unknown size, the converging pattern is loop-until-dry: keep going until K consecutive rounds surface nothing new.

One detail makes or breaks it, and almost everyone gets it wrong first time: dedupe against everything seen, not against confirmed results.

Otherwise rejected findings reappear every round, the loop never runs dry, and you’ve built a machine that pays forever to rediscover the same dead ends.

06. Loops - who checks the answer

A converging loop still converges on whatever the model believed. The fix is a verifier - something outside the model’s own judgment whose only job is to try to kill the finding.

If it survives, it passes. If not, it never reaches the answer.

Three patterns worth having in hand:

• Adversarial verify: for each finding, spawn N independent skeptics prompted to refute it; keep it only if a majority survive.

• Perspective-diverse verify: give each verifier a distinct lens - correctness, security, does-it-reproduce - because diversity catches failure modes that N identical checks never will.

• Judge panel: generate N attempts from different angles, score with parallel judges, synthesize from the winner while grafting the best of the runners-up.

Note where this points. A verifier is an eval running inline - the same discipline as Layer III, applied per-result instead of per-release. Teams that build good verifiers find step 11 much easier, because they’ve already written down what “good” means.

07. Graphs - what runs in parallel

Most people write agents as a straight line - step one, step two, step three, each politely waiting for the last. Then they notice half those steps never needed to wait at all.

A node is a unit of work an edge means this output feeds that input. If no data crosses, there is no edge - and the wait is pure waste.

The workhorse shape is the diamond: fan out to gather breadth, reduce with plain code, synthesize with one agent.

The reduce step deserves emphasis because it’s where money leaks - flattening and deduping is flatMap and a Set, not an agent.

Edges are free. Spend agents on judgment, not plumbing.

The ceiling is genuinely high. Claude Code’s dynamic workflows coordinate up to 1,000 parallel subagents in one run, and the orchestration costs zero model tokens because it’s a script, not a conversation.

That architecture is how a team ported ~960,000 lines of the Bun runtime from Zig to Rust in six days, with 99.8% of the test suite still passing.

08. Graphs - what the shape costs you

Topology isn’t cosmetic - it’s the biggest lever you have on both latency and spend, and two choices carry most of it.

First, parallel() versus pipeline(). A parallel() barrier makes everything wait for the slowest node before the next stage starts.

A pipeline() streams each item through all stages independently - item A can be in stage 3 while B is still in stage 1.

Default to pipeline. Reach for a barrier only when a stage genuinely needs every prior result at once, like a cross-set dedupe. “It felt cleaner” is not a reason; barrier latency is real, measurable, wasted time.

Second, model tiering per node. Every subagent inherits your session model unless the script overrides it, so a big run bills entirely at your top tier by default.

Bounded, repetitive nodes - extract this field, classify this ticket - belong on a cheaper model the merge node where judgment actually happens stays high.

A hundred cheap fan-out nodes feeding one top-tier synthesis costs a fraction of the same job run flat, at the same final quality.

09. Harness - surviving session death

Anthropic frames the problem perfectly: a long task is a software project staffed by engineers working in shifts, where each new engineer arrives with no memory of the previous shift.

Compaction alone doesn’t fix it. Even a frontier model looping across context windows on “build a clone of claude.ai” falls short - and it fails in two specific, repeatable ways.

First, the agent tries to one-shot the app, runs out of context mid-implementation, and leaves the next session a half-built undocumented feature to reverse-engineer.

Second, later in a project, an agent looks around, sees real progress, and declares the job done.

Both are fixed before any coding starts, by an initializer agent that runs once and builds the environment: an init.sh that starts the dev server, a progress log, an initial git commit, and a feature list in JSON - over 200 entries for the claude.ai clone, every one marked failing.

One detail worth stealing outright: they chose JSON over Markdown for that list, because models are measurably less likely to inappropriately rewrite JSON.

10. Harness - one increment per session

With the environment scaffolded, each coding session gets a contract: get oriented, pick exactly one feature, verify it as a user would, leave the repo clean.

“Clean” means what it means on a real team - no major bugs, orderly code, and a next engineer who can start work without cleaning up someone else’s mess.

The orientation ritual is mechanical and saves tokens every time: pwd, read the progress file, read the git log, read the feature list, run init.sh, and test that the basics still work before touching anything.

That last check matters more than it sounds - without it, an agent starting a new feature on top of a broken app just makes the breakage deeper.

And on marking things done: Claude’s tendency is to complete a change, run some unit tests, and call it finished without ever checking the feature end to end.

Give it real testing tools and require it to verify as a human user would - browser automation, actual clicks.

That single requirement dramatically improved performance in Anthropic’s experiments, catching bugs invisible from the code alone.

11. Evals - a number, not a vibe

The breaking point is always the same sentence: users report the agent feels worse after changes, and the team has no way to verify except guess-and-check.

Without evals, debugging is reactive - wait for complaints, reproduce manually, fix, hope nothing else regressed. You can’t tell a real regression from noise.

Start smaller than you think. Teams delay because they imagine needing hundreds of tasks; 20–50 drawn from real failures is a great start, because early changes have large effect sizes.

Pull them from what you already test manually, your bug tracker, your support queue. Write them so two domain experts would independently reach the same verdict - ambiguity in a task becomes noise in the metric.

And build balanced sets: test where a behavior should fire and where it shouldn’t, or you’ll optimize an agent that searches for everything.

Combine three grader types deliberately.

• Code-based - fast, cheap, objective; use wherever possible.

• Model-based - rubrics for nuance, calibrated against humans, ideally one isolated judge per dimension rather than one judge scoring everything.

• Human - the gold standard, used sparingly to calibrate the others.

And grade what the agent produced, not the path it took: checking for an exact sequence of tool calls is brittle, because agents regularly find valid approaches you didn’t anticipate.

12. Evals - keeping the number honest

A suite nobody reads is a number nobody should trust. You won’t know your graders work until you read transcripts from many trials - when a task fails, the transcript tells you whether the agent made a genuine mistake or your grader rejected a valid solution.

Failures should feel fair: obvious what went wrong and why.

Two traps make good agents look bad.

• A 0% pass rate across many trials usually means a broken task, not an incapable agent. Opus 4.5 initially scored 42% on CORE-Bench - then a researcher found rigid grading that rejected “96.12” when expecting “96.124991…”, ambiguous specs, and irreproducible tasks. After the fixes: 95%.

• The opposite trap is saturation - an eval at 100% tracks regressions but gives you no hill to climb, and real capability gains start showing up as noise.

Then hold two metrics apart. pass@k is the odds of at least one success in k tries - it rises with k. pass^k is the odds that all k succeed - it falls, fast. At 75% per trial, three trials all passing is only ~42%.

Use pass@k where one success is enough; use pass^k for anything customer-facing, where users expect it to work every time.

Finally, make it routine. Wire the suite into CI so it runs on every change and every model upgrade.

This is what turns a new model release from weeks of manual testing into a day of running your suite and reading the diff - and it’s the compounding advantage teams without evals never catch up on.

7 jobs to run with Claude - one per pillar

• Open the window you never looked at. Measure first, then cut. A heavy memory-files number means an overweight CLAUDE.md - diagnosed in one command, fixed in an afternoon.

› /context  then  /doctor

• Audit your tool descriptions. Every description should say when the tool applies, not just what it does. That sentence is the discovery surface - without it, a connected tool is one Claude never picks.

› Review every tool description in this MCP server. Add a “use when” sentence to each and tighten the parameter types.

• **Move state onto the filesystem. **Have Claude keep a plan file it rewrites as it works. It survives compaction because it lives on disk -and long tasks stop losing the thread halfway through.

› Before you start, write the plan to plan.md and update it after each step. Re-read it whenever you resume.

• Put a verifier on your loop. Pick any task where Claude decides for itself that it’s done. Add an external check and watch how often the first answer doesn’t survive three skeptics.

› After each fix, spawn three verifiers - correctness, security, reproducibility. Accept only what survives two of three.

• **Turn one linear task into a fan-out. **Find a task where you loop over files or sources sequentially. One agent per item, running at once, then one merge. The wall-clock difference is the whole lesson.

› Run a workflow to audit every route under src/routes/. One agent per file, verify each finding, then synthesize.

• Scaffold a multi-session project. Before a long build, have Claude write init.sh, a progress log, and a JSON feature list with everything marked failing.

› Act as an initializer agent: write init.sh, claude-progress.txt, and feature_list.json covering every requirement, all passes:false.

• **Turn last week’s bugs into a suite. **Open your bug tracker and convert real failures into 20 tasks with unambiguous pass criteria. That suite is worth more than any framework you could adopt this quarter.

› Here are 20 real failures. Write each as an eval task with deterministic graders where possible and a rubric where not.

**Conclusion: **

Anyone can get a demo working. The job is everything after that.

The models keep getting better on their own schedule, and that improvement is free - it arrives whether or not you did anything.

What doesn’t arrive for free is the system around the model.

• Context that stays lean. Tools an agent can actually pick between.

• Memory that survives the window. Loops that converge and get checked.

• Graphs that fan out instead of queueing. A harness that lets the next session continue the last one. And a number that tells you whether any of it got better.

Most people will keep waiting for a model good enough to not need any of this.

The ones who build the twelve steps will ship agents that work on days the model is having a bad one - which, in production, is the only reliability that has ever mattered.