How to Build a Multi-Agent Engineering System

How to use Agent Teams to accomplish tasks that a single Agent cannot. The core message of Luke’s talk boils down to one sentence:

The bottleneck in software engineering is no longer intelligence, but human attention.

Even top-tier engineers can only advance a few tasks a day, yet the real-world backlog might include 50 features, 20 bugs, 10 migrations, and a pile of legacy systems to clean up. Models are getting smarter, and theoretically, many tasks can be handled by them. What is truly missing is the ability to execute complex tasks over long periods with low supervision and verifiable results.

Luke’s goal is straightforward: After listening to this 20-minute talk, you should understand how to assemble an Agent team capable of running for days to complete complex software engineering tasks that a single Agent cannot handle.

His background is solid: Luke previously spun out Goose from Block, a project now donated to the Agentic AI Foundation. He is now responsible for the core Agent Harness at Factory, with the goal of gradually moving the entire software development lifecycle toward autonomy.

Multi-Agent Is Not Just Stacking Bots; It’s a Collaboration Pattern

The landscape of multi-agent frameworks is currently chaotic. Everyone talks about “multi-agent,” but there is no unified definition of what it means. Luke started by doing something valuable: creating a taxonomy for frontier patterns.

He breaks down multi-agent collaboration into five typical patterns.

The first is Delegation, where a parent Agent spawns a child Agent. For example, if the main Agent determines that the current task requires database design, it dispatches a child Agent specifically to design the schema. This is the most common pattern and the first one supported by many tools.

The second is Creator-Verifier. One Agent writes code, while another, completely independent Agent reviews it. The key to this pattern is solving sunk cost bias. The Agent writing the code tends to defend its implementation, whereas an independent verifier lacks this psychological baggage and is more likely to spot issues.

The third is Direct Communication, where Agents chat directly with each other. While this sounds natural, it is actually difficult because state becomes fragmented, there is no single source of truth, and it often leads to a situation where “every Agent thinks it knows the global state, but no one truly does.”

The fourth is Negotiation. When multiple Agents work around shared resources—such as the same API, module, or code segment—negotiation is required. Good negotiation doesn’t mean blocking each other; it means finding positive-sum solutions.

The fifth is Broadcast. One Agent pushes new constraints, state updates, or plan changes to everyone. This pattern is particularly important for long-term tasks because, once a task runs for days, the biggest fear is context drift. The value of Broadcast is keeping the entire system continuously aligned.

These five patterns may seem simple, but the Missions system at Factory essentially combines them into a closed loop capable of running long-term.

Missions: Not an Agent Session, but an Agent Ecosystem

Factory’s core system is called Missions.

It is not just a longer Agent session, nor is it about artificially inflating the context window. It is an Agent ecosystem composed of multiple roles, shared state, structured handoffs, and verification mechanisms.

There are three core roles in Missions: Orchestrator, Workers, and Validators.

The Orchestrator is the commander. It helps you scope requirements in conversation, asks strategic questions, and turns vague goals into actionable plans. Most importantly, before writing code, it outputs two things: a Plan and a Validation Contract.

The Plan defines how to do it. The Validation Contract defines what “Done” looks like.

This is critical. Traditional Agents often write code first and add tests later, resulting in tests that accommodate the implementation rather than truly verifying the requirements. Missions flips this by defining acceptance criteria before implementation.

Workers are the laborers. They receive a clean context and are responsible for implementing specific features. Once a Worker finishes, it submits a Git Commit. The next Worker inherits a clean codebase, not a messy chat log.

Validators are the auditors. They do not try to smooth things over; they strictly verify whether the task is truly complete.

The biggest innovation here is: The Validation Contract is written during the planning phase.

It may contain hundreds of assertions independent of the implementation, with each feature bound to corresponding assertions. Upon completing a milestone, the system must prove these assertions are covered, not just “look approximately correct.”

The Verification System Is the Core Preventing Drift in Long-Term Tasks

The biggest problem with Agents writing code is often not that the first step is done poorly, but that long-term tasks drift.

The initial direction is correct, but as writing continues, it goes off track; after a while, tests start accommodating the implementation; eventually, the system looks busy but moves further away from the actual goal.

The verification system in Missions is designed to solve this.

The first type of verifier is the Scrutiny Validator. It runs tests, type checks, and linters, and spawns independent Code Review Agents for each feature. This Code Review Agent operates in a completely new context and was not involved in the implementation, making it more like a true external reviewer.

The second type is stronger: the User Testing Validator. Like QA, it launches the application, uses Computer Use to actually click pages, fill out forms, and go through complete workflows. In other words, it doesn’t just look at code; it verifies whether the product actually works, just like a user would.

Both types of verification are adversarial. The verifiers haven’t seen the implementation process and have no sunk costs, so they won’t naturally side with the implementer.

Luke mentioned an important data point: the verification phase consumes most of the wall clock time, but this is precisely the key to keeping long-term tasks on track. It sacrifices some speed in exchange for long-term correctness.

Structured Handoff: Not Relying on Memory, but on Forced Documentation

To keep a task running for days or even weeks, the biggest danger isn’t that the model isn’t smart enough, but that information is lost.

Therefore, Missions places great emphasis on Structured Handoff.

After each Worker completes a feature, it must fill out a handoff form. This must clearly state: what was completed, what was left behind, which commands were run, what the exit codes were, what issues were discovered, and whether the Orchestrator’s defined processes were followed.

This may sound like a cumbersome process, but it solves the most fatal problem in long-term tasks: the system cannot run on memory; it must run on structured records.

At milestone boundaries, the system checks all handoffs. If unresolved issues are found, it automatically creates Follow-up Features to bring the task back on track.

This is Missions’ self-healing mechanism. It doesn’t assume Agents never make mistakes; instead, it assumes they will, and uses structured handoffs and milestone checks to expose errors in time.

Luke said the longest Mission has already run for 16 days, with a goal of reaching 30 days. This number itself indicates that the core of a long-term Agent system is not bigger prompts, but stronger handoff and verification discipline.

Why Not Full Parallelism? Because Software Engineering Is Not Simple Subcontracting

In the early days, they tried full parallelism, but it failed.

The reasons were practical: conflicts, duplicated effort, and architectural inconsistency. Multiple Agents modifying code simultaneously may look fast, but the cost of merging and fixing issues explodes.

Therefore, Missions adopts a more restrained strategy: Serial Mainline + Local Parallelism.

Features are processed serially. At any given time, only one Worker and its corresponding Validator advance the mainline task, ensuring the evolution of the codebase is continuous and understandable.

However, read-only operations can be parallelized, such as code search, API research, and code review. These tasks do not directly modify the mainline, so parallelization yields high benefits with lower risk.

This design closely mirrors a real engineering team: core architecture and mainline development require sequence, but research, reviews, and information gathering can be parallel. It doesn’t pursue maximum superficial concurrency but aims for the compound interest of long-term correctness.

Mission Control: Multi-Day Tasks Need a Dedicated Dashboard

When a task runs for days, traditional chat windows are no longer sufficient.

So, Factory built Mission Control, a dashboard specifically for viewing long-term tasks.

You can see in real-time what the current Worker is doing, the handoff summary, the next steps, the project completion status, and how much of the token budget has been consumed.

More importantly, it supports full asynchronicity. You can go to sleep, hang out with friends, or attend meetings while the system continues running. When you return, you aren’t facing a wall of chat logs but a structured state: where you are now, what has been completed, where it’s stuck, and what to do next.

This is the difference between an Agent Harness and a regular Chatbot. A Chatbot is a conversational interface; a Harness is a task execution system.

Model Selection Is an Art: Droid Whispering

Luke refers to model selection as Droid Whispering, a fitting term.

No single model is perfect for all roles. Planning requires a strong, slow-thinking model because it needs to perform strategic breakdowns and constraint design. Implementation requires a model with strong code fluency and creativity to rapidly land features. Validation requires a model that precisely follows instructions because it must strictly check results against the acceptance contract.

He also suggests that different roles should ideally use different providers to avoid source bias. For example, one model family handles writing, while another handles reviewing, making it easier to spot blind spots.

Interestingly, the structured architecture of Missions also allows open-source models to perform well. This is because mechanisms like Validation Contracts, milestone checks, and handoffs provide discipline. In other words, the system doesn’t rely entirely on the model’s self-awareness; it constrains model behavior through architecture.

Real-World Case: Building a Slack Clone from Scratch

Luke gave a real-world example: building a Slack clone from scratch.

In this task, approximately 60% of the time and tokens were spent on Implementation. However, validation rarely passes on the first try; the system automatically creates Follow-up Features to continue fixing and advancing.

The final result was that about 50% of the code was tests, with coverage exceeding 90%. This is interesting: the Agent didn’t just produce more code; under the drive of verification mechanisms, it produced a more maintainable codebase.

They also heavily used Prompt Caching to control costs; otherwise, the token consumption for such long-term tasks would be astronomical.

Enterprise use cases are also clear: prototyping a new feature overnight, large-scale refactors or migrations, modernizing legacy codebases, and making it easier for subsequent Agents to take over.

These are exactly the tasks where a single Agent is most likely to fail: long cycle times, many dependencies, complex context, and difficult verification. The value of Missions lies in breaking these tasks into handoff-able, verifiable, and recoverable long-term execution flows.

Architectural Philosophy: Embracing the Bitter Lesson

What impressed me most about this talk was Luke’s architectural trade-offs.

They didn’t write all orchestration logic into complex hardcoded state machines. Instead, much of the logic is written in Prompts + Skills, with the entire system consisting of only about 700 lines of text.

Why?

Because this embraces the Bitter Lesson: as models upgrade, the system automatically gets stronger.

If you hardcode too much intelligence into the code, you don’t benefit from model improvements. But if you write behavioral patterns, role definitions, and collaboration protocols in prompts and skills, then the stronger the model, the stronger the entire system.

Of course, it’s not entirely devoid of deterministic logic. Missions still retains a thin layer of hard logic, such as handoff checks, blocking conditions, and milestone boundaries. Beyond that, it leaves intelligence to the model as much as possible.

This is likely the direction future Agent Harnesses will move toward: Deterministic logic handles boundaries and discipline; models handle judgment and execution.

What Truly Changes Is the Economics of Software Engineering

Missions ultimately changes not “how much code a single Agent can write,” but the team’s throughput model.

Luke said that a 5-person team might previously advance 10 workstreams simultaneously. With Missions, they might advance 30.

This isn’t because humans suddenly become three times more productive, but because humans are liberated from execution to move to higher-leverage positions: architectural judgment, product decisions, acceptance criteria, and priority sorting.

Execution is handed to the system, verification to the system, handoff to the system, and self-healing to the system. Humans truly take responsibility for defining direction and making critical judgments.

More importantly, the codebases produced by Missions don’t get messier over time; they may become cleaner than when they started. Because they leave behind tests, structure, Skills, handoffs, and validation contracts. These elements form a positive flywheel, making it easier for subsequent Agents to continue working.

Summary in One Sentence

Missions is essentially a combination of Delegation, Creator-Verifier, Broadcast, Negotiation, Structured Handoff, and Validation Contract.

Its true strength isn’t making Agents appear smarter, but completely decoupling “humans defining what they want” from “the system autonomously executing for multiple days.”

Humans are responsible for scoping, planning, acceptance, and critical decisions; the Agent team is responsible for implementation, verification, handoff, and repair.

This is where multi-agent truly adds value: not a few Agents chatting in a room, but a production system that is runnable, verifiable, and recoverable.

Luke’s final advice is practical: try /missions on Open Droid, argue clearly with the Orchestrator about the Scope, carefully approve the Plan, and then you can go do other things.

His last words are worth remembering:

Those who develop the intuition for “how different models collaborate under long-term, high-pressure tasks” will lead the next generation of innovation.

For more inspiring talks from AI Engineers, welcome to Podwise to get free series breakdowns