How Uber Eats Uses a Self-Tuning AI Multi-Agent System (And Why It Matters)

Uber Eats processes millions of food photos across 10,000+ cities. To fix bad merchant photos without generating fake-looking "AI slop," they built an automated multi-agent architecture. Here is how it works under the hood: 1. The Architecture (The "Swiss Cheese" Model) Instead of one giant AI model, Uber stacks specialized agents in layers: Router Agent: Decides whether to fix a photo or skip it (optimized for high recall so bad photos don't slip through). Editor & QA Agents: Run in a continuous loop to edit the photo, evaluate quality, and retry if needed. Publish-Ready Gate: The final safety net checking for policy violations, physical glitches, and brand alignment before going live. 2. How the Agents Collaborate Pairwise Comparisons: The QA agent puts the original and edited photos side-by-side to check for hallucinations (e.g., adding 2 extra chicken wings or removing dipping sauce). Self-Correction: If QA rejects an edit, it feeds explicit instructions back to the editor ("Fix portion size"), capping attempts at K retries (Pass@K metric). 3. Why Centralized Logging is Non-Negotiable All agents output into a single, flat JSON log trace. Anyone—engineers, product managers, or designers—can inspect exact failure points. Takeaway: You cannot optimize or auto-tune an agent pipeline if you don't log every micro-decision first. 4. The Magic: Self-Auto-Tuning Loop When models drift or fail on edge cases, no human writes new code. An autonomous Diagnoser Agent takes over: Finds the Fault: Inspects production logs and human feedback to pinpoint which agent messed up. Reflects & Synthesizes: Sub-agents analyze failure patterns and rewrite the prompt configurations automatically. Benchmarks: The new prompt is tested against an immovable "Golden Dataset" of human-labeled photos. If it passes, it auto-deploys straight to production.