@huxlab: How to do production-grade Agent continual learning? Replit gives the most solid answer. Replit AI head Michele Catasta (@pirroh) shares a very substantive article about how they built a continual learning system for the Replit Agent over the past year...

How to do production-grade Agent continual learning? Replit gives the most solid answer. Replit AI head Michele Catasta (@pirroh) shares a very substantive article about how they built a continual learning system for the Replit Agent over the past year. The core insight directly targets the current pain point in Agent development: Most production Agents are based on closed-source frontier models (Fable 5, GPT-5.6, etc.) and cannot be fine-tuned at the weight level. This means traditional Continual Learning is infeasible for most teams. But Agents can still keep improving—the key is to break improvements into three layers: • Model layer: limited by closed-source models, little room for change • Harness layer: continuously optimize code, tools, and instructions via production traces (fully controllable) • Context layer: personalize at the agent / user / org level (gets better every day) Replit's approach turns evaluation and improvement into a closed-loop system: • Offline validation using their in-house ViBench (an end-to-end benchmark for "vibe coding") • Online validation through A/B testing and real user behavior • Clustering production traces with Telescope to discover hidden failure patterns • Finally, letting the Agent itself participate in the improvement loop (read trace → propose hypothesis → modify code → validate → suggest whether to ship) This system allows them to quickly convert real production failures into daily deployable improvements, rather than relying solely on model updates. If you are building a production-grade AI Agent and care about how to continuously improve the system without model updates, this article is worth reading carefully. It contains many actionable engineering details and trade-offs.