How we structure context for AI agents in production (static vs dynamic vs session layers)

The most common reason we have seen agents underperform in production is not the model and not the prompt but that the context architecture was never designed, it just accumulated over time. Everything ended up in the system prompt because that was the easiest place to put it, and the agent started breaking in ways that were hard to diagnose because the failure was structural rather than logical. The framing that has helped us most is treating context as three distinct layers that need to be managed separately, because each one has a different update cadence, a different failure mode, and a different recovery path when something goes wrong. The static layer covers everything that describes how the business works: terminology, decision boundaries, product rules, escalation paths. This changes rarely but it needs to be accurate when it does change, because stale static context produces confident wrong answers that are harder to catch than obvious errors. We keep this outside the system prompt in a structured knowledge base the agent retrieves from rather than embedding it directly, which makes it easier to update without touching the agent logic itself. The dynamic layer is live data the agent needs at runtime: order status, customer history, inventory, account details. This is where most context gaps actually live in production, because agents that look impressive in testing often had clean, complete dynamic data in the test environment that the production environment does not reliably provide. The fix is not better prompting but making the dynamic data retrieval explicit about what it found versus what it could not find, so the agent is not filling gaps silently with inference. The session layer covers what has happened in this specific conversation or workflow run, and the mistake we see most often here is storing raw conversation history and passing it forward, which balloons the context window and buries the signal in noise. Storing structured decision records instead, meaning what was decided, on what basis, and at what point in the run, keeps the session context lean and makes the agent's reasoning auditable when something goes wrong later. The architectural question worth asking before building any agent is which of these three layers is most likely to cause a failure in your specific context, and whether you have a recovery path when it does, because most agent reliability problems we have seen trace back to one of the three being missing, stale, or collapsed into one of the others in a way that makes it impossible to update or debug independently.