Stop putting your AI agent’s memory inside the LLM context window

Hey everyone, been shipping a few agentic workflows into production lately and wanted to rant/share a massive architectural mistake I keep seeing people make. Stop treating the LLM context window or massive vector embedding as your agent’s long term memory. If your agent needs to hold state, remember past false positives, handle a human in the loop workflow or maintain an audit trail, passing a giant JSON blob of session history back and forth into the prompt is just a recipe for silent failures and terrible token bills. The only architecture that’s actually surviving production for us relies on a strict separation of concerns. First, durable state and memory has to live completely outside the agent in a boring, highly structured transactional database like Postgres or Lakebase. The agent should just read from it on boot and write to it on tool execution. It shouldn’t be the database. Second, use deterministic control flow. If you have an explicit business constraint like “always as a human before writing data”, code that logic into Python or a state graph framework like Langgraph. Don’t rely on system prompts to enforce safety boundaries. Lastly, treat the LLM as a judgement layer. Use the model strictly for processing the unstructured inputs, generating tool parameters or summarising evidence. Moving the state layer to a dedicated DB means that we can actually pause, replay and unit test agent execution without worrying about context drift or hallucinations wiping out the agent‘s history. Curious to hear how others are handling persistent state for multi-day workflows? Are you wrapping everything in custom SQL tables or relying on framework memory features? .