Last week I built an AI Agent, this week I added memory!

Been building an ecommerce AI agent from scratch. Maybe you saw last reddit thread as well 🤫 No LangChain. Just raw Anthropic SDK + TypeScript with Node.js. After getting the basic tool calling loop working, the first thing I added was memory. The idea looked really simple: Save the conversation. Load it again when the user comes back. (Episodic memory) // save messages after every turn ConversationStore.saveMessages(sessionId, result.messages); // restore conversation const history = ConversationStore.getMessages(sessionId); And it worked. The agent restarted and still remembered previous conversations. Then I checked the stored JSON file after a few days... { "sessions": { "USER_1": { "messages": [ { "role": "user", "content": "hi" }, { "role": "assistant", "content": [ { "type": "tool_use", "name": "search_products" } ] } // hundreds of more messages... ] } } } I had created another problem. Every request was sending: • old user messages • assistant responses • tool calls • tool results • intermediate reasoning context Hundreds of messages were going back to the LLM every single time. Memory worked. But cost increased and context started filling quickly. The mistake was treating all memory as the same. AI agents usually need different types of memory: Working Memory This is your current messages array. The short term context your agent is using right now. while (true) { const response = await llm(messages); if (!toolCalls) break; const results = await runTools(toolCalls); messages.push(results); } Fast and simple. But restart your process and it's gone. Episodic Memory This is what I implemented first. Basically the conversation timeline. class ConversationStore { saveMessages(sessionId, messages) { store.sessions[sessionId] = { updatedAt: new Date(), messages }; saveStore(store); } } Useful because the agent remembers exactly what happened. But storing everything forever doesn't scale. A 10 minute conversation is fine. A customer using your agent for months? Not fine. Semantic Memory This is the better approach for long term memory. Don't remember every sentence. Remember important facts. Example: Instead of: "User asked for running shoes" "Assistant searched Nike shoes" "Tool returned 20 products" "User selected size 10" Store: { "name": "Nikhil", "preferences": [ "likes running shoes", "shoe size 10", "prefers Nike" ] } After a session, run a cheaper model: const memory = await anthropic.messages.create({ model: "claude-haiku-4-5", system: ` Extract important user facts. Return JSON only. `, messages: [{ role: "user", content: JSON.stringify(messages) }] }); Next conversation: Send 5 useful facts. Not 500 old messages. Same personalization. Much smaller context. Much cheaper. Procedural Memory This is not about the user. It's about how your agent behaves. Similar to Claude Skills or Cursor rules. Example: Always show prices in INR. Never suggest unavailable products. Ask size before recommending shoes. It rarely changes. It's basically your agent's operating manual. The architecture I'm moving towards: Working Memory = current conversation Episodic Memory = raw history when needed Semantic Memory = long term user knowledge Procedural Memory = agent behaviour/ you know SKILLS MD file The interesting part is that "adding memory" is easy. Designing memory that still works after thousands of users is the real challenge. Building the semantic extraction layer next. Would love to hear how others are handling this in production. You would definitely need to use database like vectors/postgresql. Ref Video Attached in comment