An educational walkthrough of what happens between a user's message and a chatbot's reply, covering tokenization, context windows, conversation memory, chunking/embeddings, RAG retrieval, system prompts, function calling, and safety evaluation.
Crash course: how a real chatbot gets built from scratch I'll walk you through the whole journey, from the customer's first message to the reply, and everything that happens in between. 1. The model doesn't read words, it reads tokens Every sentence gets chopped into small pieces called tokens, and the model predicts the next piece, one at a time. There's also a limit on how much it can read at once, called the context window. And you pay for every token, so cost and speed are part of the design from day one. 2. The model remembers nothing Every time you send a message, the code sends the entire conversation history along with it. Memory is an illusion, and the developer is the one building it. When conversations get long, you have to summarize or trim, and that decision affects quality. 3. Your data: chunks and embeddings Your files (PDFs, pricing, policies) get split into chunks. Each chunk is turned into a list of numbers (an embedding) that represents its meaning, then stored in a vector database. If the chunks are too big, answers get muddy. If they're too small, the meaning gets cut off. 4. Retrieval The customer's question also gets turned into numbers, and the system finds the 3 to 5 chunks closest in meaning. It sends those to the model along with the question: "Answer from this text only." That's RAG, and it's what cuts down on made-up answers. 5. The system prompt A fixed set of instructions goes out with every message: personality, language, boundaries, and what to do when it can't find an answer. How it's worded matters a lot. 6. Tools (function calling) The model can't actually do anything on its own. We describe tools to it (book an appointment, check an order), and it replies with "I want to call tool X with these values." The code is what actually runs it and hands the result back. 7. Safety and evaluation Before launch, you test hundreds of questions and measure accuracy. You also guard against people trying to trick the bot, and you keep monitoring its replies after it goes live. Bottom line: every one of these steps involves judgment calls, and one bad call in any of them can ruin the whole experience, even with a great model. Happy to go deeper on any step. Which one should I break down next? 👇
A prompt engineering technique that asks ChatGPT to first identify assumptions, missing information, and common mistakes before answering, to get more thoughtful and tailored responses.
The article identifies a common failure in AI agent handoffs to humans on WhatsApp, where the bot says it will transfer but no human responds, breaking trust. It outlines a solution with mode tracking, history injection, and real task creation.
OpenAI Academy guide on prompting fundamentals that teaches users how to write clear, effective prompts to get better responses from ChatGPT through techniques like being specific, adding context, specifying output format, and breaking down complex tasks.
This article discusses the persistent difficulty AI chatbots have with correctly handling time-related queries, exploring the underlying reasons and user frustrations.
A developer details how their AI agent's silence was caused by safety guards failing closed, timeouts, and nested JSON issues, emphasizing that silent failures are worse than wrong answers in customer-facing chatbots.