our agent said yes to something we do not sell, and the logs could not tell me why

one of the shops we support does photo prints. someone asked if they also print diplomas. they do not. the assistant gave a vague answer, the customer asked again more directly, and the second time it said yes. they sent the file to print. nobody lost money, we sorted it out. what stayed with me was the debugging afterwards. i wanted to know whether that yes was grounded in anything at all, and i could not tell. tool calls we log with the message id, so those i can check per turn. the knowledge base lookup is not stored anywhere. it runs inline at generate time, the text gets pasted into the prompt, and then it is gone. so for the one turn that actually mattered i had the answer and no record of what it was based on. the useful bit came out of arguing about it with a few people this week. a sycophantic yes and a correct yes look identical in the text. they differ in whether anything entered the context between the two turns. position moved and no retrieval landed in between is the bad case. position moved and a lookup did land is just the system working. that turns a judgement call into a log check, which is much cheaper, but only if retrievals are attributable to the turn that triggered them. ours were not, and i suspect that is common. for anyone running agents in production: can you answer "what was this specific answer grounded in" without guessing?