Agents hallucinated because they were committed to the prompt: how RLHF broke Winograd and Flores's 1986 state machine (and the 1-line fix)

Reddit r/AI_Agents News

Summary

作者提出 LLM 幻觉源于 RLHF 强制模型在 Token 1 无条件承诺(Promise),导致生成过程中无法反悔只能编造 API;建议用 Winograd/Flores 对话行动状态机的『Zero-Shot Flores Handshake』显式授权模型先反问或拒绝,再生成代码。

We have all watched an LLM reply "Certainly! Here is how to do Y with PackageX" at Token 1, open a code block, and then at Line 15 calmly invent a performY method that has never existed in the history of the universe. Most developers assume this is a memory defect ("it forgot the docs"). Yet if you open a fresh chat and ask the same model "Does PackageX have a native method for Y?", it usually tells you no and explains the manual workaround. Back in 1999 I wrote a Perl tool called spew that reversed a weighted BNF parser—walking down a grammar tree from START to generate valid output. An autoregressive LLM writing code is a multi-billion-parameter BNF un-parser. In Terry Winograd and Fernando Flores's 1986 Conversation for Action state machine, a Performer responding to a Customer Request has three legal moves after checking reality: Promise, Counter-Offer, or Decline. Commercial RLHF preference tuning sawed off Counter-Offer and Decline—forcing an unconditional Promise at Token 1. Because token generation is append-only, once the model emits "Certainly!" and opens a code fence at Token 10, it cannot backspace when it reaches Line 15 and realizes PackageX has no API for Y. To close the enclosing syntax tree of the promised code block, its only mathematical escape hatch is to hallucinate the identifier that would have existed if its premature Promise were true. Because Winograd, Flores, and John Searle's speech-act formalisms are already deeply encoded in pre-training weights, starting a cold prompt with what I call the Zero-Shot Flores Handshake—requiring the model to emit Witnessed Assertions (?Assert) first and explicitly authorizing a ?Counter-Offer or ?Decline alongside a testable By-When—stops the "Certainly!" reflex at Token 1 and makes the model propose a real workaround before opening a code fence. Has anyone else experimented with speech-act protocols or formal negotiation states to override the RLHF yes-man prior before code generation starts?
Original Article

Similar Articles

My agents kept lying nonstop so I made them show their work

Reddit r/AI_Agents

The author describes building a 'gate' that logs tool calls and verifies that claims in an AI agent's answer correspond to actual log entries, forcing the agent to show its work and reducing hallucinated responses.