A browser agent failure that is easy to miss: the page said no and the agent kept going

Something I ran into repeatedly while building a browser tool for agents, which I think generalises beyond my own case. When a web form rejects a submit, it usually does not add any new controls. It just prints a message near the fields. If your agent's action result only reports structural change, a refused submit and a successful one look identical. The agent reads success, moves to the next step, and now every remaining action runs against a screen that never advanced. The task fails three steps later, somewhere that looks unrelated to the real cause. Screenshot based agents have a harder version of the same problem, because the refusal is a few red pixels the model has to notice and interpret correctly. What fixed it for me was making the action result carry what the page said, not only what changed structurally, and then treating a refusal as a stop condition for the rest of the batch: 4. click "Save Delivery Details" page says: "Please fix the highlighted fields below.", "Full Name is required.", "Delivery Address is required." the page refused this step, so the remaining 2 steps were not attempted Two suggestions for anyone building in this space. First, sample visible text in your observation, not just the control tree, or you will miss every validation message. Second, make refusal a first class outcome, distinct from both success and error, because it needs a different recovery: the agent should fix the named fields, not retry the click or abandon the task. Worth adding that my own test suite did not catch this. The flows passed either way, because the tests re-read the page after each step and the error was visible there. Only the per-action result was blind, which is exactly what an agent reads when it batches steps. I build browser tooling for agents. Happy to go into detail in the comments.