Giving an agent a typed tool per website beat giving it a generic scraper — writeup

Spent a while trying to get an agent to reliably pull structured data off a set of sites and the generic-scraper approach kept failing in the same three ways: The agent gets a page of markdown and re-derives the structure every single call. Slow, expensive, and the output shape wobbles between runs. Sites change and the agent silently starts returning garbage, confidently. Debugging is miserable, because the failure is inside a model call rather than in code. What worked better was inverting it: derive the extraction schema once with an LLM, cache it, and expose each site to the agent as its own typed tool. The agent's call is now deterministic and cheap. No model in the hot path and when a site changes, the schema re-derives itself underneath rather than the agent hallucinating around the breakage. The other piece that mattered more than expected: grounding every extracted field against the source HTML and explicitly flagging values that don't appear in it. An agent acting on silently-wrong data is worse than one that errors, and "the model made this up" is otherwise invisible. Happy to go into the drift-detection heuristics, they're the least solid part. (I built the open-source implementation of this — dropping the link in a comment per rule 3.)