An AI document generator saved my client hours until it quietly invented a clause

Reddit r/AI_Agents News

Summary

An AI document generator automated contract creation for a small agency but introduced a hallucinated clause, leading to a solution where critical sections are fenced off from AI generation to prevent errors.

I build automations for small teams. One of the more common asks lately is an ai document generator. Feed it a few inputs, get back a finished contract, SOW, or onboarding doc. Sounds obvious. I built one earlier this year for a small agency that was rewriting the same statement of work over and over. The setup was simple. An intake form feeds the details into an LLM, it fills a template, spits out a formatted doc. In the demo it looked great. It wrote cleaner SOWs than the founder did, faster. Then about three weeks in, one of their clients flagged a payment term nobody had agreed to. The model had added a net-30 clause because most of the SOWs it had seen probably had one. It wasn't in the template. It wasn't in the intake. It just decided a professional SOW should have that line, so it wrote it in. Nobody caught it because the doc read perfectly. That's the part that scared me. A wrong number in a dashboard looks wrong. A fabricated clause in a contract reads exactly like a real one. The output being fluent is the whole problem. What I changed: the LLM no longer writes the legal or numeric parts at all. Those get slotted in from the intake form directly, no generation. The model only handles the wording of the descriptive sections, and even then I run a diff against the template so any sentence it added that isn't traceable to an input gets flagged before the doc goes out. A lot of people building doc generators are one confident hallucination away from this. If the document carries obligations, money, or dates, the model should assemble, not author. How do others handle it? Do you let the model write the whole doc, or fence off the parts that actually matter?
Original Article

Similar Articles