How do you balance free-form AI writing with strict factual traceability?

I’m working on a long-form LLM generation system where outputs must be well-written, factually accurate, and traceable to source documents. The challenge is balancing two things: Accuracy: Every financial figure must be correct and traceable to its original source. A wrong number can sink an application. Writing quality: The model must produce persuasive narratives, not just repeat extracted facts. Reasonable inferences should be allowed without inventing information. We’ve tried two approaches. Attempt 1: Strict templates Every sentence had to be constructed from predefined prose and fact references. Figures were safe, but the writing was mechanical, repetitive and unconvincing. The model couldn’t build proper arguments. Attempt 2: Free writing with structured citations (current) Python extracts facts and fills tables. The LLM writes narrative sections freely, but must return JSON containing the text, sentence-level footnotes, and declarations for every numerical figure. A Python validator checks figures against their declared sources. Separate LLM agents review the narrative, repair flagged sentences and evaluate the final application. This improved writing flexibility, but introduced new problems. Where we’re stuck: Too much bookkeeping: In one 295-sentence test, 285 sentences had footnotes, including 98 with no numerical figures. Internal metadata accounts for 52% of stored characters, more than the actual narrative. Overly restrictive verification: Our editor rejects reasonable inferences because they aren’t explicitly stated in the documents. For example, a company exported in 2019 and has documented exports in 2024 and 2025. The sentence “The company has kept selling abroad since 2019” gets rejected because uninterrupted activity isn’t explicitly documented. Internal notes leak: The model sometimes copies its missing-information list into the actual application. Slow, noisy evaluation: Full runs take 25–45 minutes, and our LLM jury gives inconsistent scores to identical text. What we’re considering: Perhaps writing and verification should be separate processes: - Let the writer produce plain text without citation metadata. - Extract atomic factual claims afterwards. - Link each claim to its original evidence. - Verify numerical claims deterministically. - Distinguish direct facts, derived facts, reasonable inferences and unsupported claims. - Maintain an internal evidence ledger independently of the generated text. Questions for people who’ve built similar systems: Do you generate citations during writing or attribute claims afterwards? What works better in production? How do you guarantee numerical accuracy without forcing the writer to generate source references? How do you distinguish reasonable inferences from unsupported claims without making the writing overly conservative? Does strict JSON output per sentence hurt writing quality or latency? Have you moved to plain text followed by a separate structuring pass? How do you evaluate long-form generation reliably when LLM judges are inconsistent? I’m not looking for prompt-engineering tricks. I’m questioning whether our architecture is fundamentally wrong. I’d appreciate real production experience, relevant research papers, open-source projects or examples of systems that have solved similar problems. What approaches have you tried, including ones that failed?