What's the best way to get an agent to turn meeting notes into action items reliably?

I've been trying to solve what sounds like a simple problem and keeps not being simple. I want an agent that takes raw meeting notes, a transcript or a messy paste, and turns them into a clean list of action items with an owner and a due date where one was mentioned. The naive version works maybe 70% of the time, and the failures are annoying in specific ways. It invents owners that were never named. It turns "we should probably look at this someday" into a hard task. It misses the real action buried in the middle of a tangent because someone phrased it as a throwaway comment. And it drops due dates when the date was relative, like "before the next sync." Things that helped so far. I stopped asking for action items in one shot. First pass just extracts every sentence that implies someone will do something, no formatting. Second pass classifies each one as a real commitment versus a maybe, and only the real ones become tasks. Splitting extraction from judgment cut the hallucinated owners a lot, because the model isn't trying to be clever and structured at the same time. I also started forcing it to quote the exact line from the notes that each action came from. If it can't cite the source line, it doesn't get to create the task. That alone killed most of the invented ones. Still not where I want it. The owner and due-date inference is the weak spot. Anyone got a setup for this that survives real messy notes? Curious whether people are doing this with a single well-scoped prompt or a small chain of steps.