For finance agents, strict pass is a better metric than an impressive partial completion

A finance agent can retrieve the right filing, calculate most of a model correctly and still fail the task because one broken formula invalidates the deliverable. The Ling-3.0-flash-Fin release describes Finance Agent v1.1 and v2 results using a Strict-Pass rule: a task passes only when every scoring criterion is satisfied. To reduce run-to-run noise, the reported v1.1 score averages 10 runs and v2 averages 20. That is a much more useful framing for long-horizon agents than averaging partial credit across steps. In a leveraged-buyout workflow, for example, operating expenses feed EBITDA, free cash flow, debt paydown and IRR. A plausible final IRR is worthless if the debt schedule or sensitivity table is disconnected from the underlying assumptions. For production evaluation, an agent scorecard could report: full-task pass rate across repeated runs; the first stage where state became invalid; whether the agent detected its own failure; artifact-level checks on formulas and file structure; the percentage of runs that required human repair; whether a safe handoff preserved the evidence and intermediate state. The published demos are not independent validation, and financial conclusions still require professional review. But strict pass makes the right point: an agent is only as reliable as its weakest required step.