The article discusses the limitations of research agents in tracking rejected sources and emphasizes the need for transparency in exclusion reasons to aid review and validation processes.
You give a research agent a question, some PDFs and a few CSVs, then expect a structured answer with sources you can verify. You can check what it cited and trace those claims back to the material it used. What’s harder to inspect is what it rejected, and why. Maybe a paper was excluded because the methodology looked weak. Maybe the agent couldn’t map the columns in a dataset. Maybe two sources conflicted and it kept one without making that choice visible. Those are different reasons for leaving something out. A reviewer needs to be able to tell them apart. Take a hypothetical agent reviewing clinical studies on treatment efficacy. The final analysis looks reasonable and the citations check out. But two studies with null results are missing. The agent didn’t reject them because the studies were flawed. Its extraction step failed on an unfamiliar table format, returned an empty result, and the pipeline interpreted that as “no relevant data.” From the outside, a parsing failure now looks like deliberate curation. It reminds me of the limits of GKE’s GPU duty_cycle. That metric tells you how much of the sampling period the GPU spent actively processing. You still need other evidence to know whether that computation advanced the task. A research agent’s citation list has a similar limit: it shows which sources made it into the answer, but leaves you to reconstruct what happened to the rest. An agent that cites 12 papers after rejecting 30 can leave the same final reference list as one that cites 12 after rejecting three. Without the exclusions, a reviewer has little basis for judging how that list was assembled. Disclosure: I’m working with Apodex on this post. Apodex’s Task Board makes task breakdown and subagent activity visible. Statement Review separately checks selected important claims against the sources, data and calculations from the run. That helps with inspecting the work and checking whether the evidence supports the final claims. To investigate omissions, I’d also want a record of rejected sources and their exclusion reasons: incompatible metrics, missing columns, extraction failures, or unresolved conflicts between sources. Those records would need checking too. An agent could give a plausible exclusion reason that doesn’t hold up when someone opens the paper. But at least a reviewer would have a specific decision to challenge. If you had a limited evaluation budget, would you spend it validating cited claims, or sampling rejected sources and checking why they were left out? How are people keeping extraction failures separate from deliberate exclusions?
A discussion of how web-research agents should decide when to stop searching, breaking the stopping decision into coverage, progress, and budget checks, and arguing for returning unresolved questions rather than false confidence. The author asks practitioners what stopping rules they actually use for research agents.
The paper introduces ReAgent, an automated auditing framework for assessing the consistency between agent-written research documents and their associated repositories, using static and dynamic analysis to identify inconsistencies in methodologies and experimental results.
The article discusses how while coding agents can effectively generate code, they introduce a new bottleneck in reviewing and trusting the changes, questioning whether agents reduce or shift the review workload.
This paper introduces decision-checkpoint auditing, a trajectory-level measurement method that classifies scientific-search agent failures into target exposure, inspection, and answer correctness stages, finding that keyword search and read-first policies achieve identical 24.6% accuracy despite differing failure distributions on AutoResearchBench Deep.
The AQuA v2 preprint introduces a memory system for research agents that uses persistent evidence from accepted and rejected experiments to guide future proposals, emphasizing the importance of evidence lifecycle management.