Tag
A researcher shares a daily automation that curates AI news from high-signal X accounts into an HTML artifact using X MCP tools and research agents.
Perplexity releases WANDR, an open benchmark and evaluation harness for research agents, consisting of 500 realistic data-collection tasks that require both wide discovery and deep verification. Initial results show even the strongest systems achieve low scores, highlighting that wide-and-deep research remains a challenging open problem.
IdeaTrail is a dataset of multi-turn process trajectories for scientific ideation, synthesizing research processes from evidence gathering to proposal construction using a Generator–Advisor loop to ensure grounding.
This position paper reviews the current state of LLM-driven formal mathematics, identifies key limitations in applying these systems to open-ended research mathematics, and proposes a strategic roadmap for developing AI agents capable of advancing mathematical frontiers.
A paper from Yale University built a large-scale evaluation framework to compare the distribution gap between LLMs and human researchers in generating research ideas. It found that LLM ideas are highly concentrated in bridge and synthesis types, while human ideas are more broadly distributed. This reveals differences in 'research taste' and poses a challenge to the diversity of Research Agents.
This paper identifies a failure mode in long-horizon research agents where optimizing an aggregate metric can select candidates that improve the headline number but break critical subgroups (inversion). It proposes a search-discipline protocol with an external control loop that audits candidates based on disaggregated behavior rather than the score.
Science Superpowers is an open-source computational-science methodology for AI research agents, enforcing pre-registration and reproducible workflows to prevent p-hacking and HARKing.
ScientistOne introduces Chain-of-Evidence, a verifiability framework for autonomous research agents that ensures every claim is traceable to evidence, achieving zero hallucinated references, perfect score verification, and the highest method-code alignment across 75 papers while matching or exceeding human expert performance on frontier research tasks.
The Onyx open-source deep research system achieves top ranking by stripping search access from its orchestrator agent, forcing it to decompose queries into focused research threads. Its three-phase pipeline and two-level architecture prevent information distortion and premature answering, outperforming proprietary solutions from OpenAI, Anthropic, and Google.
NineLayer, an MCP-based search engine for coding and research agents, has improved latency from 40s to 1.5s and is seeking user input on which platform integrations to prioritize.
This paper introduces REFLECT, a meta-evaluation benchmark for assessing the reliability of LLM judges in evaluating deep research agents. Experiments show current LLM judges remain unreliable, with overall accuracies below 55% across reasoning, tool-use, and report-quality failures.