@ms_aifrontiers: SentinelBench tests agents in time-evolving web environments where success requires waiting. How you wait matters: on 4…
Summary
SentinelBench is a new benchmark for testing AI agents in time-evolving web environments. It finds that agents using a specialized change-detection tool outperform those using sleep-and-poll loops, reducing cost by 9.7x.
Similar Articles
@ms_aifrontiers: A lot of agent benchmarks assume the world changes only when the agent acts. Many real tasks are different: tickets go …
Discusses a limitation of current agent benchmarks that assume the world changes only when the agent acts, whereas many real-world tasks require the agent to wait for external events before acting.
@ms_aifrontiers: Running every benchmark on every checkpoint is slow and expensive. New work from the MS AI Frontiers team asks: do you …
Microsoft AI Frontiers introduces BenchPress, a method to predict benchmark scores without running the actual benchmarks, saving time and computation.
HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents
This paper introduces HealthAgentBench, a suite of 54 realistic healthcare tasks for evaluating frontier AI agents. It finds that even the best agent (Codex GPT-5.5) achieves only ~42% success, highlighting substantial room for improvement.
DSAgentBench: Can Agents Automate End-to-End Data-Science Workflows in Real Computer Environments?
This paper introduces DSAgentBench, the first benchmark for evaluating autonomous agents on complete, multi-tool data-science workflows in real computer environments. Results show that even the strongest agent (Claude-4.6-Sonnet) achieves only 56.70% task success, while open-source agents remain below 1%, revealing a substantial capability gap.
@OkhayIea: Everyone's racing to build "AI scientists." So we asked a blunt question: Can today's best coding agents beat the publi…
Introduces NatureBench, a cross-disciplinary benchmark of 90 tasks from Nature papers to test AI coding agents, finding the best agent (Claude Opus 4.7) surpasses SOTA on only 17.8% of tasks and often succeeds by reducing science to supervised ML rather than genuine discovery.