Tag
The paper introduces CRATE, a two-stage framework using step-level consequence reasoning to evaluate mobile agents, achieving high F1-scores on benchmarks like AndroidWorld and MobileRisk.
This paper proposes offline preference-based trajectory evaluation for agentic systems, which compares trajectories via temporal preferences rather than binary success metrics. It shows that this approach reduces ties from roughly 75% to 35%, improving discriminative power and data efficiency across diverse benchmarks.