Tag
This paper introduces a deterministic testbed for evaluating LLM judges on agent trajectories, showing that outcome-only judges miss silent faults while step-based judges achieve higher recall with better calibration.
This paper introduces OAT, a lightweight failure attribution tool for LLM-based agentic systems that trains only on successful trajectories and uses neural controlled differential equations to detect error steps, outperforming expensive baselines by orders of magnitude in speed and accuracy.
IdeaTrail is a dataset of multi-turn process trajectories for scientific ideation, synthesizing research processes from evidence gathering to proposal construction using a Generator–Advisor loop to ensure grounding.
This paper introduces the Simple Strands Agent (SSA), a minimal harness designed to reduce the intent-execution gap between AI models and their agentic behavior, and analyzes 138k trajectories across various model families to reveal fine-grained behavioral differences.
BraveGuard is a self-evolving defense framework that trains guard models using open-world threat signals and realistic agent trajectories to improve safety detection in computer-use agents, achieving significant accuracy gains on the AgentHazard benchmark.
TraceGraph is a graph-based framework that constructs shared decision landscapes from multi-model agent trajectories, enabling diagnosis of failure regions and improvement via trap-aware recovery pipelines.
This paper introduces a claim-centric auditing framework for identifying error spans in deep-research agent trajectories, along with a new benchmark TELBench, improving process-level reliability assessment.
Agent Context Compilation (ACC) enhances long-context reasoning in LLMs by converting multi-turn agent trajectories into structured QA pairs, enabling direct supervision of distant context integration without additional annotation.