Tag
Agent Sociodrama 是一个用于建模和验证 LLM 智能体团队中动态社会关系的框架，并提出 Trajectory-Guided Persona Recasting (TPR)，通过诊断交互轨迹中的群体失败来重塑智能体人设，在四个协作基准上最高提升 83.3% 的准确率，并在分布式信息合作与混合动机谈判任务上超越 SOTOPIA 和 AgentVerse。
该论文提出以“认知谦逊”（Epistemic Humility）评估 LLM 智能体在知识冲突下的表现，通过 Identify、Solve、Escalate 三个轨迹级维度进行衡量，发现高任务准确率并不意味着更高的认知谦逊，且模型级干预往往以牺牲准确率为代价。
Omar Santos presents a guide on using Jev, TypeSafe AI's decision model, as an LLM judge for agent evaluations, showing how reading full trajectories (tool calls, results, errors) rather than just final replies increases reliability and consistency for production-grade evaluation and monitoring systems.
TraceML is a tool for trajectory-level analysis of ML research agents, comparing Codex agent runs with Kaggle Grandmaster notebooks and finding that human experts explore broader strategy sets while agents collapse into narrow loops. Distilling human research skills into a ~1,000-token planning prompt lifted Codex scores on five of seven competitions; the work will be presented at NeurIPS 2026.
The paper proposes PAIR (Phase-Aligned Intra-question Reasoning), which treats LLM reasoning paths as phase-structured trajectories and evaluates path quality only within the same question, showing that standard correctness probes partly rely on question-level variation and that phase-wise steering yields causal evidence of path-quality directions.
AutoTraceGT automates grounded theory analysis on agent trajectories to build behavioral taxonomies, effectively recovering human failure-mode annotations and improving downstream prediction tasks.
The paper introduces ontological trust and RGE, an online monitor that decomposes trust along Role, Goal, and Evidence to detect task drift in long-horizon agents. RGE outperforms baselines on cross-domain benchmarks with high Drift F1 scores while maintaining benign coverage.
This paper conducts a trajectory-level investigation of PGD attacks on CNNs trained on Fashion-MNIST, showing that while a robustness hierarchy exists, trajectory metrics like loss evolution and gradient alignment do not independently measure robustness, with steps-to-failure providing clearer separation.
Introduces the Behavioral Consistency Metric (BCM) to quantify behavioral consistency across tasks in language model agents, demonstrating it is distinct from success rate and can be measured via execution traces.
The paper proposes RUPA, a framework that models LLM agent execution as a dependency graph to propagate uncertainty, improving failure detection and confidence estimation in long trajectories.
New research from Microsoft Research measures cross-lingual policy retention of tool-using agents by comparing action traces rather than final answers. After removing five confounds, four frontier models retain 71–73% of their action policy across languages, with code and 2.38M rollouts released.
This paper proposes a three-stream detector that combines residual-stream motion with coarse regions and fine directions to better identify reasoning errors in LLMs, improving selection accuracy by up to 12% over state-of-the-art displacement-only methods.
Introduces STRACE, a framework that performs structural trajectory analysis and causal extraction to construct high signal-to-noise optimization contexts for improving long-horizon agents, outperforming baselines on a formal verification task.
Jugmax is a tool that evaluates AI agents by analyzing their full execution trajectory rather than just final outputs, identifying inefficiencies, errors, and wasted tokens. The founders offer free evaluation for two production agents to validate their product.
This paper introduces AgentAtlas, a framework that goes beyond outcome-only leaderboards for LLM agents by proposing a six-state control-decision taxonomy and a nine-category trajectory-failure taxonomy to evaluate agent behavior more comprehensively.
This paper investigates whether reasoning-trained language models simply allocate more compute (longer chains of thought) or follow qualitatively different internal trajectories by analyzing hidden-state trajectory geometry across code, math, and SAT domains. After correcting for generation length, they find that reasoning-trained models exhibit distinct trajectory geometry—most clearly in code—indicating reasoning training changes how computation unfolds, not just how much is used.
This paper investigates the latent structure of multimodal embeddings from a masked autoencoder for pediatric sleep analysis. It shows that augmenting embeddings with geometric, topological, and clinical features improves prediction and calibration for sleep-related events.
AgentLens is a framework for process-level assessment of software engineering agent trajectories, revealing that over 10% of passing trajectories exhibit a 'Lucky Pass' behavior. It introduces AgentLens-Bench, a dataset annotated with quality scores, and shows that ranking by quality score can shift model rankings significantly.
Large-scale study of 15 LLMs across 8 tasks reveals that optimization success hinges on maintaining localized search trajectories rather than initial problem-solving ability or solution novelty.