traces

Advocates for treating agent failures as evaluation benchmarks rather than just trace logs, emphasizing the need for systematic testing of AI agent behaviors.

Own the agent learning loop and rent the model; private evals, traces, and memory compound over time.

At the aiDotEngineer World's Fair, Vtrivedy10 discussed how data mining from traces is a high-leverage practice for understanding AI agents, curating data at scale, and running improvement loops.

This tweet shares best practices for agent observability, covering metrics, logs, and traces to debug and optimize production AI agents.

A developer builds a debugging tool for AI agents that compares replays against reference runs to identify where behavior first drifted, expressing frustration with manual trace debugging.

The tweet recommends an article on agent architecture in production, highlighting the use of Traces to diagnose issues and implement an iterative improvement loop.

Yohei Nakajima highlights that building an agent on activegraph automatically produces first-class traces, unlike bolted-on solutions, demonstrated with a coding agent experiment.

A tweet by Martin Casado highlighting a solution to the difficult problem of exposing traces at scale to AI agents, balancing cost and AI leverage.

Adam Łucek discusses how LangChain uses trace data to build evaluations for production agents.

Respan introduces an AI observability platform that automatically catches issues in traces, aiming to replace manual debugging for agent-based workflows.

Engine is a new tool that connects agent observability traces to automated fixes and evaluations, closing the agent improvement loop for engineering teams.

LangSmith Engine is an agent that sits on top of traces, automatically identifies issues, and suggests action items like code changes or evaluators to add.