Tag
JudgmentLabs opened general access for its platform targeting long-running agent systems, aiming to mine rich agent trace data for improving how agents are deployed on future tasks.
Judgment Labs opens for general access, aiming to help developers curate test datasets from production agent traces. The tool surfaces hidden signals in agent traces, such as what users ask for and where agents stumble.
An MIT-licensed CLI called chewme-audit audits agent trace exports to detect wasted tokens, finding 74% waste across 73 public benchmark trajectories, and offers per-provider pricing plus copy-paste fixes that reduced token waste by up to 77% in live runs.
Hugging Face hub now provides receipts for Agent Traces, displaying token usage, cache hit rate, and cost for each run.
The article discusses how successful retries in AI agent traces can complicate analysis due to uncertainty in identifying similar calls, and introduces Traser, a tool to highlight meaningful differences and reduce uncertainty for engineers.
LangSmith has rebuilt its filtering experience for agent traces, making it faster and easier to build queries, find runs, and understand matches.
Baseten Loops is a tool that helps teams train models for LangSmith Engine by fine-tuning open-weight models on agent traces and deploying checkpoints to production.
A paper investigates the contribution of agent harnesses versus models to agent behavior, collapsing traces into a compact finite-state machine validated across twelve datasets.
Agnost AI launched its first model, agnost-*******-0.1, trained on production traces from a customer, achieving a 22.9% increase in task success and a 90.2% reduction in some metric.
A hands-on review of Activeloop's Hivemind, which mined 19 reusable skills from agent traces, demonstrated selective memory by skipping already-documented workflows, improved cross-session recall on complex tasks, but showed recall issues and overhead on short tasks.
Arize Phoenix announces customizable visualizations for agent traces, enabling real-time tracking of cache hits, online eval degradations, and tool call errors in production.
World Model Optimizer is an open-source CLI tool that optimizes AI models from agent traces and serves them with a router to maintain frontier model quality at reduced cost.
LangChain released an Eval Engineering Skill that automatically generates executable Harbor evaluations by mapping agent repositories and production traces, with an iterative user interview process to refine evals.
Oodle launches Agent Observability on Hacker News, offering agent traces at $10 per million spans to help AI-native teams detect silent failures and improve reliability.
The tweet discusses the concept of self-learning as a competitive moat, highlighting browser activity and agent traces as key data sources, and introduces AG-UI, an open standard for capturing user-agent interactions to improve products.
The article discusses a method for building self-learning agents by combining agent traces with in-browser user activity, using the AG-UI protocol to capture and apply learnings, enabling products to improve through usage and create a business moat.
Arize Phoenix announces addition of trace search functionality to handle increasingly long and complex agent traces, allowing users to search across traces and view call stacks for spans.
A paper proposing a mechanism for enterprise agents to improve by safely converting messy daily work into learning data, using a data proxy and control layer, with AREAL2.0 demonstrating online RL from real interaction traces.
Morph Reflexes offers fast, cheap semantic signals from agent traces using multi-head classifiers on a shared backbone, enabling scalable monitoring of production agents.
Skill-DisCo is a framework that distills reusable procedural skills from successful agent traces and compiles them into callable, executable programs. Experiments on ALFWorld and WebArena show improved success rates and reduced agent turns.