Tag
An open-source Docker Compose setup integrating multiple open-source tools (LiteLLM, LLM Guard, OpenBao, Langfuse, etc.) to provide a governed, compliant, and auditable AI platform for organizations, with a user-friendly interface for building governed workflows.
The author discusses building six AI agent harnesses and emphasizes the need for a dedicated database to track agent execution, state, and learnings, beyond just observability tools.
Plano is an open-source proxy that sits between AI agents and LLM providers to cut costs through intelligent routing, guardrail filtering, and cost-aware selection, all configured via a single YAML file without modifying agent code.
Phoenix now includes customizable experiment charts and baselining, allowing users to compare models along performance, latency, tokens, and cost dimensions, and set baselines for preferred models.
Phoenix has a built-in Pixie assistant that helps users quickly filter out silent failure traces where agent spans have errors but model responses are normal, greatly improving trace reading efficiency.
The article compares popular developer tools for agent reliability across four layers: tracing/evals, runtime guardrails, and gateway. It finds that no single open-source tool covers all layers, and most developers use a combination.
The AI agent ecosystem has many frameworks for building agents, but lacks operational layers for deployment and governance, prompting discussion about the need for agent control planes.
LangSmith highlights Finch Legal, a startup using AI agents for pre-litigation in personal injury law, achieving 10x growth and utilizing LangSmith for production observability.
Discusses that production agent evaluations should include failure replay and resume capabilities, not just happy-path task success, emphasizing the need for observability that enables recovery.
This paper proposes a controllability–observability framework for compressing deep neural networks by reducing hidden-state redundancy, demonstrating significant compression with minimal accuracy loss on MNIST and CIFAR-10.
Schneider Electric uses LangChain's LangSmith to run over 60 production AI agents across 100+ countries, serving 160,000 employees with their AI Assistant, demonstrating enterprise-scale LLMOps.
A detailed comparison of LLM evaluation and observability platforms based on months of research for a 250-person company rollout, covering full-stack platforms, observability tools, and open-source frameworks.
Arize Phoenix announces addition of trace search functionality to handle increasingly long and complex agent traces, allowing users to search across traces and view call stacks for spans.
A statistic reveals that while 89% of agent teams in production have observability, only 52% run evaluations, raising questions about how prompt changes are gated.
A developer argues that most production AI agents lack essential observability like session traces and cost tracking, comparing it to deploying a web app without monitoring. The article questions whether agent observability is an unsolved problem.
Eve is a framework for building durable, production-ready AI agents with built-in observability, introspection, and self-improvement capabilities, designed to work seamlessly with Next.js and Vercel deployments.
TraceSage is a new MIT-licensed, local-first observability tool for LangChain and LangGraph that runs entirely on your machine, providing live topology graphs, step-by-step replay, token tracking, and OpenTelemetry export without sending data to the cloud.
GreptimeDB v1.1.2 is a patch release that fixes scheduled Flows now() binding for deterministic EVAL INTERVAL windows, along with bug fixes for Kafka SASL password redaction, GC index file listing, parquet metadata cache size, Prometheus label discovery scan, and PromQL time binary aggregation. It is recommended for users to upgrade.
The article argues that ClickHouse has become the dominant database for observability due to its performance handling high-volume, time-ordered logs, solving long-standing log management challenges.
The article argues that diagnosis—explaining why an agent failed in operational terms and what is safe to do next—is a missing first-class skill in production agent stacks, more critical than making agents sound smart.