I open-sourced a full agent observability stack: Record -> Inspect -> Diff -> Act (all MIT)

Reddit r/AI_Agents Tools

Summary

The article introduces an open-sourced agent observability stack with four MIT-licensed repositories that focus on extracting and inspecting model beliefs to enhance AI agent debugging and performance.

Your agent tells you why it failed, in plain language, every single time. Then it gets ignored - because "observability" means reading traces, and traces only show what it DID, not what it BELIEVED while doing it. I went the other way: read beliefs straight from the model's own streamed output, and keep everything self-hosted and open. The stack (four MIT repos): Axion - middleware that tees the model's streamed output and extracts beliefs (assumption / intention / planning) as structured events. Zero added latency, no code changes to your agent. Real test output: "I believe the user wants X" -> [intention] confidence 0.80. Includes PII redaction and a webhook channel (axion.belief_batch.v1) that feeds belief metadata into Langfuse/Arize/Braintrust spans. VisReplay - records full sessions (thoughts, tool calls, errors) into versioned files you replay frame by frame. VisCompile - deterministic behavioral diffs between agent runs. Byte-exact. Gate your CI on regressions (exit code 2 on regression, works in pipelines). VisBoard - shared agent/human workspace: boards, versioned notes with ETag semantics, scoped agent token workflows, live SSE events, integration sync-links. I verified each one live this week before shipping anything: SSE belief streams with redaction, byte-deterministic snapshots, full board CRUD + agent auth semantics against a real Postgres. Honest gaps: belief extraction is pattern-based (regex + clause rules), deterministic and free by design - the patterns are OSS so you extend them. Self-hosted only, no hosted tier yet. VisBoard automation is webhook/notify actions only.
Original Article

Similar Articles