Created deterministic prose claim verifier for coding agents on macOS

Reddit r/AI_Agents Tools

Summary

A deterministic macOS menu bar app that watches coding agent transcripts (Cursor, Claude Code, etc.), extracts claims using regex patterns, and cross-references them against evidence to flag inconsistencies—all running locally without LLM or API calls.

Hey everyone! I wanted something that just watches and tells me when the agent's words don't match what actually happened. So I built Snitch. It watches your agent's transcript files (Cursor, Claude Code, Codex, Pi, OpenCode), extracts claims from the prose using deterministic regex patterns, and cross-references them against evidence — tool calls, shell output, filesystem, git, and 3-turn session history. When a claim doesn't add up, it flags it with what it found. It normalizes tool names across agents under the hood (Claude's Bash → Shell, Codex's apply\_patch → StrReplace) so the verification pipeline works the same regardless of which agent you use. Everything runs locally — no LLM, no API calls, nothing leaves your machine. macOS menu bar app. Install is: brew tap fristovic/snitch && brew install snitch && snitch start v0.4.2, \~10K lines of Go, Apache 2.0. Consider this an alpha — almost a proof of concept for something bigger. I'm sharing it early to see if the approach resonates before going deeper. Fair warning: there will be false positives. I'm still tuning the patterns. If you try it and Snitch flags something that's actually fine, I'd love it if you opened an issue (or better yet, a PR) with the specific case you hit. The pattern registry is designed to be easy to add to — there's a guide in the repo and CI enforces that new patterns ship with examples. On the roadmap: a community labeling feature so users can mark verdicts correct/incorrect (training data for a local classifier to reduce noise), and a team dashboard for orgs running multiple agents. I'm hoping we can put our heads together as a community and build something genuinely useful here. All questions welcome. Tell me what breaks.
Original Article

Similar Articles

AgentPeek

Product Hunt

AgentPeek is a macOS utility that displays the status of AI coding agents like Claude Code and Codex in the Mac notch.

How to setup a local coding agent on macOS

Hacker News Top

A detailed tutorial on setting up a local coding agent on macOS using Gemma 4 with MTP draft model and llama.cpp, achieving ~24% speed improvement through speculative decoding.