Vidai Community is now available: one Rust binary for cost attribution, guardrails and multi-provider routing on every LLM call

Reddit r/LocalLLaMA Tools

Summary

Vidai Community is a free, self-hostable Rust binary that provides cost attribution, guardrails, and multi-provider routing for LLM calls with a one-line integration and minimal overhead.

https://preview.redd.it/dhq90p7jp84h1.png?width=1200&format=png&auto=webp&s=107cdb803051d0bb2e012297fcb2ef7a57972876 Cost control on LLM traffic isn't an unsolved problem. Most teams I see have already wired up some version of it — a library for SDK translation, a custom OpenTelemetry exporter, Python middleware for spend caps, regex guards on prompts, a JSON of provider prices someone refreshes on Tuesdays. It works. Nobody chose that stack though. It grew. **Vidai Community is now available.** One Rust binary doing those seven jobs in one pass, inside your own infrastructure. * **One-line integration.** Change the `base_url` on your existing OpenAI / Anthropic / Google GenAI SDK. Keep your code, your tool-call format, your streaming semantics. Bidirectional native translation handles the rest — call with the Anthropic SDK, route to OpenAI, response comes back in Anthropic shape. * **In-path cost attribution at return time.** Per user, key, team, app and model. Hard per-team budgets that stop the next call when a cap breaks, not alerts about the call that already ran. * **Free public rate-card service.** Pricing data for every major provider, refreshed on a standard cadence. Stop maintaining your own JSON. * **25 MB. 1.95 ms median overhead. 21,803 RPS verified on a single node.** Free, no expiry, self-serve at [vidai.uk/community](https://vidai.uk/community) · [docs](https://docs.vidai.uk/) If you spin it up and find the rough edges, hate one of the defaults, or wish a flag did something different: [open an issue on the quickstart repo](https://github.com/vidaiUK/vidai-quickstart/issues) or reply here. The product gets sharper every time someone runs it on hardware that is not ours, and this is the kind of feedback we cannot get from inside our own stack.
Original Article

Similar Articles

We built a source-available LLM reliability library (free for research / personal / internal eval) that can cut inference cost by half at matched quality, and you adopt it by changing one import [P] [R]

Reddit r/MachineLearning

AgentCodec is a source-available library unifying 28 LLM reliability techniques (retries, ensembling, generator/critic refinement, etc.) under a single OpenAI-compatible API, with adaptive routers that can reduce inference costs by ~56% at matched quality. It adopts a communication-theory framing and supports drop-in replacement for OpenAI, Anthropic, and Ollama clients.