Tag
A thread by @rauchg shares data from Vercel AI Gateway showing that on June 27, Anthropic, OpenAI, and Google hit an all-time high combined spend share of 97.09%, while open model spend has recently declined to an all-time low.
OmniRoute is a free, open-source AI gateway that unifies access to 268+ providers and 500+ models through a single local endpoint, featuring smart routing, token compression, and free tier aggregation to reduce costs and complexity for developers.
A tweet lists key projects to build in inference engineering for understanding production LLM systems, including inference servers, paged KV cache, speculative decoding, quantization libraries, and guardrails.
Vercel opens AI Gateway leaderboard data under CC BY 4.0, providing an API endpoint for data export and shareable chart images, allowing users to analyze production AI usage.
Vercel's AI Gateway Production Index for July 2026 reports that open-weight models now account for 29% of token volume, with DeepSeek leading among open-weight labs, while average token prices flattened as cheap open-weight volume offset rising frontier model costs.
An open-source Docker Compose setup integrating multiple open-source tools (LiteLLM, LLM Guard, OpenBao, Langfuse, etc.) to provide a governed, compliant, and auditable AI platform for organizations, with a user-friendly interface for building governed workflows.
Announcement of model release week featuring Meta Spark 1.1, Grok 4.5, and GLM 5.2, with AI Gateway now supporting Muse Spark 1.1 for agentic tasks.
The author describes the common enterprise problem where PII redaction before sending data to LLMs breaks the output, and they are building a solution that rehydrates responses without exposing raw data.
The tweet presents an animation of token spend from Vercel AI Gateway, illustrating shifts in usage among AI labs, Anthropic's dominance, and the growth of open weight AI.
Vercel's AI Gateway now supports routing rules that allow developers to dynamically rewrite model routes (e.g., from retired models like Claude Fable-5 to Claude Opus-5) without code changes, ensuring production workloads remain resilient.
An open-source, self-hosted AI gateway providing a single endpoint for 237 LLM providers with auto-fallback, token compression, and routing. It has gained significant traction with 9.8K GitHub stars and 280+ contributors in 4.5 months.
OmniRoute is a free and open-source AI gateway that unifies 236 AI provider APIs into a single OpenAI-compatible interface, supporting self-hosting, automatic fallback, intelligent routing, and token compression.
Grok Imagine Video has become the most popular video generation model on Vercel's AI Gateway, accounting for ~50% of all videos generated by developers.
LiteLLM has migrated from Python to Rust, achieving massive performance improvements: request overhead reduced by 150x to 0.05ms, throughput increased by 15x, memory usage reduced by 11x to 32MB.
A discussion about unexpected high AI API costs due to bad loops, unauthorized key usage, and lack of monitoring; seeking advice on detection and prevention.
Meta's Applied AI unit faces record-low morale and a multi-billion dollar cost crisis as employees artificially inflate AI token usage ('tokenmaxxing') in response to performance metrics tied to AI consumption, leading to internal rebellion and strict token budgets.
This tweet introduces various development features provided by Cloudflare, including object storage R2, backend API Workers, AI gateway AI Gateway, containers, cache KV, database D1, and PostgreSQL connection HyperDrive, emphasizing their low cost, rich features, and generous free tier.
AI Gateway's May 2026 data shows DeepSeek's token share surged to 17% with minimal spend, while Anthropic retained 65% of spend, indicating cost-conscious routing and growing overall usage.
A comprehensive guide explaining model routing as a technique to intelligently select the best AI model per request to optimize cost, quality, and latency, contrasting it with AI gateways and emphasizing its importance for agentic AI workloads.
Respan Gateway is an AI gateway with built-in observability and evaluation features for developers.