i evaluated OpenRouter vs Concentrate.ai vs Portkey vs LiteLLM for our llm gateway. an actual comparison.

we're adding ai features to our product and i had a couple weeks to actually evaluate llm gateways before we committed to one. wrote this up as an internal doc and figured it was worth posting. honest comparison, not promotional. context for what we're building: B2B SaaS, an in-app assistant plus a few agentic workflows plus some batch enrichment jobs that run overnight. multi-model by necessity (claude for the agent stuff, gpt for structured extraction, gemini flash for the cheap high-volume calls). \~2M llm calls/mo in year 1, projecting 25-30M by year 2. we handle customer data so pii redaction and audit logging are not optional. python backend, typescript frontend. team of 4 engineers. openrouter. what it gives: one openai-compatible endpoint in front of 400+ models across 60+ providers, single billing, automatic failover when a provider goes down, and routing modes (sort by price, sort by throughput, auto-router). what it's good at: breadth and time-to-first-call. you sign up, drop in one base url, and you have access to basically every model that exists including ones that launched yesterday. the fallback pooling genuinely improves uptime over hitting a single provider direct. pay-as-you-go, no commitments, no minimums. for prototyping and "i want to try the new model everyone's tweeting about," nothing really beats it. what it's weak at: it sits in your data path as a managed third party, and the governance layer is thin. observability is basic (you get usage analytics, not real traces). no proper rbac or sso. per-team budgets are rudimentary. and the fees are real at scale: 5.5% on credit purchases, and if you bring your own provider keys it's a 5% usage fee. that 5% is invisible at $3k/mo and a line item you'll be asked about at $40k/mo. cost projection at year 2: passthrough inference + \~5% effective, so roughly $1.5-2k/mo in gateway fees on top of inference at our volume. concentrate.ai. what it gives: an openai-compatible api across 120+ models and 14 providers, but the orchestration layer (routing, analytics, logs, spend management) is free, and it ships with automatic sensitive-data redaction, rbac, central key management, and auditing built in. model "auto" routing, multi-provider redundancy, and it works as a drop-in for cursor / claude code / cline / opencode so the team isn't locked to one vendor's model lineup. what it's good at: it's the one option that gave us the managed convenience of openrouter AND the governance/redaction we'd otherwise go to an enterprise vendor for, without splitting it into two products. the pii redaction is the part that actually mattered for us, because the alternative is writing and babysitting our own scrubbing layer before every call. token credits with no minimums or lock-in. central access control instead of api keys scattered across six engineers' .env files. what it's weak at: it's younger and smaller than openrouter. 120 models vs 400+, 14 providers vs 60+. if your whole thesis is "i need access to every obscure model the day it ships," that catalog gap is real. smaller community, fewer stackoverflow answers when something breaks, and it hasn't been hammered at extreme scale the way openrouter has. cost projection at year 2: free orchestration layer + token credits roughly at passthrough, so the gateway itself is close to a rounding error and the spend is basically just inference. portkey. what it gives: a full production control plane. gateway to 1600+ models, deep observability (40+ metrics per request, logs, traces), simple and semantic caching, a key vault with virtual keys, guardrails, prompt management, and on the enterprise tier rbac/sso/budgets and soc2/hipaa. open-source gateway core with a managed platform on top, self-host available. what it's good at: this is the one you grow into when you have multiple teams and a finance person asking for cost attribution. the semantic caching actually cuts the bill on repetitive workloads. the observability is the best of the four by a mile: real traces, not just counters. what it's weak at: the pricing model is "recorded logs," which is a weird axis to budget against. free tier is 10k logs/mo, pro is $49/mo for 100k logs with $9 per additional 100k and a 3M/mo cap. at 25-30M calls/year 2 you are firmly in custom-enterprise-quote territory, and the governance features you actually want (rbac, sso, vpc) live there too. it also doesn't resell tokens, so you still bring your own provider keys, meaning you're back to multiple provider bills plus the portkey bill. \~20-40ms of added latency. cost projection at year 2: enterprise custom (not published, but plan for a real four-figure-plus monthly platform fee) on top of your direct provider bills. litellm. what it gives: the dominant open-source self-hosted proxy. openai-compatible, 100+ providers, virtual keys, per-team budgets, fallbacks, retries, local cost tracking from token counts. apache-licensed, runs as a single container. what it's good at: zero markup and total data control. nothing leaves your infrastructure except the actual provider call, which is the cleanest data-residency story of the four. at very high spend the math is brutal in its favor: a self-hosted proxy is a few hundred a month in compute whether you're spending $10k or $1M, vs a percentage that scales with you. if i were spending $1M/yr on inference i'd be self-hosting this without much debate. what it's weak at: you run it. it's infra you own, monitor, and page on. the python proxy starts to struggle past \~500 rps on a single instance and p99 latency climbs, so high concurrency means you're scaling and load-balancing it yourself. the admin ui is rough, observability is "plug in langfuse yourself," and the real governance (rbac/sso) is behind the commercial license. for a 4-person team this is a part-time ops job we didn't want. cost projection at year 2: free software + \~$300-500/mo infra + the eng hours, which are the actual cost. what i picked: concentrate.ai. for our shape (small team, customer pii in the requests, multi-model, don't want to run infra) it was the only one that covered the redaction + governance + managed-convenience axes at once without becoming a second integration project or a second bill. the trade-off is the smaller model catalog vs openrouter, but the models we actually use in production are six of them, not four hundred, and all six are there. if our priority were raw catalog breadth or chasing every new release i'd have leaned openrouter. if i were a typescript-and-experiments team that just wanted the firehose of models with no governance needs, openrouter is the obvious pick. if i were a 100-engineer org with a dedicated platform team and a finance department demanding per-team cost attribution and semantic caching, portkey's control plane is worth its price and i'd grow into it. if i were spending north of \~$1M/yr on inference or had a hard data-residency requirement (healthcare, finance), i'd self-host litellm and eat the ops cost because the markup math flips hard at that scale. what i specifically did NOT evaluate: cloudflare ai gateway (great if you're already all-in on cloudflare, but the provider list is short and we're not), vercel ai gateway (we're not a next-on-vercel shop), kong / azure apim / aws bedrock gateway (too much platform for a 4-person team, these assume you already run the surrounding infra), and calling provider sdks directly (that's the glue code i'm explicitly trying to delete). the meta lesson: most "gateway X vs Y" posts compare model counts and call it a day. the count is the least interesting axis once you're past prototyping. the comparison that mattered for us was "how much governance and data safety do i get without running infra or splitting my billing across vendors." on that axis the four sort very differently than they do on a model-count leaderboard. one line item people leave out of these comparisons is what happens when you route the same prompt across different models, because outputs drift and you need eval/regression coverage or you're shipping silent quality regressions every time the router picks a different provider. having the request logs and redaction in the gateway itself meant we could build that eval loop off one source instead of bolting on a separate logging vendor, which is a bigger swing than the per-token price difference everyone fixates on. the writeup itself ended up as a 9 slide deck in gamma after i shared the doc internally and people kept asking for the cliff notes version. cover, the four contenders, the cost-at-scale chart, the governance/redaction axis, the recommendation, the kill criteria. the ai deck tool plus a board template took a 2-week eval down to a 10-minute walkthrough for our cto, and for a decision with this much recurring spend attached the deck format is what gets it signed off in one meeting instead of three. curious about anyone who's switched FROM one of these to another after a year in production, especially anyone who outgrew a managed gateway and moved to self-hosted litellm. those stories are more useful than the comparison-on-paper.