Cerebras OpenAI deal capacity has effectively killed the waitlist for everyone else [D]

Reddit r/MachineLearning News

Summary

Cerebras' deal with OpenAI to supply $20 billion worth of chips has pre-allocated near-term inference capacity, causing their API waitlist to become effectively infinite for other startups seeking fast inference.

I’m pretty annoyed. We’re a small AI startup building a real-time coding agent. Our p95 latency requirements are tight (and self imposed, but thats the product). We need sustained high-throughput inference with ~1-2k tokens/second. Been on the Cerebras waitlist for months trying to get API access. We’re not doing training so don’t need a warehouse of H100s. We need fast, high-throughput ASIC inference for a specific production workload. Cerebras’ just went public and they basically have no compute how is that possible? Well turns out OpenAI and Cerebras for OpenAI to buy like $20b worth of these chips. This has effectively pre-allocated the vast majority of Cerebras’ near-term inference capacity to a single customer. I mean, none of us can compete with that The result is that this deal situation has made their API waitlist functionally infinite for anyone who isn’t a hyperscaler. Legit making me pull my hair out.
Original Article

Similar Articles

OpenAI partners with Cerebras 

OpenAI Blog

OpenAI partners with Cerebras to integrate 750MW of ultra low-latency AI compute into its platform, aiming to accelerate inference and enable faster real-time AI responses across various workloads.

AMD and Cerebras Launch AI Inference Solution (10 minute read)

TLDR AI

AMD and Cerebras announced a joint AI inference solution combining AMD Helios rackscale solutions with Cerebras Wafer-Scale Engine, aiming for ultra-low latency and high throughput. The disaggregated inference workflow is expected to deliver up to 5x higher tokens per second per watt.