Tag
The article analyzes the rapid rise of fal, a Silicon Valley cloud inference platform for AI image and video generation: its annual revenue is approximately $800 million, and its valuation could reach $15–20 billion, compared with annual revenue of only about $10 million two years ago. Rather than selling GPUs, fal’s core business model is to offer a lineup of models that users can switch between at any time, profiting from a market where models evolve rapidly and no winner stays on top for long.
GPUHedge is an open-source tool that uses speculative execution to hedge between serverless GPU providers, reducing cold start p95 latency from 117s to 30s.
Serverless GPUs may have higher hourly rates but can be more cost-effective overall depending on workload peak-to-average demand. The article on Modal's blog illustrates this with a widget.
Modal has announced that replicas of vLLM and SGLang servers now start up 3-10x faster, leveraging improvements in GPU health management and CUDA context checkpointing.