Tag
An arXiv paper presents an AI-driven framework for thermal-aware data center capacity planning that predicts temperatures in milliseconds, achieving high accuracy on unseen designs with a 10,000x speedup over high-fidelity CFD simulations, enabling real-time optimization of workload distribution and HVAC cooling for LLM-driven data centers.
Anthropic's confidential IPO prospectus reveals a potential $84.5 billion compute bill with SpaceX through 2029, nearly double previous estimates, with the ability to cancel most spending with 90 days' notice.
GitHub experienced a significant outage on August 17 due to capacity failures, disrupting services for hours. The company is accelerating efforts to improve infrastructure reliability and scale systems to handle growth.
This paper argues that the optimal tokenizer vocabulary size is not fixed but depends on the deployment regime, such as batch size and inference volume. Through roofline analysis and experiments on A10G and A100 GPUs, it shows the lifecycle-optimal vocabulary can shift by up to 16x between on-device and datacenter serving, with minimal quality impact.
Explores how to quantify the contribution of each latency percentile to mean latency using the Lorenz curve, and connects it to concurrency via Little's law, demonstrating that tail latency often dominates cost and capacity.
Fastly uses the Gini coefficient, a macroeconomic metric for inequality, to model traffic inequality and plan edge capacity, outperforming complex ML models for rare but critical events.
An article explaining why queues are not an effective solution for handling system overload and discussing better approaches.
Token costs are emerging as a key enterprise concern for AI adoption, with CIOs struggling to manage spending across different models and use cases. OpenAI announced Guaranteed Capacity to address long-term compute access.