Tag
Epoch AI analyzes whether financing will bottleneck AI compute scaling, using Anthropic's $50B infrastructure buildout funded by debt as a case study. It argues institutional investors are willing to lend against long-term payment commitments, especially with supplier backing, making financing unlikely to be the immediate limit on frontier compute growth.
A detailed speculative thread on the near future of AI, arguing that algorithmic progress will surprise many, with 4-10 orders of magnitude improvement in intelligence possible, and that we are in an early takeoff where AI accelerating AI research will lead to rapid advances.
A personal observation that acquiring more compute only reveals further needs for compute, reflecting the insatiable demand in AI and tech.
This paper proposes a Cognitive Kardashev Scale ranking civilizations by their sustained AI-grade computation capacity, using total power and efficiency. It places current humanity at K≈0.73 and explores future scaling trajectories.
Presents a unified neural scaling law that accurately models deep neural network scaling across multiple dimensions including parameters, dataset size, training steps, and compute, validated across diverse architectures and tasks.
A research paper from Stanford University proposes that with sufficient compute, the best data filtering strategy is no filtering. Experiments show that large-scale models are robust to low-quality data, and unfiltered data pools perform better at larger scales. However, this conclusion applies to standard pre-training of dense models, and filtering remains important when compute is limited.
A critical take on the scaling argument for AI reasoning, arguing that autoregressive LLMs cannot achieve correctness through more compute alone, and highlighting alternative architectures like EBMs and formal verification as superior for critical applications.
Ex-Google CEO Eric Schmidt states that the real limit to AI is financial, not energy, estimating that 10 gigawatts of compute could cost half a trillion dollars, which only a few entities like the US or China can afford.