Tag
This paper proposes a Cognitive Kardashev Scale ranking civilizations by their sustained AI-grade computation capacity, using total power and efficiency. It places current humanity at K≈0.73 and explores future scaling trajectories.
Presents a unified neural scaling law that accurately models deep neural network scaling across multiple dimensions including parameters, dataset size, training steps, and compute, validated across diverse architectures and tasks.
A research paper from Stanford University proposes that with sufficient compute, the best data filtering strategy is no filtering. Experiments show that large-scale models are robust to low-quality data, and unfiltered data pools perform better at larger scales. However, this conclusion applies to standard pre-training of dense models, and filtering remains important when compute is limited.
A critical take on the scaling argument for AI reasoning, arguing that autoregressive LLMs cannot achieve correctness through more compute alone, and highlighting alternative architectures like EBMs and formal verification as superior for critical applications.
Ex-Google CEO Eric Schmidt states that the real limit to AI is financial, not energy, estimating that 10 gigawatts of compute could cost half a trillion dollars, which only a few entities like the US or China can afford.