Tag
This paper introduces Coupled Scaling, a task-conditioned framework that explains how neural scaling laws vary based on the relationship between task structure and the geometric representations accessible by architecture-optimization systems.
The paper introduces the Skaling law, a generalized neural scaling law that couples model capacity and data through an interaction exponent, reducing prediction error by 1.5-3x and enabling full-grid extrapolation using roughly 10x less compute.
Presents a unified neural scaling law that accurately models deep neural network scaling across multiple dimensions including parameters, dataset size, training steps, and compute, validated across diverse architectures and tasks.