Tag
The article argues that text optimization—modifying prompts, context, memory, and retrieval—should be treated as a legitimate learning mechanism alongside weight optimization, highlighting its sample efficiency and ability to scale via update-time compute.
The author argues that text optimization (prompts, context, memory) is a legitimate and sample-efficient learning mechanism that should be taken more seriously by the ML community, enabling a new scaling axis of update-time compute.