What's the next big "scaling era" for LLMs

Reddit r/singularity News
llm-scaling scaling-laws chain-of-thought agents recursive-self-improvement ai-trends speculation

Summary

The author reflects on past LLM scaling eras (model size, chain-of-thought, agents) and speculates about the next big scaling dimension, suggesting recursive self-improvement as a potential breakthrough.

imo there have been three eras bigger model better Chain-of-thought Agents with many minor improvements under these (longer context, multimodality, moe, distillation, quantization etc) Both chain of though and agents were things that bubbled under the surface for some time before it broke through. Also imo, they are advancements that saturate and long-term improvements are slower than linear wrt the compute put into it. what is it now that is "bubbling" and could be considered as a next dimension of scaling or big step change? Update: RSI - imo - answers the "who" not the "what". Ofc it means more compute allocated to training again. What breakthrough is likely to be RSI-d ?
Original Article

Similar Articles

The Scaling Laws of Skills in LLM Agent Systems

arXiv cs.CL

This paper identifies two coupled scaling laws for skill libraries in LLM agent systems: routing accuracy decays logarithmically with library size, and execution dynamics show a rescue effect. The laws are validated across 15 models and over a million decisions, and law-guided optimization significantly improves performance.

Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key

Hugging Face Daily Papers

This paper introduces ScaleLogic, a framework demonstrating that RL training compute scales as a power law with reasoning depth in LLMs. It highlights that logical expressiveness is key to improving downstream transfer and training efficiency.

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Hugging Face Daily Papers

This paper introduces AutoTTS, an environment-driven framework that automates the discovery of test-time scaling strategies for LLMs by formulating it as controller synthesis. It demonstrates improved accuracy-cost tradeoffs on mathematical reasoning benchmarks with minimal computational overhead.

Why scaling alone will not give us rational AI

Reddit r/ArtificialInteligence

This article argues that fundamental architectural limitations, not scaling deficits, prevent current LLMs from achieving true rationality—the ability to recognize and switch frames—citing empirical failures like the reversal curse and frame-transfer issues, and suggests that scaling alone may not bridge this gap.

Rethinking Continual Experience Internalization for Self-Evolving LLM Agents

arXiv cs.CL

This paper investigates why LLM agents suffer from progressive capability collapse under multi-iteration experience internalization and proposes a robust recipe addressing experience granularity, injection patterns, and training regime. Key findings include that principle-level experience, step-wise injection, and off-policy context-distillation yield more stable and sustainable continual learning.