@rohanpaul_ai: Brilliant piece by Zhipu Founder Tang Jie. AI scaling is moving past parameter growth. “How many parameters?” is becomi…

X AI KOLs Following News
ai-scaling language-models parameters training-data compute post-training

Summary

Zhipu Founder Tang Jie discusses how AI scaling is evolving beyond parameter count to include factors like training data, compute per forward pass, and post-training, with GLM-5.3 as an example.

Brilliant piece by Zhipu Founder Tang Jie. AI scaling is moving past parameter growth. “How many parameters?” is becoming a weak way to describe how capable a model should be. That model scaling now has several independent dials: parameters, training data, compute per forward pass, and post-training. The best place to spend the next unit of compute depends on what the model needs to do and how often it will be used. “Total parameters appear to matter up to a threshold — enough to hold the world — after which additional capability comes from scaling elsewhere: effective depth per forward pass, and above all post-training.” “Put inference into the objective and the optimum moves toward smaller models trained far longer.” GLM-5.3 is Tang’s implented example: same base, architecture, total parameters, and activated parameters as GLM-5.2, but 1 more month spent on long-horizon environments and RL.
Original Article
View Cached Full Text

Cached at: 08/20/26, 03:11 PM

Brilliant piece by Zhipu Founder Tang Jie.

AI scaling is moving past parameter growth.

“How many parameters?” is becoming a weak way to describe how capable a model should be.

That model scaling now has several independent dials: parameters, training data, compute per forward pass, and post-training.

The best place to spend the next unit of compute depends on what the model needs to do and how often it will be used.

“Total parameters appear to matter up to a threshold — enough to hold the world — after which additional capability comes from scaling elsewhere: effective depth per forward pass, and above all post-training.”

“Put inference into the objective and the optimum moves toward smaller models trained far longer.”

GLM-5.3 is Tang’s implented example: same base, architecture, total parameters, and activated parameters as GLM-5.2, but 1 more month spent on long-horizon environments and RL.

jietang (@jietang): Thoughts About Scaling Law

Scaling, but not only of parameters. Every model release now ends with the same question: how many parameters? It isn’t a question that can be answered on its own. Parameter count is only meaningful alongside three others — how much data you have,

Similar Articles