@snowboat84: 你有没有发现，AI里模型的诞生其实相当随意？拿语言模型举例子：先是RNN，再到LSTM，某天突然说Transformer效果好就全换上，后来又拆成Encoder和Decoder，一会儿说BERT一桶浆糊，一会儿又说GPT可以有涌现能力，S…

你有没有发现，AI里模型的诞生其实相当随意？拿语言模型举例子：先是RNN，再到LSTM，某天突然说Transformer效果好就全换上，后来又拆成Encoder和Decoder，一会儿说BERT一桶浆糊，一会儿又说GPT可以有涌现能力，Scaling Law。整个过程几乎没有理论指导。现在试图在大语言模型之外异军突起的“世界模型“也一样，没什么理论根基，先把模型搭起来。

说白了，现在造模型基本就三招:（1）在别人的模型上改一点点，（2）自己以前做过类似的模型，就惯性套到别的场景试试，（3）突发奇想攒出一个架构，也不知道行不行，先试试看。模型能不能打，全靠benchmark + 算力 + 运气，不是因为它更接近某种真理。虽然今天Transformer看起来一统江湖，但是谁知道它能撑到什么时候，Transformer的边界又在那里。说不定过了几年，大语言模型到达了边界地带，我们又要模出新模型。

我们的想法是：应该给备选模型建立一个资料库，这样我们就可以把随意选模型的整个过程给工程化。

至于模型本身，可以从物理学里获得灵感。其实不少AI模型本来就来自于物理的多体系统模型。比如从Ising获得启发制造出来的玻尔兹曼机和限制玻尔兹曼机，从统计力学启发得到的能量模型，以及生成图片和视频的扩散模型，背后全是物理模型的灵感。

物理学是一个巨大的资源库，里面还躺着大量没人试过的模型。Transformer看着一统江湖，但即便今天，它在很多任务上做得其实并不好，比如把训练时没见过的更长输入丢给它，它就崩。这些Transformer啃不动的地方，恰恰可能是某个物理模型的主场。

与其继续靠运气摸彩票，不如把这件事工程化：把物理学里那些成熟的模型，一个个翻出来、做成可训练的网络，拿去跟主流硬碰，记下哪一个擅长哪一类任务。这样下一次需要新模型时，我们不是凭感觉攒一个，而是去库里查、去挑。

我先把所有可能的备选物理模型整理成了一张表，开了个repo，准备一格一格做下去。

Have you ever noticed that the birth of models in AI is actually quite arbitrary? Take language models as an example: first there was RNN, then LSTM, and one day they suddenly said Transformers worked better and everyone switched over, later splitting them into Encoder and Decoder, one moment saying BERT was a mixed bag, the next saying GPT could have emergent abilities, Scaling Law. The whole process has almost no theoretical guidance. Now, the “world models” that are suddenly rising outside of large language models are the same—no solid theoretical foundation, just build the model first.

To put it bluntly, building models these days basically boils down to three tricks: (1) tweak someone else’s model a tiny bit, (2) take a similar model you’ve done before and inertia-apply it to a new scenario to see, (3) have a wild idea and cobble together an architecture, not knowing if it’ll work, but give it a shot first. Whether a model succeeds or not all depends on benchmark + compute power + luck, not because it’s closer to some truth. Though Transformers look like they’re dominating the scene today, who knows how long that’ll last, or where the boundaries of Transformers even are. Maybe in a few years, large language models hit their limit, and we’ll have to model out something new.

Our idea is: we should build a repository for alternative models, so we can engineer the whole process of arbitrarily selecting models.

As for the models themselves, we can draw inspiration from physics. Plenty of AI models originally come from physics’ many-body system models. For example, Boltzmann machines and restricted Boltzmann machines inspired by the Ising model, energy models inspired by statistical mechanics, and diffusion models for generating images and videos—all backed by inspirations from physics models.

Physics is a massive resource library, with tons of untried models just sitting there. Transformers may look like they’re ruling the roost, but even today, they actually don’t perform well on a lot of tasks—for instance, throw it a longer input it hasn’t seen during training, and it falls apart. Those spots where Transformers struggle are precisely where some physics model might have the home-field advantage.

Instead of keep relying on luck to draw lottery tickets, why not engineer this: pull out those mature models from physics one by one, turn them into trainable networks, pit them against the mainstream, and log which ones excel at which types of tasks. That way, next time we need a new model, we’re not cobbling one together on a hunch—we go to the library, query, and pick.

I’ve gone ahead and compiled all possible alternative physics models into a table, opened a repo, and plan to tackle it cell by cell.

坑已经挖好

谢谢支持 谢谢star，最近一定经常更新。

工程也需要理论指导

谢谢