An LLM inside an LLM - Same guy who put a RISK-V CPU inside Qwen has now made Karpathy's stories260k run on an NPU compiled inside Qwen 3.5-4B's weights

Reddit r/LocalLLaMA Tools
llm-in-llm qwen npu-emulation model-compilation open-source embeddings

Summary

开发者将 Karpathy 的 stories260k 模型编译成一个 NPU，并直接嵌入 Qwen 3.5-4B 的权重中，实现了一个 LLM 在另一个 LLM 内部逐位精确地运行（包含矩阵乘、softmax 和注意力），无需外部工具。

Code: https://github.com/divagr18/QwenPU Weights: https://huggingface.co/divagr1925/QwenPU From the original tweet: 1/3 A CPU inside an LLM is one thing, but running another LLM inside an LLM would be taking things a bit too far, wouldn't it? Anyways, here's an LLM running inside an LLM : QwenPU, an LLM (@karpathy's stories260k) running on an NPU compiled inside Qwen 3.5-4B's weights. 2/3 stories260K is just a C program. To run it inside Qwen, I compiled an NPU into Qwen's weights that does matmuls, softmax and attention bit-exact. The LLM weights are in Qwen's own memory, with no external tools. A lot of work went into making it fast, more on that soon. 3/3 I'm super sick right now, so the full writeup is still a work in progress, and I'm working on optimisations and size reductions too. Stay tuned, I'll finish the blog & post it soon.
Original Article

Similar Articles

I ran Doom inside the Qwen model weights!

Reddit r/LocalLLaMA

作者受到 qwenputer 项目启发，在 Qwen 模型权重内构建的虚拟机上成功运行了基础版 Doom，虽然速度很慢，但实现了「在 LLM 上跑 Doom」的趣味演示，并开源了项目仓库。

Janas: Run LLMs larger than RAM by streaming experts from SSD

Hacker News Top

Janas 是一个开源工具，通过从 SSD 流式加载专家模型（mixture-of-experts），让内存小于模型体积的笔记本也能运行 LLM。在 32GB 内存的笔记本上，Qwen3-Next-80B（48GB）推理速度达 37 tok/s，显著超越 llama.cpp 的 2.8 tok/s。

poorman inference engine for 16GB GPU and 35B moe Qwen 3.6for coding

Reddit r/LocalLLaMA

作者将 llama.cpp 的 server 分支为 AgrillaMoE，针对 Qwen3.6-35B-A3B MoE 模型在 16GB 显存 GPU 上优化，配合 2-bit 量化可达 ~57-60 tok/s，并支持 OpenAI/Anthropic API 兼容接口；其 MoE 扩展补丁在运行时增加每 token 激活专家数（20 vs 8），无需重训即可在 GPQA-Diamond 上提升约 2.5 分。