Ran a six-model swarm like a 1930s design bureau. The swarm added +0.05, splitting the work did everything else

Reddit r/AI_Agents Papers

Summary

A self-run experiment compares a six-model 1-2B local swarm (modeled on a 1930s Soviet design institute's discipline-based org chart) against a single model on a document extraction task, finding the real win came from splitting inputs into blocks rather than agent collaboration (+0.05 from the swarm, with strict emergence at zero).

Everyone's stacking agents right now, so here's a boring result. Six small local models as a swarm, organized like an old Soviet design institute: disciplines, checks inside, checks across. The swarm part added +0.05 over the best single model. Cutting the input into blocks did the rest. On my extraction task (18 records in the context, return the ones matching a condition) recall goes from 0.609 at position 0 to 0.214 at position 17. Monotonic, no U. The model just says "no" more and more often the further it reads. So I did what a design institute does with any job that's too big for one guy. Cut it into disciplines. Why that was my first idea. Albert Kahn's Detroit firm more or less bootstrapped Soviet industrial design around 1930, five hundred-something factories and a few thousand trained engineers. The org chart stuck: split by discipline, check inside the discipline, check across disciplines, normocontrol before anything leaves the building. I work in one of those institutes. Yes, I know how this sounds. Setup: six local models, all 1-2B (qwen2.5-coder-1.5b, llama-3.2-1b, internvl3-2b, qwen3-1.7b, gemma-3-1b, smollm2-1.7b). 40 test tasks, 18 records each, cut into 3 blocks of 6 by a script. Not by a model, an LLM planner scored 22% in an earlier run of mine. Every model reads every block. An id that isn't in its block gets dropped as a hallucination by a dumb check. Votes for a record only count from its own block. 720 calls, about 10 minutes on a GTX 1660 SUPER 6GB. Thresholds written down before the run. Recall, head vs tail: records full context blocks positions 0-5 0.508 0.595 positions 12-17 0.304 0.574 gap 0.204 0.021 Gap down 10x. Position stopped mattering. The part I didn't expect. Exact set match per model: model full context blocks qwen2.5-coder-1.5b 0.175 0.700 llama-3.2-1b 0.400 0.725 internvl3-2b 0.575 0.825 qwen3-1.7b 0.725 0.850 gemma-3-1b 0.000 0.050 smollm2-1.7b 0.000 0.000 The weaker the model, the bigger the jump. The coder went 4x and nearly caught the leader. Which is literally Kahn's whole pitch: divide the work right and mediocre workers produce good output. gemma and smollm can't do the task at all, and no org chart fixes that. Now the embarrassing part. The actual goal was emergence, the swarm getting right what no single model could. My pre-registered threshold said 34 of 40, champagne. Then I looked at what I was comparing: the full pipeline against a raw single model. Put the best single model (qwen3-1.7b) through the same pipeline: 0.850. All six: 0.900. Oracle "any of the six got it": 0.975. Strict emergence: zero. The swarm adds +0.05 and never steps outside what at least one model already had. The 34 stays in the json next to the 0, deleting it would be cheating. So out of seven Kahn components one carried the load, the split. The in-block check caught 5 hallucinated ids for free. The cross-discipline check had nothing to do, blocks in this task are independent. That's the next experiment. Caveats before someone asks: 1-2B only, I have 6GB. Gains shrink as the model gets stronger (qwen3-1.7b: +0.125), so on 7B+ I'd expect less. Not measured. 40 tasks, one task family, one split. Block size 6 came from the recall curve, never tuned. Overall exact set 0.750 -> 0.900. The final yes/no hit 1.000 but that's the softer metric, 0.900 is the honest one. If you feed a small model a pile of documents in one go: don't. Cut it, run the pieces, check ids with code, not with another model. Model-as-judge is a separate post, short version: worse than you think. Report, protocol and scripts are in the comments, links in posts aren't allowed here. If you run multi-agent setups: did you ever put the swarm against its best single agent run through the same pipeline? Curious if anyone got more than noise. The experiment, the numbers and the dumb emergence threshold are mine. (not a native speaker, an LLM helped with the English)
Original Article

Similar Articles

Building World Models with Agent Swarms

Reddit r/AI_Agents

The author describes building a world model harness that coordinates research agents to maximize evaluation metrics, achieving a 25x score improvement in a multimodal masked reconstruction task using geospatial inputs.

Agent swarms and the new model economics

Hacker News Top

Cursor's new agent swarm design uses planner and worker models to decompose tasks into a tree structure, achieving significant cost savings and quality improvements. In a test rebuilding SQLite from scratch in Rust, the new swarm reached 80% pass rate in four hours while the old swarm failed.

The surprising leadership dynamics in agent swarms (17 minute read)

TLDR AI

Raindrop.ai 实验让来自四个不同模型的八个 AI 智能体在无指定领导的情况下协同处理模拟故障,发现仅有 5% 的实验涌现出真正能改变队友行为的“领导者”,且 Claude Sonnet 5.5 与 GPT-6.1 Sol 比 Gemini 3.1 Pro 更容易成为领导者,提示智能体编排中的细微配置(如模型选择、名称顺序)会显著影响群体协调。