a 144m model makes most of my villagers' choices, gemma 4 e4b only steps in when something goes wrong (6 gb laptop gpu)

Reddit r/LocalLLaMA Tools

Summary

A personal project describes a village simulation where a 144M-param julia-1 'gut' model scores villagers' actions on CPU, embeddingGemma retrieves relevant memories, and Gemma 4 e4b (6GB laptop GPU) is only invoked when plans fail or surprises occur. Results show the gut is rarely uncertain, and nightly rule-learning from mistakes cut injuries from 2.73 to 0.53 per valley.

i built a small village sim where 8 villagers live through a day, and i wanted to see how far small models get if each one only does one job. the gut is julia-1, 144m params from supersonic labs, using ggml-org's gguf. it runs on the cpu and doesn't generate text at all. the world hands it the few actions a villager can physically take right now and it scores them. it can't invent an action, so it can't drift into roleplay. the memory is embeddinggemma, also on cpu. before each choice it pulls the 3 past moments that matter most: recent ones, big ones, and ones that look like right now. the slow thinker is gemma 4 e4b on a 6 gb laptop gpu. it only gets called when the gut isn't sure or something surprises the villager, like a plan failing, someone getting hurt or a path being blocked. what surprised me is that the gut is rarely unsure. at crisis moments julia-1's top choice usually had 0.81 to 0.90 of the probability. so gemma mostly gets called because something went wrong, not because julia hesitated. julia-1 is used zero-shot. i never trained or fine-tuned it. the villagers still learn, because at night gemma turns what worked into a short rule (22 words max, 5 per villager) that sits in front of the models next time. in one landslide a relevant rule moved arun's chance of going to fetch a neighbour from 0.0002 to 0.638. an irrelevant rule changed nothing. two warnings if you try a scorer like this. wording matters a lot: "cross the bridge and bring back food from the far side" got 0.2%, and the same trip as "cross the bridge to the good fields and bring back a full basket" got 53% to 90%. and rules borrowed from neighbours were rarely used when i audited them. learning from their own mistakes is what worked: over 15 valleys with a rockfall, injuries went from 2.73 per valley with memory only to 0.53 with learning. full write-up (mine): https://x.com/aksssrao/status/2117264537598800067
Original Article

Similar Articles

You don't need a GPU to run gemma-4-26B-A4B

Reddit r/LocalLLaMA

The author demonstrates that the Gemma-4-26B-A4B model runs efficiently on a CPU-only system using Koboldcpp, achieving 7 tokens per second on an old desktop, suggesting that powerful GPUs may not be necessary for local LLM inference.