Trained a 1.5B to write shell commands so I'd stop googling tar flags. Runs on a laptop CPU in ~1 sec.

Reddit r/LocalLLaMA Models
fine-tuning shell-commands qwen llama-cpp open-source local-llm

Summary

The author fine-tuned Qwen2.5-Coder-1.5B on 125k natural-language/command pairs to generate shell commands, runs locally on a laptop CPU in ~1 second, and published the weights and code.

I've been googling "tar extract gz" for about ten years. and I finally did something about it. It started out as a research project and I ended up with a Fine-tuned Qwen2.5-Coder-1.5B on 125k natural-language/command pairs, merged and quantized to Q4_K_M. 941MB which runs through llama.cpp. On my laptop (i5-11320H, 4 threads): 31.9 tok/s, 0.59s median per query, 1.6GB RAM. I benchmarked it and it scores 0.620 on InterCode-ALFA. Untuned Qwen2.5-Coder-7B gets 0.613, GPT-4o gets 0.73. Not frontier, but it's roughly a 7B's answer at a quarter the parameters on a CPU. Theres a 3B variant too that scores higher. There's also few static safety checker, because it will absolutely write a command that wipes your root if you ask it to: I have published the weights: huggingface.co/ThorOdinson246/nl2sh-1.5b-Q4_K_M and Code: github.com/ThorOdinson246/whatisit-nl2sh . I posted few days ago in LocalLLM and it did well 300+ stars and so many good suggestions so I figured people here will be interested too. Both Apache-2.0. If you want to poke holes in the method or you've got ideas, please comment or open a PR. A ⭐ helps if you find it useful.
Original Article

Similar Articles

Running Qwen3.6 35b a3b on 8gb vram and 32gb ram ~190k context

Reddit r/LocalLLaMA

The author shares a high-performance local inference configuration for running Qwen3.6 35B A3B on limited hardware (8GB VRAM, 32GB RAM) using a modified llama.cpp with TurboQuant support, achieving ~37-51 tok/sec with ~190k context.