from-scratch-training

Tag

Cards List
#from-scratch-training

@bkdgiffug: Training a language model in most people's minds requires a PhD and a stack of GPUs — that view is long overdue for an update. Yet someone spent just five minutes, opened a single Colab notebook, and built a 9-million-parameter model from scratch, naming it GuppyLM. The premise is quite fun — a fish named Guppy that only speaks...

X AI KOLs Following ↗ · 2026-10-29 Cached

GuppyLM is a small ~9M-parameter language model that can be trained from scratch in 5 minutes on a single Colab notebook. It simulates a fish named Guppy talking about water, food, and light, aiming to prove that training your own language model isn't magic — it's something anyone can do.

0 favorites 0 likes
#from-scratch-training

I trained a 3.87B MoE (1.45B active) from scratch on only 86.5B tokens

Reddit r/LocalLLaMA ↗ · 2026-10-04 Cached

YOON1v released Apex-2, a from-scratch decoder-only Mixture-of-Experts LLM with 3.87B total / 1.45B active parameters, pretrained on 86.5B tokens and SFT-tuned, with full code, architecture writeup, and benchmark results (HumanEval 43.9) released on Hugging Face and GitHub.

0 favorites 0 likes
← Back to home

Submit Feedback