Tag
A personal project where a 0.5M parameter language model was trained on 1 billion tokens from the Fineweb-edu dataset.
Introduces Glimmer, a 10,000 parameter language model trained on 500K tokens of FineWeb-Edu with a standard Llama architecture, available on HuggingFace.