Multilingual Tiny (3.7B) Reasoning MoE pretrained from scratch on a consumer-grade GPU

Reddit r/LocalLLaMA Models
multilingual small-model reasoning mixture-of-experts consumer-gpu pretrained open-source

Summary

A multilingual 3.7B parameter reasoning MoE model has been pretrained from scratch on a consumer-grade GPU over several months, with support for 13 languages and available on Hugging Face.

Hello! I've just uploaded a recent checkpoint of my model trained from scratch: https://huggingface.co/piotr-ai/polanka_3.7b_exp_wip_260901 It was pre-trained, mid-trained, and fine-tuned on a single 4090 over many months. How many tokens? I lost count. Feel free to use it as a research artefact. 13 languages: PL, EN, ZH, CS, SK, UK, RU, IT, ES, FR, DE, PT, LT — with extra upscaled data for PL/EN/ZH.
Original Article

Similar Articles

Domyn-Small: A European 10B Reasoning Language Model

arXiv cs.CL

Domyn-Small is a 10-billion-parameter open-weight reasoning language model released under MIT license, trained on 9 trillion tokens and optimized for reasoning, instruction following, and tool use. It achieves strong accuracy-efficiency balance against peers like Qwen3.5-9B and OLMo-3-7B-Think.

Liquid AI reveals 8B-A1B MoE trained on 38T

Hacker News Top

Liquid AI released LFM2.5-8B-A1B, an edge MoE model trained on 38T tokens with a 128K context window, improved tool calling, and reasoning capabilities, available on Hugging Face.