Built and released BetterGPT-150M – A compact 150M parameter completion model (+ live HF Space demo)

Reddit r/LocalLLaMA Models
bettergpt-150m lightweight-model text-generation causal-language-model open-source huggingface-demo

Summary

Built and released BetterGPT-150M, a compact 150M parameter causal language model that outperforms GPT-2 Small with low resource footprint. Includes live Hugging Face Space demo for text completion.

Hey everyone, ​I recently finished pre-training BetterGPT-150M, a small, lightweight causal language model with ~152 million parameters.Trained on 15B tokens. Dataset & Training: Trained across stable and annealing phases using curated datasets (including FineWeb-Edu, fine maths, cosmopedia, starcode-python), ensuring strong capability retention while maximizing token efficiency. ​Performance: Benchmark evaluations show it outperforms GPT-2 Small while remaining on par with models trained on significantly larger token budgets. ​Since many small/tiny models tend to get buried under massive LLM releases, I wanted to share it here for anyone interested in lightweight architectures, fast CPU inference, or small edge-device experimentation. Repo: https://github.com/harikrish2727/BetterGPT ​Model Hub: https://huggingface.co/Harikrish2727/BetterGPT-150M ​Live Demo: https://huggingface.co/spaces/Harikrish2727/BetterGPT-Demo (Hosted on ZeroGPU with token streaming) ​Model Notes: ​Task: Text completion / generation (it is a standard base completion model, not instruction-tuned). ​Footprint: Very low RAM/vRAM footprint, runs instantly on standard CPUs. ​Feel free to try out prompts on the Space demo or pull the weights to run locally. Feedback, benchmark suggestions, or ideas for instruction fine-tuning are always welcome! This is my very first serious try, hope I get genuine feedback from you guys.
Original Article

Similar Articles

Better language models and their implications

OpenAI Blog

OpenAI introduces GPT-2, a 1.5 billion parameter transformer-based language model trained on 40GB of internet text that achieves state-of-the-art performance on language modeling benchmarks and demonstrates zero-shot capabilities in reading comprehension, translation, question answering, and summarization. Due to safety concerns, only a smaller model and technical paper are released publicly rather than the full trained model.

[NEW] Supra-50M Released!

Reddit r/LocalLLaMA

SupraLabs released Supra-50M, a compact 50M-parameter causal language model with base and instruct versions, trained on 20B tokens from fineweb-edu, achieving competitive benchmarks against larger models like GPT-2 and SmolLM.

Introducing GPT-5.1 for developers

OpenAI Blog

OpenAI releases GPT-5.1, a new model in the GPT-5 series that dynamically adapts thinking time based on task complexity, offering 2-3x faster performance than GPT-5 while maintaining frontier intelligence. The release includes extended prompt caching (24-hour retention), new coding tools (apply_patch and shell), and a 'no reasoning' mode for latency-sensitive applications.

MiniGPT: Rebuilding GPT from First Principles

arXiv cs.CL

This paper presents MiniGPT, a compact from-scratch implementation of GPT-style autoregressive language modeling in PyTorch, built after studying nanoGPT. It evaluates the model on the Tiny Shakespeare dataset using character-level tokenization, achieving a validation loss of 1.4780 with a 10.77M-parameter configuration.