gpt2

Pathway's BDH, a post-transformer architecture, reportedly matches GPT-2 scaling from 10M to 1B parameters while training from scratch on standard GPUs.

DrugGen-2 fine-tunes GPT-2 using supervised learning and reinforcement learning (GRPO) to generate small molecules conditioned on both disease ontology and target protein sequences, achieving superior diversity and binding affinity for drug discovery.

GPT2-BASIC implements a GPT-style transformer inference engine in FreeBASIC for DOS systems, using fixed-point arithmetic and local model artifacts to run AI-assisted tasks on vintage hardware.

Justin Angel released a complete YouTube workshop teaching you how to build your own large language model from scratch (based on GPT-2 and Qwen3.6 style), covering Transformer architecture, training pipeline, and providing Excel manual operations and Python/PyTorch code practice, with no prerequisites in math or ML.