Tag
This paper presents MiniGPT, a compact from-scratch implementation of GPT-style autoregressive language modeling in PyTorch, built after studying nanoGPT. It evaluates the model on the Tiny Shakespeare dataset using character-level tokenization, achieving a validation loss of 1.4780 with a 10.77M-parameter configuration.