Tag
A 117M parameter Silia model was trained on an H100 GPU in 5 hours using 82M tokens. The model is severely under-trained, but comparisons with nanoGPT are provided.