Tag
A 117M parameter Silia model was trained on an H100 GPU in 5 hours using 82M tokens. The model is severely under-trained, but comparisons with nanoGPT are provided.
A user announced they trained a custom 78M parameter AI architecture from scratch using 1B tokens on an 8x3090 rig over 12 hours, claiming the model can now speak.