What if i really wanna train an AI from scratch?

Reddit r/artificial News
training-from-scratch gpu datasets small-models personal-experience ai-training open-source

Summary

A personal reflection on the challenges and allure of training an AI model from scratch, highlighting the difficulties with data, hardware, and scaling, while noting that surprisingly good small models can be trained on modest hardware.

I got obsessed with this idea recently 😭 Not “build an AI app.” Not “connect GPT API.” I mean actually train a model. Like downloading datasets at 3AM, watching GPUs melt, fixing random CUDA errors for 6 hours straight, training for days just to realize the dataset was garbage 💀 Everybody online makes it sound impossible unless you have billions of dollars and a data center the size of a city. But at the same time… people are out here training surprisingly good small models from bedrooms and rented GPUs. So now I’m stuck in this weird mindset where: part of me thinks this is insanely unrealistic and the other part thinks we’re super early and nobody fully knows what’s possible yet The craziest thing is realizing the model itself is only half the battle. The REAL nightmare seems to be: collecting clean data keeping outputs consistent inference costs scaling making the AI not become completely stupid after bad training 😭 Anyone else here trying this stuff seriously instead of just wrapping APIs?
Original Article

Similar Articles

AI training is becoming the new coding revolution

Reddit r/artificial

The article argues that AI training is becoming dramatically more accessible, allowing small teams and individuals to train specialized models without large infrastructure, marking a shift from corporate-dominated AI to niche-focused development.

AI is deteriorating in realtime

Reddit r/ArtificialInteligence

AI models are deteriorating due to training on recursively generated synthetic data, leading to model collapse; multiple studies highlight the risks of scaling with synthetic data.