@sitinme: Saw Karpathy open-sourced a very interesting project autoresearch, which gives a real but small-scale LLM training task to an AI Agent, letting it do research, modify code, run experiments, look at results, and then decide whether to keep or discard the changes. The project is based on a single NVIDIA…

Saw Karpathy open-sourced a very interesting project called autoresearch, which hands a real but small-scale LLM training task to an AI Agent, letting it do research, modify code, run experiments, look at results, and then decide whether to keep or discard the changes. The project is based on the nanochat training pipeline on a single NVIDIA GPU. Each time, the Agent modifies the training code, runs for about 5 minutes, and checks if the validation metrics improve. If they improve, it keeps the changes; if not, it discards them, and then proceeds to the next round. In other words, you hand the task to the Agent at night, and when you wake up the next day, you see a series of automated experiment logs, as well as a model that may have been optimized. The project is deliberately kept very small, with just a few files: http://prepare.py handles data preparation and utility functions, http://train.py is the training code that the Agent actually modifies, and program.md serves as the research instructions for the Agent. Humans are more involved in writing the "research plan" and constraints, rather than manually tweaking model architecture, hyperparameters, and training logic bit by bit as before. It demonstrates a new way of doing AI research: not just having AI help you write code, but having AI participate in the full experimental loop. It proposes changes, executes experiments, evaluates results, and iterates continuously. This is a bit like putting a junior researcher into a controlled environment and letting them experiment through trial and error. Currently, autoresearch is still a rather experimental project, and by default it is better suited for a single NVIDIA GPU, especially high-performance cards like the H100. For Mac, Windows, or AMD platforms, some people have already forked versions.