fsdp2

HRM-Text is a 1B parameter text generation model that uses a brain-inspired hierarchical recurrent architecture to achieve efficient pretraining with only 40B tokens and ~$1000, enabling accessible foundation model training with dramatically reduced compute and data requirements.