scalable-compute

Agora enables collective, permissionless internet-scale pretraining of large language models using heterogeneous, preemptible consumer GPUs connected via internet, demonstrated by the successful Pluralis-8B training run with 330 nodes.