low-communication

DeepMind introduces Decoupled DiLoCo, a new distributed AI training architecture that enables resilient, low-bandwidth training of large models across globally dispersed data centers by isolating hardware failures.