Tag
This paper introduces Collective-State JEPA (CS-JEPA), a recurrent joint-embedding predictive architecture that lets every robot in a swarm predict the same future collective state from local observations and limited messages. The method shows label-efficient improvements in prediction error and inter-robot agreement, plus planning-relevant value estimation.
The paper proposes the state-prediction separation hypothesis, suggesting that separating state prediction from token prediction in Transformers improves language modeling performance and efficiency across scales, with experiments showing consistent gains.