interaction-learning

Spatial-Interactor is a framework that trains vision-language models to enhance spatial reasoning through interaction with the physical world, employing a three-level curriculum and two-stage training strategy to improve state transition modeling and long-horizon integration.

This paper presents an information-theoretic analysis of multimodal learning, revealing the need to capture sample-specific interactions, and proposes DMIL, a paradigm that explicitly models and learns from these interactions via variational decomposition and fine-tuning, achieving superior performance.