Learning to Think and Explore in 3D World
This paper helps vision-language models (VLMs) think and explore in 3D space by interacting with the environment through actions and observations, rather than receiving pre-rendered images. Practitioners might care because it can improve the accuracy of VLMs on 3D benchmarks.