This paper develops a framework, SpatialCLI, to help vision-language models (VLMs) better understand and use visual tools to make better decisions. By training VLMs to reason with spatial tools and then internalize those capabilities, SpatialCLI can improve the performance of VLMs in tasks that require visual reasoning.
Firehose
Filtered to tagged “vision-language models” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives
Browse by tag
artificial intelligence 57continual learning 24agentic coding 21open-weight models 20AI 16AI agents 13reinforcement learning 11cybersecurity 9AI safety 8finance 8language models 7large language models 7open-source 7productivity 7deep learning 6machine learning 6natural language processing 6Reinforcement learning 6tech 6Databricks 5robotics 5software development 5Agentic AI 4benchmarking 4Diffusion models 4multi-agent systems 4Recursive self-improvement 4world models 4AI ethics 3AI infrastructure 3
This paper proposes a new approach to improve vision-language models for visual retrieval, which can handle long visual contexts and large numbers of distractors. Practitioners might care because it can lead to better performance on image and video benchmarks.
This paper evaluates whether vision-language models can act through a physical body and how they can make decisions about what actions to take, without being hindered by issues like balance and motor control. Practitioners in AI and robotics might care because understanding how models interact with their physical bodies can help improve their ability to navigate and interact with the world.