Filtered to vision-language models · clear filter
This paper develops a vision-language model for autonomous driving that combines 3D perception, question answering, and motion planning. A practitioner might care because it demonstrates a promising approach to integrating multiple tasks in autonomous driving.
This paper introduces MJev-Spatial, a new AI model that can quickly understand and act in the physical world by using a unified classification framework to answer spatial intelligence tasks. Practitioners may care about this model because it can solve real-world problems faster and more efficiently than existing models.
This paper develops a vision-language model (VLM) that can accurately perceive objects and pixels in images, which can be useful for various applications such as self-driving cars, medical imaging, and augmented reality. Practitioners in these fields may care about the potential improvements in visual perception and object detection that this model can bring.
This paper introduces GroundingPI, a new AI model that improves its ability to understand and interact with the physical world by learning to precisely identify objects and their locations, even in cluttered environments. Practitioners may care about GroundingPI's potential to enhance applications like robotics, autonomous driving, and object recognition.
This paper develops proactive video agents that can understand and respond to user intent in real-time, improving interaction quality and response timing. Practitioners can benefit from this research to create more intuitive and responsive video interfaces.
This paper evaluates how well vision-language models can understand 3D objects from 2D observations and make precise actions. Practitioners might care because it highlights the limitations of current models in tasks that require inferring object geometry and anticipating the effects of rotation.
This paper develops a unified model that can understand physical environments, generate actions, and predict future states, using a combination of vision, language, and embodied interactions. Practitioners may care about this model because it could be used to create robots or other agents that can interact with and adapt to their physical surroundings.
This paper develops a unified model that combines perception, memory, and response for streaming video interaction, allowing for efficient and accurate processing of video content in real-time. Practitioners in the field of natural language processing and computer vision may care about this research as it aims to improve the performance of streaming video understanding models.
This paper introduces a new benchmark for video generation tasks that require both achieving a desired outcome and maintaining semantic consistency with a reference image. Practitioners might care about this research because it can help evaluate and improve the performance of video generation models in real-world applications.
This paper introduces a new model called PDRVLA that helps robots understand the physical world from visual and language inputs, and generate actions accordingly. Practitioners in robotics and AI might care about this research because it aims to improve robots' ability to perform tasks in real-world environments.
This paper helps vision-language models (VLMs) think and explore in 3D space by interacting with the environment through actions and observations, rather than receiving pre-rendered images. Practitioners might care because it can improve the accuracy of VLMs on 3D benchmarks.
This paper helps large Vision-Language Models (LVLMs) better understand and reason about 3D scenes from 2D images, a key aspect of spatial intelligence, by training them on a synthetic dataset of block-stacking problems. Practitioners might care because improving spatial intelligence can lead to better performance on various visual tasks.
This paper introduces a benchmark, SpatialScout, to evaluate vision-language model (VLM) agents on active 3D mapping, where agents must explore and map 3D scenes to construct accurate metric maps. Practitioners in AI and robotics might care about this research because it aims to develop agents that can build reliable 3D spatial representations through active perception.
This paper develops a new AI model, Hy-Nomas, that can generate interactive worlds in response to user commands, allowing for more realistic and dynamic environments. Practitioners in fields like game development or virtual reality might care about this work because it could enable more immersive and interactive experiences.
This paper investigates whether the intelligence of vision-language models (VLMs) can be transferred to robotic control, enabling robots to learn and adapt in both digital and physical worlds. Practitioners might care because it could lead to more human-like robots that can seamlessly adapt to new environments and tasks.
This paper introduces Co-RL, a framework for unsupervised multi-agent reinforcement learning that enables diverse and accurate reasoning in language and vision-language models. Practitioners can use Co-RL to improve their models' ability to reason and respond without relying on expensive ground-truth supervision.
This paper shows how a vision-language model (VLM) can control robots without needing extensive pretraining or specialized hardware, by providing a compact interface that links the model's intentions to specific actions. Practitioners might care about this because it could make robots more accessible and user-friendly.
This paper introduces a system called MotorMind that allows a general-purpose vision-language model to control a robot without relying on external models or tools, enabling zero-shot manipulation in new tasks and environments. Practitioners might care because it could simplify and improve robotic manipulation tasks.