Filtered to benchmarks · clear filter
This paper proposes a method to scale automatic research agents by replacing environment execution with a world model, which can reduce training costs and improve performance. Practitioners might care about this approach because it can accelerate training times and lead to better results for complex AI tasks.
This paper introduces GroundingPI, a new AI model that improves its ability to understand and interact with the physical world by learning to precisely identify objects and their locations, even in cluttered environments. Practitioners may care about GroundingPI's potential to enhance applications like robotics, autonomous driving, and object recognition.
This paper surveys the development of video world models that capture the underlying dynamics of the world depicted in videos, and how they relate to video foundation models. Practitioners in AI and machine learning might care about this research because it aims to create more realistic and interactive video models that can be used in applications such as virtual reality, robotics, and autonomous systems.
This paper introduces VGI-bench, a new benchmark for evaluating video generation models' visual reasoning capabilities, and finds that current models can solve some visually grounded tasks but still struggle with reliability. Practitioners may care about developing more reliable video generation models that can perform better on this benchmark.
This paper evaluates the ability of Deep Search agents to conduct end-to-end research in real-world open Web environments, where they must autonomously search for, access, and integrate relevant evidence. Practitioners may care about this research to develop agents that can effectively search and retrieve information in complex, real-world settings.
This paper introduces a new benchmark for video generation tasks that require both achieving a desired outcome and maintaining semantic consistency with a reference image. Practitioners might care about this research because it can help evaluate and improve the performance of video generation models in real-world applications.
This paper improves the efficiency of extracting the target speaker from audio-visual data in real-world acoustic scenes, which is crucial for applications like voice assistants. Practitioners may care about this research if they work on latency-sensitive voice recognition systems.
This paper introduces a method to generate high-quality long-horizon trajectories for terminal agents, allowing them to complete complex tasks in a realistic and autonomous way. Practitioners in AI/ML research and development can benefit from this approach to improve the performance and reliability of autonomous systems.
This paper proposes a framework, LongHorizon-Harness, to help large language model agents tackle long-horizon tasks by explicitly tracking task states and verifying facts from the environment. Practitioners might care because it can improve the performance of these agents on real-world tasks.
This paper introduces LoopArena, a benchmark for evaluating how well a model can guide a coding agent through a long-running task, and finds room for improvement in long-horizon loop control. Practitioners working on AI development might care because it could lead to more efficient and reliable development processes.
This paper proposes a method to optimize prompts for multi-agent systems by identifying which agent's modification resolves a failure, and then using that agent's output as supervision to extract a fine-grained gradient. Practitioners might care because it could improve the performance of large language model-based multi-agent systems.