This paper proposes a method to combine reinforcement learning with verifiable rewards and on-policy distillation to improve performance on complex tasks, and shows that this method can lead to more stable training and better results.
Firehose
Filtered to tagged “benchmarking” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives
Browse by tag
artificial intelligence 57continual learning 24agentic coding 21open-weight models 20AI 16AI agents 13reinforcement learning 11cybersecurity 9AI safety 8finance 8language models 7large language models 7open-source 7productivity 7deep learning 6machine learning 6natural language processing 6Reinforcement learning 6tech 6Databricks 5robotics 5software development 5Agentic AI 4benchmarking 4Diffusion models 4multi-agent systems 4Recursive self-improvement 4world models 4AI ethics 3AI infrastructure 3
This paper investigates whether AI agents can conduct open-ended AI research and provides early evidence that they can perform the engineering aspects but struggle with critical parts of the research lifecycle, such as making progress on research questions and judgment about publishable research.
This paper introduces a benchmark to evaluate large language model (LLM) agents on long-horizon office-suite tasks, considering their cost-effectiveness and quality. Practitioners can care about this research because it aims to ensure LLM agents can assist users efficiently and effectively.
This paper creates a benchmark to test the security capabilities of AI agents in a real-world setting, specifically incident response, and finds that current agents struggle to detect and remediate silent intrusions and produce verified plans.