This paper proposes a method to combine reinforcement learning with verifiable rewards and on-policy distillation to improve performance on complex tasks, and shows that this method can lead to more stable training and better results.
Firehose
Filtered to tagged “on-policy distillation” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives
Browse by tag
artificial intelligence 57continual learning 24agentic coding 21open-weight models 20AI 16AI agents 13reinforcement learning 11cybersecurity 9AI safety 8finance 8language models 7large language models 7open-source 7productivity 7deep learning 6machine learning 6natural language processing 6Reinforcement learning 6tech 6Databricks 5robotics 5software development 5Agentic AI 4benchmarking 4Diffusion models 4multi-agent systems 4Recursive self-improvement 4world models 4AI ethics 3AI infrastructure 3
This paper proposes a new method for training large language models in open-ended domains, using evolving contexts as in-training supervision to capture task preferences. Practitioners may care about this approach because it can lead to better performance on open-ended tasks.