Firehose

Filtered to Papers, tagged “value estimation” · clear filters

Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives

All People Companies Papers Podcasts Hacker News

Browse by tag

continual learning 26 reinforcement learning 9 language models 5 large language models 4 robotics 4 agents 3 generative models 3 multi-agent systems 3 recursive self-improvement 3 vision-language models 3

16 SEP 2026 · Paper

Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flattening

This paper investigates a common problem in reinforcement learning for language models called Value Flattening, where critics fail to accurately estimate state values, and proposes a new method, SP^3O, to mitigate this issue by supervising only a few well-separated states per response.

reinforcement learning PPO value estimation state space sparse supervision critic learning