Firehose

This paper proposes a method to combine reinforcement learning with verifiable rewards and on-policy distillation to improve performance on complex tasks, and shows that this method can lead to more stable training and better results.

This paper proposes a new method for training large language models in open-ended domains, using evolving contexts as in-training supervision to capture task preferences. Practitioners may care about this approach because it can lead to better performance on open-ended tasks.