Firehose

29 OCT 2026 · Paper

On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning

This paper investigates the problem of training-inference mismatch in reinforcement learning for large language models, where the training and inference processes can produce different results, leading to suboptimal training. A practitioner might care about this issue because it can lead to poor performance and instability in training, especially when using off-policy methods.

reinforcement learning large language models training-inference mismatch asynchronous sampling continual learning

Browse by tag

On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning