667 upvotes · 15 OCT 2026 · Bingxi Hou, Guochao Jiang, Guofeng Quan et al.
This paper develops a method to transfer policy shifts from a reinforcement learning (RL) teacher to a student model, using an intermediate teacher checkpoint to compare early and late policy shifts and gate the endpoint shift based on their agreement. Practitioners in RL and reinforcement learning may care about this paper because it proposes a more effective way to transfer policy shifts, which is crucial for improving student performance in various domains.
153 upvotes · 6 OCT 2026 · Bingxi Hou, Guochao Jiang, Guofeng Quan et al.
This paper investigates how to improve the effectiveness of On-Policy Distillation, a technique that trains a student model on its own generated data with teacher feedback. By prioritizing supervision reliability over alignment coverage, the researchers found that using a smaller subset of shared vocabulary at aligned positions can lead to comparable accuracy to using the full shared vocabulary.
96 upvotes · 17 SEP 2026 · Yuxiao Yang, Tianrun Yu, Shangzhe Li et al.
This paper investigates why some AI models can generate excessively long responses and proposes a solution to mitigate this issue by aligning the models' termination tokens. Practitioners might care about this problem because it can lead to wasted generation budgets and inefficient AI applications.
81 upvotes · 3 SEP 2026 · Zixuan Fu, Bingxiang He, Yuxin Zuo et al.
This paper investigates the role of training data in on-policy distillation, a technique used to improve large language models, and finds that even a single query can lead to significant improvements, but the process is slow and algorithm-starved.