Papers

Filtered to data-minimal limit · clear filter

Browse by term

continual learning 91reinforcement learning 43vision-language models 18benchmarking 12benchmarks 11robotics 11large language models 10language models 9world models 9video generation 6generative models 5multimodal models 5natural language processing 5recursive self-improvement 5video world models 5agents 4dialogue systems 4diffusion models 4foundation models 4multimodal large language models 4on-policy distillation 4reasoning 4self-distillation 4self-supervised learning 4conversational AI 3evaluation benchmarks 3human-computer interaction 3LLMs 3multi-agent systems 3policy optimization 3

Matching papers

Rethinking On-Policy Distillation of Large Language Models II: One Training Example

81 upvotes · 3 SEP 2026 · Zixuan Fu, Bingxiang He, Yuxin Zuo et al.

This paper investigates the role of training data in on-policy distillation, a technique used to improve large language models, and finds that even a single query can lead to significant improvements, but the process is slow and algorithm-starved.

on-policy distillationlarge language modelssingle-query trainingdata-minimal limitstate coverageteacher-student alignment