Papers

Filtered to cross-tokenizer · clear filter

Browse by term

continual learning 91reinforcement learning 43vision-language models 18benchmarking 12benchmarks 11robotics 11large language models 10language models 9world models 9video generation 6generative models 5multimodal models 5natural language processing 5recursive self-improvement 5video world models 5agents 4dialogue systems 4diffusion models 4foundation models 4multimodal large language models 4on-policy distillation 4reasoning 4self-distillation 4self-supervised learning 4conversational AI 3evaluation benchmarks 3human-computer interaction 3LLMs 3multi-agent systems 3policy optimization 3

Matching papers

Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability

153 upvotes · 6 OCT 2026 · Bingxi Hou, Guochao Jiang, Guofeng Quan et al.

This paper investigates how to improve the effectiveness of On-Policy Distillation, a technique that trains a student model on its own generated data with teacher feedback. By prioritizing supervision reliability over alignment coverage, the researchers found that using a smaller subset of shared vocabulary at aligned positions can lead to comparable accuracy to using the full shared vocabulary.

continual learningon-policy distillationsupervision reliabilitycross-tokenizeralignment coverage