Papers

Filtered to self-distillation · clear filter

Browse by term

continual learning 83reinforcement learning 50large language models 12benchmarking 11benchmarks 11language models 11vision-language models 11robotics 7natural language processing 6world models 6generative models 5recursive self-improvement 5attention mechanisms 4diffusion Transformers 4multi-agent systems 4multimodal learning 4multimodal models 4on-policy distillation 4self-distillation 4self-supervised learning 4transformers 4video generation 4vision-language-action models 4agent-based systems 3agentic models 3agentic search 3autonomous systems 3coding agents 3diffusion models 3image generation 3

Matching papers

On-Policy Self-Distillation without Any Supervision

183 upvotes · 9 AUG 2026 · Yijiang Li, Bingyang Wang, Yijun Liang et al.

This paper shows how to make large language models improve themselves without needing external guidance or supervision, by using their own internal consistency to correct mistakes. Practitioners might care about this because it could lead to more robust and self-sufficient AI models.

self-distillationon-policy methodsunsupervised learninginternal consistencyreinforcement learningcontinual learning

DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents

88 upvotes · 19 AUG 2026 · Hangrui Xu, Jiarui Wang, Yang Yang et al.

This paper proposes a new framework for training autonomous agents to perform multi-turn tool-calling tasks, addressing the challenge of dealing with vast solution spaces by using a diamond topology-aware approach. Practitioners may care about this research because it can help improve the performance of autonomous agents in complex tasks.

self-distillationcontinual learninglarge language modelsinteraction-state transition graphsdiamond topology

DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment

84 upvotes · 8 JUL 2026 · Xinyu Geng, Xuanhua He, Sixiang Chen et al.

This paper introduces a framework called DeepSearch-Evolve, which helps train self-improving web agents by iteratively refining their performance using their own experience. Practitioners might care because this approach can lead to more efficient and effective agents that can learn from their own mistakes.

web agentsself-distillationcontinual learningreinforcement learningverifiable environments

DAPD: Dual-Anchored Policy Distillation

51 upvotes · 3 AUG 2026 · Jianyu Wu, Yizhou Wang, Encheng Su et al.

This paper addresses a problem in self-distillation, where a student model learns to mimic the behavior of a privileged teacher, but performs poorly at inference due to a "privilege illusion". The authors propose a new method, Dual-Anchored Policy Distillation, to resolve this issue and improve performance.

self-distillationon-policy learningpolicy distillationprivilege illusioninformation asymmetryreinforcement learning