Post-Training Leaves Behavioral Shadows on Unrelated Decisions
This paper discovers that language models can learn from unrelated text after being trained on a specific task, and introduces a method called Active Taskless Distillation (ATD) that allows this capability transfer to happen without using target-task examples or teacher parameters. Practitioners might care because it could lead to more efficient and effective language model training.