Firehose

This paper improves a common technique in language model training called Multi-Token Prediction (MTP) by removing a noisy component that interferes with learning, resulting in faster and more accurate predictions. Practitioners may care because this improvement can lead to better performance in applications such as language translation and text generation.