Firehose

This paper proposes a new approach to fine-tuning models, focusing on the direction of updates rather than the magnitude of changes, to improve performance without introducing behavioral drift. Practitioners can benefit from this method by optimizing their fine-tuning process to achieve better results while preserving existing capabilities.