Firehose

This paper proposes a new attention mechanism for video diffusion transformers that reduces the computational cost of inference by selectively computing attention only on key/value blocks. It improves the accuracy of attention at high sparsity levels, leading to faster inference times and better video quality.