Papers

Filtered to multi-head attention · clear filter

Browse by term

continual learning 64reinforcement learning 34large language models 17benchmarking 12vision-language models 10generative models 8language models 8video generation 7multimodal models 6natural language processing 6robotics 6world models 6benchmarks 5diffusion models 5on-policy distillation 5policy optimization 5scalability 5self-distillation 5vision-language-action models 5autoregressive models 4computer vision 4diffusion transformers 4LLMs 4multimodal large language models 4verifiable rewards 4attention mechanisms 3embodied intelligence 3image editing 3long-term memory 3multimodal learning 3

Matching papers

Multi-Head Attention Residuals

8 upvotes · 22 JUL 2026 · Cheng Luo, Zefan Cai, Junjie Hu

This paper improves the Transformer model by introducing Multi-Head Attention Residuals, which allows each sublayer to attend to different parts of the model's history, reducing the impact of disagreements between different feature subspaces. Practitioners might care because this could lead to better performance in large models.

transformersattention mechanismsresidual connectionsmulti-head attentionlarge language models