Papers

Filtered to attention mechanisms · clear filter

Browse by term

continual learning 64reinforcement learning 34large language models 17benchmarking 12vision-language models 10generative models 8language models 8video generation 7multimodal models 6natural language processing 6robotics 6world models 6benchmarks 5diffusion models 5on-policy distillation 5policy optimization 5scalability 5self-distillation 5vision-language-action models 5autoregressive models 4computer vision 4diffusion transformers 4LLMs 4multimodal large language models 4verifiable rewards 4attention mechanisms 3embodied intelligence 3image editing 3long-term memory 3multimodal learning 3

Matching papers

Metis: Memory Foundation Model

256 upvotes · 29 JUL 2026 · Zeyu Zhang, Ziliang Guo, Yihang Sun et al.

This paper introduces a new type of AI model called memory foundation models, which allows the model to learn and retain information internally, rather than relying on external memory modules. This could be useful for practitioners who want to build more efficient and flexible AI agents.

memory foundation modelsmultimodal modelsnative memorymodel compressionattention mechanismscontinual learning

Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers

68 upvotes · 21 JUL 2026 · Maohua Li, Qirui Li, Yanke Zhou et al.

This paper helps us understand how text-to-image diffusion transformers work by analyzing the role of "template tokens" in generating images from text prompts. Practitioners might care because it shows how to improve the efficiency of these models without sacrificing their performance.

diffusion transformerstext-to-image modelscausal interpretabilityattention mechanismssemantic routing

Multi-Head Attention Residuals

8 upvotes · 22 JUL 2026 · Cheng Luo, Zefan Cai, Junjie Hu

This paper improves the Transformer model by introducing Multi-Head Attention Residuals, which allows each sublayer to attend to different parts of the model's history, reducing the impact of disagreements between different feature subspaces. Practitioners might care because this could lead to better performance in large models.

transformersattention mechanismsresidual connectionsmulti-head attentionlarge language models