Filtered to Transformers · clear filter
This paper introduces a new framework for building strong image generators that can produce highly photorealistic images while accurately following editing instructions. Practitioners might care about the potential applications of this framework in fields like computer vision, graphics, and art.
This paper investigates how to make Looped Transformers more efficient by scaling their depth while keeping the same number of parameters and compute usage, and it finds that this approach can lead to significant performance gains. Practitioners might care about this because it could lead to faster and more accurate models in resource-constrained environments.
This paper introduces AURORA-LM, a continuous-latent diffusion language model that generates text by learning a high-capacity, decodable text latent and its distribution directly. Practitioners might care because it achieves state-of-the-art performance on text generation and summarization tasks.
This paper proposes a new method to sparsify attention in Transformers, called Simple Attention Sparsification (SAS), which optimizes context ranking end-to-end with the language modeling loss. Practitioners might care because SAS can improve performance on downstream tasks by using attention budgets more effectively.