Papers

Filtered to efficient neural networks · clear filter

Browse by term

continual learning 91reinforcement learning 43vision-language models 18benchmarking 12benchmarks 11robotics 11large language models 10language models 9world models 9video generation 6generative models 5multimodal models 5natural language processing 5recursive self-improvement 5video world models 5agents 4dialogue systems 4diffusion models 4foundation models 4multimodal large language models 4on-policy distillation 4reasoning 4self-distillation 4self-supervised learning 4conversational AI 3evaluation benchmarks 3human-computer interaction 3LLMs 3multi-agent systems 3policy optimization 3

Matching papers

SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers

86 upvotes · 1 SEP 2026 · Shaowen Wang, Ge Zhang, Kairong Luo et al.

This paper investigates how to make Looped Transformers more efficient by scaling their depth while keeping the same number of parameters and compute usage, and it finds that this approach can lead to significant performance gains. Practitioners might care about this because it could lead to faster and more accurate models in resource-constrained environments.

Looped TransformersMixture-of-ExpertsTransformersscaling lawsefficient neural networksneural architecture search